[PATCH 00/17] media: ov5640: Rework the clock tree programming for MIPI

[PATCH 00/17] media: ov5640: Rework the clock tree programming for MIPI

[Jacopo Mondi]

Heya
   this series changes the way the ov5640 clock tree is programmed and moves the
driver to control frame rate by changing the vertical blanking when used in MIPI
mode.

A bit of background:
The OV5640 is -everywhere-. After a long long time since it has been put on the
market it still is one of the most common sensors found in EVKs from several
several vendors. Touching it is scary, it will likely make a lot of people
complain as there are a lot of active users out there.

The current driver implementation as it is barely works at least here. Testing
in UYVY mode I can only capture in 1080p from the few tests I've run. Other
resolutions do not work or have a completely off frame rate.

The current implementation operates in the following way:

- it allows to control frame rate through the s_frame_interval operation
- fixes a set of frame rates supported by each mode (15, 30 or 60FPS)
- tries to adjust the pixel rate to comply with the configured frame rate
  and capture resolution

The idea is then to fix the frame size (visible and blanking) and compute
the pixel rate dynamically and adjust the clock tree accordingly.

As it was hard to obtain correct operations in that way, and that frame rates
were fixed to only a few of the achievable ones I have decided to flip the table
and move to an opposite model:

- Set a fixed pixel clock per each supported mode according to the indications
  reported in the chip manual
- Program the clock tree according to the above pixel rate
- Adjust the frame rate by controlling the frame vertical blankings

With this new configuration I can capture in all the sensor supported modes in
YUYV, RGB565, RGB888 and SBGGR modes with the following limitations:

- 16bpp modes works in all resolutions
- RAW 8bpp mode works for high resolutions from 1280x720 up
- RGB888 mode works for low resolutions up to 1280x720

The blankings for each mode have been adjusted to work by default at 30FPS
in YUYV/RGB565 mode, and can be adjusted to precisely obtain the desired frame
rate. Unfortunately the minimum blanking values are not documented so shrinking
them too much might hinder capture operations.

I tested the series on both i.MX6Q and i.MX8MP and I get the same results on
both platforms.

There remain two issues:

- RGB888 colors are off, both on imx6 and imx8. I tested all the RGB color
  permutations the chip can perform and I am not able to get the color
  ordering "right". It might be due to how I perform decoding or to a bug in
  the sensor manual.

- I was not able to test JPEG, which I would like to support next. If anyone
  has been able to test JPEG capture in the past and would like to share it's
  setup I would be very glad.

Are we there yet ? No, I don't think so. My main concern at the moment is that
this sensor supports -a lot- of different formats, and the pixel rates I
have fixed are compromises to a have all formats working. In example, for
640x480 I could have gone for 96MHz instead of 48. This would allow to
push the sensor up to 60FPS per second in YUYV/RGB565 mode, but would break
capturing in RGB888 mode, for which 96Mhz @24bpp is too high as a pixel rate.
-Ideally-  userspace should be able to select the pixel rate which is
ideal for the mode currently in use. The PIXEL_RATE control is however marked
as READ_ONLY by the framework, and the only alternative I'm left with is to
create per-mode pixel rates but this would make the driver a lot mode
complicated... Ideas ?

The series changes the clock tree programming for MIPI only. I have not tried
to maintain the current behaviour for parallel mode which, as far as I'm aware,
works correctly. As I cannot test parallel, I would really appreciate a
confirmation I have not broken parallel mode :)

Ah, v4l2-compliance seems happy too:
Total for device /dev/v4l-subdev3: 43, Succeeded: 43, Failed: 0, Warnings: 0

Thanks
   j

Jacopo Mondi (21):
  media: ov5640: Add pixel rate to modes
  media: ov5604: Re-arrange modes definition
  media: ov5640: Add is_mipi() function
  media: ov5640: Associate bpp with formats
  media: ov5640: Update pixel_rate and link_freq
  media: ov5640: Rework CSI-2 clock tree
  media: ov5640: Rework timings programming
  media: ov5640: Re-sort per-mode register tables
  media: ov5640: Remove ov5640_mode_init_data
  media: ov5640: Add HBLANK control
  media: ov5640: Add VBLANK control
  media: ov5640: Fix durations to comply with FPS
  media: ov5640: Initialize try format
  media: ov5640: Implement get_selection
  media: ov5640: Limit format to FPS in DVP mode only
  media: ov5640: Disable s_frame_interval in MIPI mode
  media: ov5640: Register device properties
  media: ov5640: Add RGB565_1X16 format
  media: ov5640: Add RGB888/BGR888 formats
  media: ov5640: Restrict sizes to mbus code
  media: ov5640: Adjust format to bpp in s_fmt

 drivers/media/i2c/ov5640.c | 1063 ++++++++++++++++++++++++++----------
 1 file changed, 778 insertions(+), 285 deletions(-)

--
2.35.0

[PATCH 01/21] media: ov5640: Add pixel rate to modes

[Jacopo Mondi]

Add to each mode supported by the sensor the ideal pixel rate, as
defined by Table 2.1 in the chip manual.

The ideal pixel rate will be used to compute the MIPI CSI-2 clock tree.

Create the V4L2_CID_LINK_FREQ control and define the enumeration
of possible MIPI CSI-2 link frequencies from the list of supported
pixel clocks.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 66 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 63 insertions(+), 3 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index db5a19babe67..d915c9652302 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -118,6 +118,44 @@ enum ov5640_frame_rate {
 	OV5640_NUM_FRAMERATES,
 };
 
+enum ov5640_pixel_rate_id {
+	OV5640_PIXEL_RATE_168M,
+	OV5640_PIXEL_RATE_148M,
+	OV5640_PIXEL_RATE_124M,
+	OV5640_PIXEL_RATE_96M,
+	OV5640_PIXEL_RATE_48M,
+	OV5640_NUM_PIXEL_RATES,
+};
+
+/*
+ * The chip manual suggests 24/48/96/192 MHz pixel clocks.
+ *
+ * 192MHz exceeds the sysclk limits; use 168MHz as maximum pixel rate for
+ * full resolution mode @15 FPS.
+ */
+static const u32 ov5640_pixel_rates[] = {
+	[OV5640_PIXEL_RATE_168M] = 168000000,
+	[OV5640_PIXEL_RATE_148M] = 148000000,
+	[OV5640_PIXEL_RATE_124M] = 124000000,
+	[OV5640_PIXEL_RATE_96M] = 96000000,
+	[OV5640_PIXEL_RATE_48M] = 48000000,
+};
+
+/*
+ * MIPI CSI-2 link frequencies.
+ *
+ * Derived from the above defined pixel rate for bpp = (8, 16, 24) and
+ * data_lanes = (1, 2)
+ *
+ * link_freq = (pixel_rate * bpp) / (2 * data_lanes)
+ */
+static const s64 ov5640_csi2_link_freqs[] = {
+	992000000, 888000000, 768000000, 744000000, 672000000, 672000000,
+	592000000, 592000000, 576000000, 576000000, 496000000, 496000000,
+	384000000, 384000000, 384000000, 336000000, 296000000, 288000000,
+	248000000, 192000000, 192000000, 192000000, 96000000,
+};
+
 enum ov5640_format_mux {
 	OV5640_FMT_MUX_YUV422 = 0,
 	OV5640_FMT_MUX_RGB,
@@ -189,6 +227,7 @@ struct reg_value {
 struct ov5640_mode_info {
 	enum ov5640_mode_id id;
 	enum ov5640_downsize_mode dn_mode;
+	enum ov5640_pixel_rate_id pixel_rate;
 	u32 hact;
 	u32 htot;
 	u32 vact;
@@ -201,6 +240,7 @@ struct ov5640_mode_info {
 struct ov5640_ctrls {
 	struct v4l2_ctrl_handler handler;
 	struct v4l2_ctrl *pixel_rate;
+	struct v4l2_ctrl *link_freq;
 	struct {
 		struct v4l2_ctrl *auto_exp;
 		struct v4l2_ctrl *exposure;
@@ -565,7 +605,9 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
 
 /* power-on sensor init reg table */
 static const struct ov5640_mode_info ov5640_mode_init_data = {
-	0, SUBSAMPLING, 640, 1896, 480, 984,
+	0, SUBSAMPLING,
+	OV5640_PIXEL_RATE_96M,
+	640, 1896, 480, 984,
 	ov5640_init_setting_30fps_VGA,
 	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
 	OV5640_30_FPS,
@@ -574,51 +616,61 @@ static const struct ov5640_mode_info ov5640_mode_init_data = {
 static const struct ov5640_mode_info
 ov5640_mode_data[OV5640_NUM_MODES] = {
 	{OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_48M,
 	 160, 1896, 120, 984,
 	 ov5640_setting_QQVGA_160_120,
 	 ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
 	 OV5640_30_FPS},
 	{OV5640_MODE_QCIF_176_144, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_48M,
 	 176, 1896, 144, 984,
 	 ov5640_setting_QCIF_176_144,
 	 ARRAY_SIZE(ov5640_setting_QCIF_176_144),
 	 OV5640_30_FPS},
 	{OV5640_MODE_QVGA_320_240, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_48M,
 	 320, 1896, 240, 984,
 	 ov5640_setting_QVGA_320_240,
 	 ARRAY_SIZE(ov5640_setting_QVGA_320_240),
 	 OV5640_30_FPS},
 	{OV5640_MODE_VGA_640_480, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_48M,
 	 640, 1896, 480, 1080,
 	 ov5640_setting_VGA_640_480,
 	 ARRAY_SIZE(ov5640_setting_VGA_640_480),
 	 OV5640_60_FPS},
 	{OV5640_MODE_NTSC_720_480, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_96M,
 	 720, 1896, 480, 984,
 	 ov5640_setting_NTSC_720_480,
 	 ARRAY_SIZE(ov5640_setting_NTSC_720_480),
 	OV5640_30_FPS},
 	{OV5640_MODE_PAL_720_576, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_96M,
 	 720, 1896, 576, 984,
 	 ov5640_setting_PAL_720_576,
 	 ARRAY_SIZE(ov5640_setting_PAL_720_576),
 	 OV5640_30_FPS},
 	{OV5640_MODE_XGA_1024_768, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_96M,
 	 1024, 1896, 768, 1080,
 	 ov5640_setting_XGA_1024_768,
 	 ARRAY_SIZE(ov5640_setting_XGA_1024_768),
 	 OV5640_30_FPS},
 	{OV5640_MODE_720P_1280_720, SUBSAMPLING,
+	 OV5640_PIXEL_RATE_124M,
 	 1280, 1892, 720, 740,
 	 ov5640_setting_720P_1280_720,
 	 ARRAY_SIZE(ov5640_setting_720P_1280_720),
 	 OV5640_30_FPS},
 	{OV5640_MODE_1080P_1920_1080, SCALING,
+	 OV5640_PIXEL_RATE_148M,
 	 1920, 2500, 1080, 1120,
 	 ov5640_setting_1080P_1920_1080,
 	 ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
 	 OV5640_30_FPS},
 	{OV5640_MODE_QSXGA_2592_1944, SCALING,
+	 OV5640_PIXEL_RATE_168M,
 	 2592, 2844, 1944, 1968,
 	 ov5640_setting_QSXGA_2592_1944,
 	 ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
@@ -2743,6 +2795,7 @@ static const struct v4l2_ctrl_ops ov5640_ctrl_ops = {
 
 static int ov5640_init_controls(struct ov5640_dev *sensor)
 {
+	const struct ov5640_mode_info *mode = sensor->current_mode;
 	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
 	struct ov5640_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
@@ -2755,8 +2808,14 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 
 	/* Clock related controls */
 	ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_PIXEL_RATE,
-					      0, INT_MAX, 1,
-					      ov5640_calc_pixel_rate(sensor));
+				      ov5640_pixel_rates[OV5640_NUM_PIXEL_RATES - 1],
+				      ov5640_pixel_rates[0], 1,
+				      ov5640_pixel_rates[mode->pixel_rate]);
+
+	ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, ops,
+					V4L2_CID_LINK_FREQ,
+					ARRAY_SIZE(ov5640_csi2_link_freqs) - 1,
+					4, ov5640_csi2_link_freqs);
 
 	/* Auto/manual white balance */
 	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
@@ -2806,6 +2865,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 	}
 
 	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 	ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
 	ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
 
-- 
2.35.0

[PATCH 02/21] media: ov5604: Re-arrange modes definition

[Jacopo Mondi]

The array of supported modes is close to unreadable.
Re-arrange it giving it some room to breath.

Cosmetic change only.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 141 +++++++++++++++++++++----------------
 1 file changed, 81 insertions(+), 60 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index d915c9652302..7e7732f30486 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -615,66 +615,87 @@ static const struct ov5640_mode_info ov5640_mode_init_data = {
 
 static const struct ov5640_mode_info
 ov5640_mode_data[OV5640_NUM_MODES] = {
-	{OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_48M,
-	 160, 1896, 120, 984,
-	 ov5640_setting_QQVGA_160_120,
-	 ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
-	 OV5640_30_FPS},
-	{OV5640_MODE_QCIF_176_144, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_48M,
-	 176, 1896, 144, 984,
-	 ov5640_setting_QCIF_176_144,
-	 ARRAY_SIZE(ov5640_setting_QCIF_176_144),
-	 OV5640_30_FPS},
-	{OV5640_MODE_QVGA_320_240, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_48M,
-	 320, 1896, 240, 984,
-	 ov5640_setting_QVGA_320_240,
-	 ARRAY_SIZE(ov5640_setting_QVGA_320_240),
-	 OV5640_30_FPS},
-	{OV5640_MODE_VGA_640_480, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_48M,
-	 640, 1896, 480, 1080,
-	 ov5640_setting_VGA_640_480,
-	 ARRAY_SIZE(ov5640_setting_VGA_640_480),
-	 OV5640_60_FPS},
-	{OV5640_MODE_NTSC_720_480, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_96M,
-	 720, 1896, 480, 984,
-	 ov5640_setting_NTSC_720_480,
-	 ARRAY_SIZE(ov5640_setting_NTSC_720_480),
-	OV5640_30_FPS},
-	{OV5640_MODE_PAL_720_576, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_96M,
-	 720, 1896, 576, 984,
-	 ov5640_setting_PAL_720_576,
-	 ARRAY_SIZE(ov5640_setting_PAL_720_576),
-	 OV5640_30_FPS},
-	{OV5640_MODE_XGA_1024_768, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_96M,
-	 1024, 1896, 768, 1080,
-	 ov5640_setting_XGA_1024_768,
-	 ARRAY_SIZE(ov5640_setting_XGA_1024_768),
-	 OV5640_30_FPS},
-	{OV5640_MODE_720P_1280_720, SUBSAMPLING,
-	 OV5640_PIXEL_RATE_124M,
-	 1280, 1892, 720, 740,
-	 ov5640_setting_720P_1280_720,
-	 ARRAY_SIZE(ov5640_setting_720P_1280_720),
-	 OV5640_30_FPS},
-	{OV5640_MODE_1080P_1920_1080, SCALING,
-	 OV5640_PIXEL_RATE_148M,
-	 1920, 2500, 1080, 1120,
-	 ov5640_setting_1080P_1920_1080,
-	 ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
-	 OV5640_30_FPS},
-	{OV5640_MODE_QSXGA_2592_1944, SCALING,
-	 OV5640_PIXEL_RATE_168M,
-	 2592, 2844, 1944, 1968,
-	 ov5640_setting_QSXGA_2592_1944,
-	 ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
-	 OV5640_15_FPS},
+	{
+		/* 160x120 */
+		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
+		OV5640_PIXEL_RATE_48M,
+		160, 1896, 120, 984,
+		ov5640_setting_QQVGA_160_120,
+		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
+		OV5640_30_FPS
+	}, {
+		/* 176x144 */
+		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
+		OV5640_PIXEL_RATE_48M,
+		176, 1896, 144, 984,
+		ov5640_setting_QCIF_176_144,
+		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
+		OV5640_30_FPS
+	}, {
+		/* 320x240 */
+		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
+		OV5640_PIXEL_RATE_48M,
+		320, 1896, 240, 984,
+		ov5640_setting_QVGA_320_240,
+		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
+		OV5640_30_FPS
+	}, {
+		/* 640x480 */
+		OV5640_MODE_VGA_640_480, SUBSAMPLING,
+		OV5640_PIXEL_RATE_48M,
+		640, 1896, 480, 1080,
+		ov5640_setting_VGA_640_480,
+		ARRAY_SIZE(ov5640_setting_VGA_640_480),
+		OV5640_60_FPS
+	}, {
+		/* 720x480 */
+		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
+		OV5640_PIXEL_RATE_96M,
+		720, 1896, 480, 984,
+		ov5640_setting_NTSC_720_480,
+		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
+		OV5640_30_FPS
+	}, {
+		/* 720x576 */
+		OV5640_MODE_PAL_720_576, SUBSAMPLING,
+		OV5640_PIXEL_RATE_96M,
+		720, 1896, 576, 984,
+		ov5640_setting_PAL_720_576,
+		ARRAY_SIZE(ov5640_setting_PAL_720_576),
+		OV5640_30_FPS
+	}, {
+		/* 1024x768 */
+		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
+		OV5640_PIXEL_RATE_96M,
+		1024, 1896, 768, 1080,
+		ov5640_setting_XGA_1024_768,
+		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
+		OV5640_30_FPS
+	}, {
+		/* 1280x720 */
+		OV5640_MODE_720P_1280_720, SUBSAMPLING,
+		OV5640_PIXEL_RATE_124M,
+		1280, 1892, 720, 740,
+		ov5640_setting_720P_1280_720,
+		ARRAY_SIZE(ov5640_setting_720P_1280_720),
+		OV5640_30_FPS
+	}, {
+		/* 1920x1080 */
+		OV5640_MODE_1080P_1920_1080, SCALING,
+		OV5640_PIXEL_RATE_148M,
+		1920, 2500, 1080, 1120,
+		ov5640_setting_1080P_1920_1080,
+		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
+		OV5640_30_FPS
+	}, {
+		/* 2592x1944 */
+		OV5640_MODE_QSXGA_2592_1944, SCALING,
+		OV5640_PIXEL_RATE_168M,
+		2592, 2844, 1944, 1968,
+		ov5640_setting_QSXGA_2592_1944,
+		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
+		OV5640_15_FPS
+	},
 };
 
 static int ov5640_init_slave_id(struct ov5640_dev *sensor)
-- 
2.35.0

[PATCH 03/21] media: ov5640: Add is_mipi() function

[Jacopo Mondi]

Checking if the sensor is used in DVP or MIPI mode is a repeated
pattern which is about to be repeated more often.

Provide an inline function to shortcut that.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 7e7732f30486..fc3e4f61709c 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -310,6 +310,11 @@ static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
 			     ctrls.handler)->sd;
 }
 
+static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
+{
+	return sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY;
+}
+
 /*
  * FIXME: all of these register tables are likely filled with
  * entries that set the register to their power-on default values,
@@ -1224,7 +1229,7 @@ static int ov5640_load_regs(struct ov5640_dev *sensor,
 		/* remain in power down mode for DVP */
 		if (regs->reg_addr == OV5640_REG_SYS_CTRL0 &&
 		    val == OV5640_REG_SYS_CTRL0_SW_PWUP &&
-		    sensor->ep.bus_type != V4L2_MBUS_CSI2_DPHY)
+		    !ov5640_is_mipi(sensor))
 			continue;
 
 		if (mask)
@@ -1859,7 +1864,7 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
 	 * the same rate than YUV, so we can just use 16 bpp all the time.
 	 */
 	rate = ov5640_calc_pixel_rate(sensor) * 16;
-	if (sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY) {
+	if (ov5640_is_mipi(sensor)) {
 		rate = rate / sensor->ep.bus.mipi_csi2.num_data_lanes;
 		ret = ov5640_set_mipi_pclk(sensor, rate);
 	} else {
@@ -3042,7 +3047,7 @@ static int ov5640_s_stream(struct v4l2_subdev *sd, int enable)
 			sensor->pending_fmt_change = false;
 		}
 
-		if (sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY)
+		if (ov5640_is_mipi(sensor))
 			ret = ov5640_set_stream_mipi(sensor, enable);
 		else
 			ret = ov5640_set_stream_dvp(sensor, enable);
-- 
2.35.0

[PATCH 04/21] media: ov5640: Associate bpp with formats

[Jacopo Mondi]

Associate the bit depth to each format supported by the sensor.

The bpp will be used to calculate the line length.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 39 ++++++++++++++++++++++++--------------
 1 file changed, 25 insertions(+), 14 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index fc3e4f61709c..8322b99eb2b7 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -165,24 +165,35 @@ enum ov5640_format_mux {
 	OV5640_FMT_MUX_RAW_CIP,
 };
 
-struct ov5640_pixfmt {
+static const struct ov5640_pixfmt {
 	u32 code;
 	u32 colorspace;
+	u8 bpp;
+} ov5640_formats[] = {
+	{ MEDIA_BUS_FMT_JPEG_1X8, V4L2_COLORSPACE_JPEG, 16},
+	{ MEDIA_BUS_FMT_UYVY8_2X8, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_UYVY8_1X16, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_YUYV8_2X8, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, 16,},
+	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
+	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
+	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
+	{ MEDIA_BUS_FMT_SRGGB8_1X8, V4L2_COLORSPACE_SRGB, 8},
 };
 
-static const struct ov5640_pixfmt ov5640_formats[] = {
-	{ MEDIA_BUS_FMT_JPEG_1X8, V4L2_COLORSPACE_JPEG, },
-	{ MEDIA_BUS_FMT_UYVY8_2X8, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_UYVY8_1X16, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_YUYV8_2X8, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, },
-	{ MEDIA_BUS_FMT_SRGGB8_1X8, V4L2_COLORSPACE_SRGB, },
-};
+static u32 ov5640_code_to_bpp(u32 code)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(ov5640_formats); ++i) {
+		if (ov5640_formats[i].code == code)
+			return ov5640_formats[i].bpp;
+	}
+
+	return 0;
+}
 
 /*
  * FIXME: remove this when a subdev API becomes available
-- 
2.35.0

[PATCH 05/21] media: ov5640: Update pixel_rate and link_freq

[Jacopo Mondi]

After having set a new format re-calculate the pixel_rate and link_freq
control values and update them when in MIPI mode.

Take into account the limitation of the link frequency having to be
strictly smaller than 1GHz when computing the desired link_freq, and
adjust the resulting pixel_rate acounting for the clock tree
configuration.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 63 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 61 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 8322b99eb2b7..457f76030163 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -2375,6 +2375,65 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
+{
+	const struct ov5640_mode_info *mode = sensor->current_mode;
+	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
+	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
+	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
+	unsigned int i = 0;
+	u32 pixel_rate;
+	s64 link_freq;
+	u32 bpp;
+
+	/*
+	 * Update the pixel rate control value.
+	 *
+	 * For DVP mode, maintain the pixel rate calculation using fixed FPS.
+	 */
+	if (!ov5640_is_mipi(sensor)) {
+		__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate,
+					 ov5640_calc_pixel_rate(sensor));
+
+		return 0;
+	}
+
+	/*
+	 * The MIPI CSI-2 link frequency should comply with the CSI-2
+	 * specifications and be lower than 1GHz.
+	 *
+	 * Start from the suggested pixel_rate for the current mode and
+	 * progressively slow it down if it exceeds 1GHz.
+	 */
+	bpp = ov5640_code_to_bpp(fmt->code);
+	do {
+		pixel_rate = ov5640_pixel_rates[pixel_rate_id];
+		link_freq = pixel_rate * bpp / (2 * num_lanes);
+	} while (link_freq >= 1000000000U &&
+		 ++pixel_rate_id < OV5640_NUM_PIXEL_RATES);
+
+	/*
+	 * Higher link rates require the clock tree to be programmed with
+	 * 'mipi_div' = 1; this has the effect of halving the actual output
+	 * pixel rate in the MIPI domain.
+	 *
+	 * Adjust the pixel rate control value to report it correctly to
+	 * userspace.
+	 */
+	if (link_freq > OV5640_LINK_RATE_MAX)
+		pixel_rate /= 2;
+
+	for (i = 0; i < ARRAY_SIZE(ov5640_csi2_link_freqs); ++i) {
+		if (ov5640_csi2_link_freqs[i] == link_freq)
+			break;
+	}
+
+	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate, pixel_rate);
+	__v4l2_ctrl_s_ctrl(sensor->ctrls.link_freq, i);
+
+	return 0;
+}
+
 static int ov5640_set_fmt(struct v4l2_subdev *sd,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
@@ -2414,8 +2473,8 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
 	/* update format even if code is unchanged, resolution might change */
 	sensor->fmt = *mbus_fmt;
 
-	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate,
-				 ov5640_calc_pixel_rate(sensor));
+	ov5640_update_pixel_rate(sensor);
+
 out:
 	mutex_unlock(&sensor->lock);
 	return ret;
-- 
2.35.0

[PATCH 06/21] media: ov5640: Rework CSI-2 clock tree

[Jacopo Mondi]

Re-work the ov5640_set_mipi_pclk() function to calculate the
PLL configuration using the pixel_rate and link_freq values set at
s_fmt time.

Rework the DVP clock mode settings to calculate the pixel clock
internally and remove the assumption on the 16bpp format.

Tested in MIPI CSI-2 mode with 2 data lanes with:
- all the sensor supported resolutions in UYVY and RGB565 formats.
- resolutions >= 1280x720 in RAW Bayer format.
- resolutions < 1280x720 in RGB888 format.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>

---

Not tested with JPEG.
Not tested with 1 data lane.
Not tested in DVP mode.

If I get come Tested-by: tags for the above use cases, it would be
great.
---
 drivers/media/i2c/ov5640.c | 175 ++++++++++++++++++++-----------------
 1 file changed, 94 insertions(+), 81 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 457f76030163..acc636500907 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -29,6 +29,8 @@
 #define OV5640_XCLK_MIN  6000000
 #define OV5640_XCLK_MAX 54000000
 
+#define OV5640_LINK_RATE_MAX	490000000U
+
 #define OV5640_DEFAULT_SLAVE_ID 0x3c
 
 #define OV5640_REG_SYS_RESET02		0x3002
@@ -88,6 +90,7 @@
 #define OV5640_REG_POLARITY_CTRL00	0x4740
 #define OV5640_REG_MIPI_CTRL00		0x4800
 #define OV5640_REG_DEBUG_MODE		0x4814
+#define OV5640_REG_PCLK_PERIOD		0x4837
 #define OV5640_REG_ISP_FORMAT_MUX_CTRL	0x501f
 #define OV5640_REG_PRE_ISP_TEST_SET1	0x503d
 #define OV5640_REG_SDE_CTRL0		0x5580
@@ -1035,69 +1038,80 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor,
  * ov5640_set_mipi_pclk() - Calculate the clock tree configuration values
  *			    for the MIPI CSI-2 output.
  *
- * @rate: The requested bandwidth per lane in bytes per second.
- *	  'Bandwidth Per Lane' is calculated as:
- *	  bpl = HTOT * VTOT * FPS * bpp / num_lanes;
- *
- * This function use the requested bandwidth to calculate:
- * - sample_rate = bpl / (bpp / num_lanes);
- *	         = bpl / (PLL_RDIV * BIT_DIV * PCLK_DIV * MIPI_DIV / num_lanes);
- *
- * - mipi_sclk   = bpl / MIPI_DIV / 2; ( / 2 is for CSI-2 DDR)
- *
- * with these fixed parameters:
- *	PLL_RDIV	= 2;
- *	BIT_DIVIDER	= 2; (MIPI_BIT_MODE == 8 ? 2 : 2,5);
- *	PCLK_DIV	= 1;
- *
- * The MIPI clock generation differs for modes that use the scaler and modes
- * that do not. In case the scaler is in use, the MIPI_SCLK generates the MIPI
- * BIT CLk, and thus:
- *
- * - mipi_sclk = bpl / MIPI_DIV / 2;
- *   MIPI_DIV = 1;
- *
- * For modes that do not go through the scaler, the MIPI BIT CLOCK is generated
- * from the pixel clock, and thus:
- *
- * - sample_rate = bpl / (bpp / num_lanes);
- *	         = bpl / (2 * 2 * 1 * MIPI_DIV / num_lanes);
- *		 = bpl / (4 * MIPI_DIV / num_lanes);
- * - MIPI_DIV	 = bpp / (4 * num_lanes);
- *
- * FIXME: this have been tested with 16bpp and 2 lanes setup only.
- * MIPI_DIV is fixed to value 2, but it -might- be changed according to the
- * above formula for setups with 1 lane or image formats with different bpp.
- *
- * FIXME: this deviates from the sensor manual documentation which is quite
- * thin on the MIPI clock tree generation part.
+ * FIXME: tested with 2 lanes only.
  */
-static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
-				unsigned long rate)
+static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor)
 {
-	const struct ov5640_mode_info *mode = sensor->current_mode;
+	u8 bit_div, mipi_div, pclk_div, sclk_div, sclk2x_div, root_div;
+	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
 	u8 prediv, mult, sysdiv;
-	u8 mipi_div;
+	unsigned long sysclk;
+	unsigned long sample_rate;
+	u8 pclk_period;
+	s64 link_freq;
 	int ret;
 
+	/* Use the link frequency computed at s_fmt time. */
+	link_freq = ov5640_csi2_link_freqs[sensor->ctrls.link_freq->cur.val];
+
 	/*
-	 * 1280x720 is reported to use 'SUBSAMPLING' only,
-	 * but according to the sensor manual it goes through the
-	 * scaler before subsampling.
+	 * - mipi_div - Additional divider for the MIPI lane clock.
+	 *
+	 * Higher link frequencies would make sysclk > 1GHz.
+	 * Keep the sysclk low and do not divide in the MIPI domain.
 	 */
-	if (mode->dn_mode == SCALING ||
-	   (mode->id == OV5640_MODE_720P_1280_720))
-		mipi_div = OV5640_MIPI_DIV_SCLK;
+	if (link_freq > OV5640_LINK_RATE_MAX)
+		mipi_div = 1;
 	else
-		mipi_div = OV5640_MIPI_DIV_PCLK;
+		mipi_div = 2;
 
-	ov5640_calc_sys_clk(sensor, rate, &prediv, &mult, &sysdiv);
+	sysclk = link_freq * mipi_div;
+	ov5640_calc_sys_clk(sensor, sysclk, &prediv, &mult, &sysdiv);
 
-	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0,
-			     0x0f, OV5640_PLL_CTRL0_MIPI_MODE_8BIT);
+	/*
+	 * Adjust PLL parameters to maintain the MIPI_SCLK-to-PCLK ratio;
+	 *
+	 * - root_div = 2 (fixed)
+	 * - bit_div : MIPI 8-bit = 2
+	 *	       MIPI 10-bit = 2,5
+	 * - pclk_div = 1 (fixed)
+	 * - pll_div  = (2 lanes ? mipi_div : 2 * mipi_div)
+	 *   2 lanes: MIPI_SCLK = (4 or 5) * PCLK
+	 *   1 lanes: MIPI_SCLK = (8 or 10) * PCLK
+	 *
+	 * TODO: support 10-bit formats
+	 * TODO: test with 1 data lane
+	 */
+	root_div = OV5640_PLL_CTRL3_PLL_ROOT_DIV_2;
+	bit_div =  OV5640_PLL_CTRL0_MIPI_MODE_8BIT;
+	pclk_div = OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS;
 
-	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1,
-			     0xff, sysdiv << 4 | mipi_div);
+	/*
+	 * Scaler clock:
+	 * - YUV: PCLK >= 2 * SCLK
+	 * - RAW or JPEG: PCLK >= SCLK
+	 * - sclk2x_div = sclk_div / 2
+	 *
+	 * TODO: test with JPEG.
+	 */
+	sclk_div = ilog2(OV5640_SCLK_ROOT_DIV);
+	sclk2x_div = ilog2(OV5640_SCLK2X_ROOT_DIV);
+
+	/*
+	 * Set the sample period expressed in ns with 1-bit decimal
+	 * (0x01=0.5ns).
+	 */
+	sample_rate = ov5640_pixel_rates[sensor->current_mode->pixel_rate]
+		    * (ov5640_code_to_bpp(fmt->code) / 8);
+	pclk_period = 2000000000U / sample_rate;
+
+	/* Program the clock tree registers. */
+	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0, 0x0f, bit_div);
+	if (ret)
+		return ret;
+
+	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1, 0xff,
+			     sysdiv << 4 | mipi_div);
 	if (ret)
 		return ret;
 
@@ -1105,13 +1119,27 @@ static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
 	if (ret)
 		return ret;
 
-	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3,
-			     0x1f, OV5640_PLL_CTRL3_PLL_ROOT_DIV_2 | prediv);
+	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3, 0x1f,
+			     root_div | prediv);
 	if (ret)
 		return ret;
 
-	return ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER,
-			      0x30, OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS);
+	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
+			     (pclk_div << 4) | (sclk2x_div << 2) | sclk_div);
+	if (ret)
+		return ret;
+
+	return ov5640_write_reg(sensor, OV5640_REG_PCLK_PERIOD, pclk_period);
+}
+
+static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
+{
+	u64 rate;
+
+	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
+	rate *= ov5640_framerates[sensor->current_fr];
+
+	return rate;
 }
 
 static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
@@ -1131,11 +1159,16 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
 	return _rate / *pll_rdiv / *bit_div / *pclk_div;
 }
 
-static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate)
+static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor)
 {
 	u8 prediv, mult, sysdiv, pll_rdiv, bit_div, pclk_div;
+	u64 rate;
 	int ret;
 
+	rate = ov5640_calc_pixel_rate(sensor);
+	rate *= ov5640_code_to_bpp(sensor->fmt.code);
+	do_div(rate, sensor->ep.bus.parallel.bus_width);
+
 	ov5640_calc_pclk(sensor, rate, &prediv, &mult, &sysdiv, &pll_rdiv,
 			 &bit_div, &pclk_div);
 
@@ -1660,16 +1693,6 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
 	return mode;
 }
 
-static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
-{
-	u64 rate;
-
-	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
-	rate *= ov5640_framerates[sensor->current_fr];
-
-	return rate;
-}
-
 /*
  * sensor changes between scaling and subsampling, go through
  * exposure calculation
@@ -1851,7 +1874,6 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
 	enum ov5640_downsize_mode dn_mode, orig_dn_mode;
 	bool auto_gain = sensor->ctrls.auto_gain->val == 1;
 	bool auto_exp =  sensor->ctrls.auto_exp->val == V4L2_EXPOSURE_AUTO;
-	unsigned long rate;
 	int ret;
 
 	dn_mode = mode->dn_mode;
@@ -1870,19 +1892,10 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
 			goto restore_auto_gain;
 	}
 
-	/*
-	 * All the formats we support have 16 bits per pixel, seems to require
-	 * the same rate than YUV, so we can just use 16 bpp all the time.
-	 */
-	rate = ov5640_calc_pixel_rate(sensor) * 16;
-	if (ov5640_is_mipi(sensor)) {
-		rate = rate / sensor->ep.bus.mipi_csi2.num_data_lanes;
-		ret = ov5640_set_mipi_pclk(sensor, rate);
-	} else {
-		rate = rate / sensor->ep.bus.parallel.bus_width;
-		ret = ov5640_set_dvp_pclk(sensor, rate);
-	}
-
+	if (ov5640_is_mipi(sensor))
+		ret = ov5640_set_mipi_pclk(sensor);
+	else
+		ret = ov5640_set_dvp_pclk(sensor);
 	if (ret < 0)
 		return 0;
 
-- 
2.35.0

[PATCH 07/21] media: ov5640: Rework timings programming

[Jacopo Mondi]

The current definition of a sensor mode defines timings as follows:

- hact, vact: Visible width and height
- htot, vtot: Total sizes invluding blankings

This makes difficult to clearly separate the visible sizes from the
blankings and to make the vertical blanking programmable.

Rework the sensor modes sizes definition to:
- Report the analog crop sizes
- Report the visible crop size
- Report the total pixels per line as HBLANK is fixed
- Report the VBLANK value to make it programmable

Also modify the ov5640_set_timings() function to program all the
windowing registers are remove them from the per-mode register-value
tables.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 451 +++++++++++++++++++++++++------------
 1 file changed, 304 insertions(+), 147 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index acc636500907..bd14e2ad22f6 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -61,10 +61,16 @@
 #define OV5640_REG_AEC_PK_MANUAL	0x3503
 #define OV5640_REG_AEC_PK_REAL_GAIN	0x350a
 #define OV5640_REG_AEC_PK_VTS		0x350c
+#define OV5640_REG_TIMING_HS		0x3800
+#define OV5640_REG_TIMING_VS		0x3802
+#define OV5640_REG_TIMING_HW		0x3804
+#define OV5640_REG_TIMING_VH		0x3806
 #define OV5640_REG_TIMING_DVPHO		0x3808
 #define OV5640_REG_TIMING_DVPVO		0x380a
 #define OV5640_REG_TIMING_HTS		0x380c
 #define OV5640_REG_TIMING_VTS		0x380e
+#define OV5640_REG_TIMING_HOFFS		0x3810
+#define OV5640_REG_TIMING_VOFFS		0x3812
 #define OV5640_REG_TIMING_TC_REG20	0x3820
 #define OV5640_REG_TIMING_TC_REG21	0x3821
 #define OV5640_REG_AEC_CTRL00		0x3a00
@@ -242,12 +248,17 @@ struct ov5640_mode_info {
 	enum ov5640_mode_id id;
 	enum ov5640_downsize_mode dn_mode;
 	enum ov5640_pixel_rate_id pixel_rate;
-	u32 hact;
-	u32 htot;
-	u32 vact;
-	u32 vtot;
+	/*  Analog crop rectangle. */
+	struct v4l2_rect analog_crop;
+	/* Visibile crop: from analog crop top-left corner. */
+	struct v4l2_rect crop;
+	/* Total pixels per line: crop.width + fixed hblank. */
+	u32 ppl;
+	/* Total frame height = crop.height + vblank. */
+	u32 vblank_def;
 	const struct reg_value *reg_data;
 	u32 reg_data_size;
+	/* DVP only; ignored in MIPI mode. */
 	u32 max_fps;
 };
 
@@ -353,11 +364,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
 	{0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -425,11 +432,7 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -444,11 +447,7 @@ static const struct reg_value ov5640_setting_XGA_1024_768[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -463,11 +462,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -482,11 +477,7 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -500,11 +491,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -557,11 +536,7 @@ static const struct reg_value ov5640_setting_720P_1280_720[] = {
 	{0x3c07, 0x07, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+	{0x3815, 0x31, 0, 0},
 	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
 	{0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
@@ -576,11 +551,7 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x11, 0, 0},
-	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+	{0x3815, 0x11, 0, 0},
 	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
 	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -591,9 +562,6 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
 	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0},
 	{0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
-	{0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
-	{0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
-	{0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0},
 	{0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
 	{0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
 	{0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
@@ -607,11 +575,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x11, 0, 0},
-	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
-	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
-	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
-	{0x3810, 0x00, 0, 0},
-	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+	{0x3815, 0x11, 0, 0},
 	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
 	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
 	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
@@ -624,96 +588,250 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
 
 /* power-on sensor init reg table */
 static const struct ov5640_mode_info ov5640_mode_init_data = {
-	0, SUBSAMPLING,
-	OV5640_PIXEL_RATE_96M,
-	640, 1896, 480, 984,
-	ov5640_init_setting_30fps_VGA,
-	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
-	OV5640_30_FPS,
+		.id		= 0,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_96M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 640,
+			.height	= 480,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 504,
+		.reg_data	= ov5640_init_setting_30fps_VGA,
+		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
+		.max_fps	= OV5640_30_FPS
 };
 
 static const struct ov5640_mode_info
 ov5640_mode_data[OV5640_NUM_MODES] = {
 	{
 		/* 160x120 */
-		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
-		OV5640_PIXEL_RATE_48M,
-		160, 1896, 120, 984,
-		ov5640_setting_QQVGA_160_120,
-		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_QQVGA_160_120,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_48M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 160,
+			.height	= 120,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 864,
+		.reg_data	= ov5640_setting_QQVGA_160_120,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 176x144 */
-		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
-		OV5640_PIXEL_RATE_48M,
-		176, 1896, 144, 984,
-		ov5640_setting_QCIF_176_144,
-		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_QCIF_176_144,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_48M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 176,
+			.height	= 144,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 840,
+		.reg_data	= ov5640_setting_QCIF_176_144,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 320x240 */
-		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
-		OV5640_PIXEL_RATE_48M,
-		320, 1896, 240, 984,
-		ov5640_setting_QVGA_320_240,
-		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_QVGA_320_240,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_48M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 320,
+			.height	= 240,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 744,
+		.reg_data	= ov5640_setting_QVGA_320_240,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 640x480 */
-		OV5640_MODE_VGA_640_480, SUBSAMPLING,
-		OV5640_PIXEL_RATE_48M,
-		640, 1896, 480, 1080,
-		ov5640_setting_VGA_640_480,
-		ARRAY_SIZE(ov5640_setting_VGA_640_480),
-		OV5640_60_FPS
+		.id		= OV5640_MODE_VGA_640_480,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_48M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 640,
+			.height	= 480,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 600,
+		.reg_data	= ov5640_setting_VGA_640_480,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
+		.max_fps	= OV5640_60_FPS
 	}, {
 		/* 720x480 */
-		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
-		OV5640_PIXEL_RATE_96M,
-		720, 1896, 480, 984,
-		ov5640_setting_NTSC_720_480,
-		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_NTSC_720_480,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_96M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 56,
+			.top	= 60,
+			.width	= 720,
+			.height	= 480,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 504,
+		.reg_data	= ov5640_setting_NTSC_720_480,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 720x576 */
-		OV5640_MODE_PAL_720_576, SUBSAMPLING,
-		OV5640_PIXEL_RATE_96M,
-		720, 1896, 576, 984,
-		ov5640_setting_PAL_720_576,
-		ARRAY_SIZE(ov5640_setting_PAL_720_576),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_PAL_720_576,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_96M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 56,
+			.top	= 60,
+			.width	= 720,
+			.height	= 576,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 408,
+		.reg_data	= ov5640_setting_PAL_720_576,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 1024x768 */
-		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
-		OV5640_PIXEL_RATE_96M,
-		1024, 1896, 768, 1080,
-		ov5640_setting_XGA_1024_768,
-		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_XGA_1024_768,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_96M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 4,
+			.width	= 2623,
+			.height	= 1947,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 6,
+			.width	= 1024,
+			.height	= 768,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 312,
+		.reg_data	= ov5640_setting_XGA_1024_768,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 1280x720 */
-		OV5640_MODE_720P_1280_720, SUBSAMPLING,
-		OV5640_PIXEL_RATE_124M,
-		1280, 1892, 720, 740,
-		ov5640_setting_720P_1280_720,
-		ARRAY_SIZE(ov5640_setting_720P_1280_720),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_720P_1280_720,
+		.dn_mode	= SUBSAMPLING,
+		.pixel_rate	= OV5640_PIXEL_RATE_124M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 250,
+			.width	= 2623,
+			.height	= 1705,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 4,
+			.width	= 1280,
+			.height	= 720,
+		},
+		.ppl		= 1896,
+		.vblank_def	= 20,
+		.reg_data	= ov5640_setting_720P_1280_720,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 1920x1080 */
-		OV5640_MODE_1080P_1920_1080, SCALING,
-		OV5640_PIXEL_RATE_148M,
-		1920, 2500, 1080, 1120,
-		ov5640_setting_1080P_1920_1080,
-		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
-		OV5640_30_FPS
+		.id		= OV5640_MODE_1080P_1920_1080,
+		.dn_mode	= SCALING,
+		.pixel_rate	= OV5640_PIXEL_RATE_148M,
+		.analog_crop = {
+			.left	= 336,
+			.top	= 434,
+			.width	= 2287,
+			.height	= 1521,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 4,
+			.width	= 1920,
+			.height	= 1080,
+		},
+		.ppl		= 2500,
+		.vblank_def	= 40,
+		.reg_data	= ov5640_setting_1080P_1920_1080,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
+		.max_fps	= OV5640_30_FPS
 	}, {
 		/* 2592x1944 */
-		OV5640_MODE_QSXGA_2592_1944, SCALING,
-		OV5640_PIXEL_RATE_168M,
-		2592, 2844, 1944, 1968,
-		ov5640_setting_QSXGA_2592_1944,
-		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
-		OV5640_15_FPS
+		.id		= OV5640_MODE_QSXGA_2592_1944,
+		.dn_mode	= SCALING,
+		.pixel_rate	= OV5640_PIXEL_RATE_168M,
+		.analog_crop = {
+			.left	= 0,
+			.top	= 0,
+			.width	= 2623,
+			.height	= 1951,
+		},
+		.crop = {
+			.left	= 16,
+			.top	= 4,
+			.width	= 2592,
+			.height	= 1944,
+		},
+		.ppl		= 2844,
+		.vblank_def	= 24,
+		.reg_data	= ov5640_setting_QSXGA_2592_1944,
+		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
+		.max_fps	= OV5640_15_FPS
 	},
 };
 
@@ -1136,7 +1254,8 @@ static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
 {
 	u64 rate;
 
-	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
+	rate = sensor->current_mode->ppl
+	     * (sensor->current_mode->crop.height + sensor->current_mode->vblank_def);
 	rate *= ov5640_framerates[sensor->current_fr];
 
 	return rate;
@@ -1220,17 +1339,21 @@ static int ov5640_set_jpeg_timings(struct ov5640_dev *sensor,
 	if (ret < 0)
 		return ret;
 
-	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE, mode->hact);
+	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE,
+				 mode->crop.width);
 	if (ret < 0)
 		return ret;
 
-	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE, mode->vact);
+	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE,
+				  mode->crop.height);
 }
 
 /* download ov5640 settings to sensor through i2c */
 static int ov5640_set_timings(struct ov5640_dev *sensor,
 			      const struct ov5640_mode_info *mode)
 {
+	const struct v4l2_rect *analog_crop = &mode->analog_crop;
+	const struct v4l2_rect *crop = &mode->crop;
 	int ret;
 
 	if (sensor->fmt.code == MEDIA_BUS_FMT_JPEG_1X8) {
@@ -1239,19 +1362,54 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
 			return ret;
 	}
 
-	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HS,
+				 analog_crop->left);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VS,
+				 analog_crop->top);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HW,
+				 analog_crop->width);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VH,
+				 analog_crop->height);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, crop->width);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, crop->height);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->ppl);
+	if (ret < 0)
+		return ret;
+
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
+				 crop->height + mode->vblank_def);
 	if (ret < 0)
 		return ret;
 
-	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HOFFS,
+				 crop->left);
 	if (ret < 0)
 		return ret;
 
-	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
+	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VOFFS,
+				 crop->top);
 	if (ret < 0)
 		return ret;
 
-	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
+	return 0;
 }
 
 static int ov5640_load_regs(struct ov5640_dev *sensor,
@@ -1679,11 +1837,11 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
 
 	mode = v4l2_find_nearest_size(ov5640_mode_data,
 				      ARRAY_SIZE(ov5640_mode_data),
-				      hact, vact,
-				      width, height);
+				      crop.width, crop.height, width, height);
 
 	if (!mode ||
-	    (!nearest && (mode->hact != width || mode->vact != height)))
+	    (!nearest &&
+	     (mode->crop.width != width || mode->crop.height != height)))
 		return NULL;
 
 	/* Check to see if the current mode exceeds the max frame rate */
@@ -2367,8 +2525,8 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
 	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
 	if (!mode)
 		return -EINVAL;
-	fmt->width = mode->hact;
-	fmt->height = mode->vact;
+	fmt->width = mode->crop.width;
+	fmt->height = mode->crop.height;
 
 	if (new_mode)
 		*new_mode = mode;
@@ -2999,11 +3157,9 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
 	if (fse->index >= OV5640_NUM_MODES)
 		return -EINVAL;
 
-	fse->min_width =
-		ov5640_mode_data[fse->index].hact;
+	fse->min_width = ov5640_mode_data[fse->index].crop.width;
 	fse->max_width = fse->min_width;
-	fse->min_height =
-		ov5640_mode_data[fse->index].vact;
+	fse->min_height = ov5640_mode_data[fse->index].crop.height;
 	fse->max_height = fse->min_height;
 
 	return 0;
@@ -3067,15 +3223,16 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
 	mode = sensor->current_mode;
 
 	frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
-					       mode->hact, mode->vact);
+					       mode->crop.width,
+					       mode->crop.height);
 	if (frame_rate < 0) {
 		/* Always return a valid frame interval value */
 		fi->interval = sensor->frame_interval;
 		goto out;
 	}
 
-	mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
-				mode->vact, true);
+	mode = ov5640_find_mode(sensor, frame_rate, mode->crop.width,
+				mode->crop.height, true);
 	if (!mode) {
 		ret = -EINVAL;
 		goto out;
-- 
2.35.0

[PATCH 08/21] media: ov5640: Re-sort per-mode register tables

[Jacopo Mondi]

The per-mode register tables are not sorted by size. Fix it.

Cosmetic change only.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index bd14e2ad22f6..d966cca78e92 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -428,7 +428,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
 	{0x3a1f, 0x14, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3c00, 0x04, 0, 300},
 };
 
-static const struct reg_value ov5640_setting_VGA_640_480[] = {
+static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
@@ -439,11 +439,10 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
 	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
 	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
 	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
-	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
-	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
 };
 
-static const struct reg_value ov5640_setting_XGA_1024_768[] = {
+static const struct reg_value ov5640_setting_QCIF_176_144[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
@@ -473,7 +472,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
 	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
 };
 
-static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
+static const struct reg_value ov5640_setting_VGA_640_480[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
@@ -484,10 +483,11 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
 	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
 	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
 	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
-	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
+	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
 };
 
-static const struct reg_value ov5640_setting_QCIF_176_144[] = {
+static const struct reg_value ov5640_setting_NTSC_720_480[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
@@ -502,7 +502,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
 	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
 };
 
-static const struct reg_value ov5640_setting_NTSC_720_480[] = {
+static const struct reg_value ov5640_setting_PAL_720_576[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
@@ -517,7 +517,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
 	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
 };
 
-static const struct reg_value ov5640_setting_PAL_720_576[] = {
+static const struct reg_value ov5640_setting_XGA_1024_768[] = {
 	{0x3c07, 0x08, 0, 0},
 	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
 	{0x3814, 0x31, 0, 0},
-- 
2.35.0

[PATCH 09/21] media: ov5640: Remove ov5640_mode_init_data

[Jacopo Mondi]

The ov5640_mode_init_data is a fictional sensor more which is used to
program the initial sensor settings.

It is only used to initialize the sensor and can be replaced
it with a throw-away mode which just wraps the register table.

Also rename the register table to drop the format from the name to make
it clear an actual sensor mode has to be applied after the initial
programming.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 32 +++++---------------------------
 1 file changed, 5 insertions(+), 27 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index d966cca78e92..1e2f37c93f0d 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -348,7 +348,7 @@ static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
  * over i2c.
  */
 /* YUV422 UYVY VGA@30fps */
-static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
+static const struct reg_value ov5640_init_setting[] = {
 	{0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
 	{0x3103, 0x03, 0, 0}, {0x3630, 0x36, 0, 0},
 	{0x3631, 0x0e, 0, 0}, {0x3632, 0xe2, 0, 0}, {0x3633, 0x12, 0, 0},
@@ -586,30 +586,6 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
 	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
 };
 
-/* power-on sensor init reg table */
-static const struct ov5640_mode_info ov5640_mode_init_data = {
-		.id		= 0,
-		.dn_mode	= SUBSAMPLING,
-		.pixel_rate	= OV5640_PIXEL_RATE_96M,
-		.analog_crop = {
-			.left	= 0,
-			.top	= 4,
-			.width	= 2623,
-			.height	= 1947,
-		},
-		.crop = {
-			.left	= 16,
-			.top	= 6,
-			.width	= 640,
-			.height	= 480,
-		},
-		.ppl		= 1896,
-		.vblank_def	= 504,
-		.reg_data	= ov5640_init_setting_30fps_VGA,
-		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
-		.max_fps	= OV5640_30_FPS
-};
-
 static const struct ov5640_mode_info
 ov5640_mode_data[OV5640_NUM_MODES] = {
 	{
@@ -2117,13 +2093,15 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
 /* restore the last set video mode after chip power-on */
 static int ov5640_restore_mode(struct ov5640_dev *sensor)
 {
+	struct ov5640_mode_info init_data = {};
 	int ret;
 
 	/* first load the initial register values */
-	ret = ov5640_load_regs(sensor, &ov5640_mode_init_data);
+	init_data.reg_data = ov5640_init_setting;
+	init_data.reg_data_size = ARRAY_SIZE(ov5640_init_setting);
+	ret = ov5640_load_regs(sensor, &init_data);
 	if (ret < 0)
 		return ret;
-	sensor->last_mode = &ov5640_mode_init_data;
 
 	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
 			     (ilog2(OV5640_SCLK2X_ROOT_DIV) << 2) |
-- 
2.35.0

[PATCH 10/21] media: ov5640: Add HBLANK control

[Jacopo Mondi]

Add the HBLANK control as read-only.

The hblank value is fixed in the mode definition and is updated
everytime a new format is applied.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 1e2f37c93f0d..b2961e8e07c3 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -266,6 +266,7 @@ struct ov5640_ctrls {
 	struct v4l2_ctrl_handler handler;
 	struct v4l2_ctrl *pixel_rate;
 	struct v4l2_ctrl *link_freq;
+	struct v4l2_ctrl *hblank;
 	struct {
 		struct v4l2_ctrl *auto_exp;
 		struct v4l2_ctrl *exposure;
@@ -2530,6 +2531,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
 	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
 	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
 	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
+	unsigned int hblank;
 	unsigned int i = 0;
 	u32 pixel_rate;
 	s64 link_freq;
@@ -2580,6 +2582,10 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
 	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate, pixel_rate);
 	__v4l2_ctrl_s_ctrl(sensor->ctrls.link_freq, i);
 
+	hblank = mode->ppl - mode->crop.width;
+	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
+				 hblank, hblank, 1, hblank);
+
 	return 0;
 }
 
@@ -3044,6 +3050,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
 	struct ov5640_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+	unsigned int hblank;
 	int ret;
 
 	v4l2_ctrl_handler_init(hdl, 32);
@@ -3062,6 +3069,10 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 					ARRAY_SIZE(ov5640_csi2_link_freqs) - 1,
 					4, ov5640_csi2_link_freqs);
 
+	hblank = mode->ppl - mode->crop.width;
+	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
+					  hblank, 1, hblank);
+
 	/* Auto/manual white balance */
 	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
 					   V4L2_CID_AUTO_WHITE_BALANCE,
@@ -3111,6 +3122,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 
 	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
+	ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 	ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
 	ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
 
-- 
2.35.0

[PATCH 11/21] media: ov5640: Add VBLANK control

[Jacopo Mondi]

Add the VBLANK control which allows to select the duration of the
frame vertical blankings and allows to control the framerate.

The VBLANK control also modifies the exposure time range, which cannot
exceed the maximum frame length.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index b2961e8e07c3..6eeb50724195 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -31,6 +31,10 @@
 
 #define OV5640_LINK_RATE_MAX	490000000U
 
+/* FIXME: not documented. */
+#define OV5640_MIN_VBLANK	24
+#define OV5640_MAX_VTS		1968
+
 #define OV5640_DEFAULT_SLAVE_ID 0x3c
 
 #define OV5640_REG_SYS_RESET02		0x3002
@@ -267,6 +271,7 @@ struct ov5640_ctrls {
 	struct v4l2_ctrl *pixel_rate;
 	struct v4l2_ctrl *link_freq;
 	struct v4l2_ctrl *hblank;
+	struct v4l2_ctrl *vblank;
 	struct {
 		struct v4l2_ctrl *auto_exp;
 		struct v4l2_ctrl *exposure;
@@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
 	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
 	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
 	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
+	s64 exposure_val, exposure_max;
 	unsigned int hblank;
 	unsigned int i = 0;
 	u32 pixel_rate;
@@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
 	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
 				 hblank, hblank, 1, hblank);
 
+	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
+				 OV5640_MIN_VBLANK,
+				 OV5640_MAX_VTS - mode->crop.height, 1,
+				 mode->vblank_def);
+	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
+
+	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;
+	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
+			     (s64)sensor->ctrls.exposure->minimum,
+			     (s64)exposure_max);
+	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
+				 sensor->ctrls.exposure->minimum,
+				 exposure_max, 1, exposure_val);
+
 	return 0;
 }
 
@@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
 			      (BIT(2) | BIT(1)) : 0);
 }
 
+static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
+{
+	const struct ov5640_mode_info *mode = sensor->current_mode;
+
+	/* Update the VTOT timing register value. */
+	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
+				  mode->crop.height + value);
+}
+
 static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
@@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
 	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+	const struct ov5640_mode_info *mode = sensor->current_mode;
 	int ret;
 
 	/* v4l2_ctrl_lock() locks our own mutex */
 
+	switch (ctrl->id) {
+	case V4L2_CID_VBLANK:
+		/* Update the exposure range to the newly programmed vblank. */
+		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
+
+		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
+					 sensor->ctrls.exposure->minimum,
+					 max, sensor->ctrls.exposure->step, max);
+		break;
+	}
+
 	/*
 	 * If the device is not powered up by the host driver do
 	 * not apply any controls to H/W at this time. Instead
@@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_VFLIP:
 		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
 		break;
+	case V4L2_CID_VBLANK:
+		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
+		break;
 	default:
 		ret = -EINVAL;
 		break;
@@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
 	struct ov5640_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+	unsigned int max_vblank;
 	unsigned int hblank;
 	int ret;
 
@@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
 					  hblank, 1, hblank);
 
+	max_vblank = OV5640_MAX_VTS - mode->crop.height;
+	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
+					  OV5640_MIN_VBLANK, max_vblank,
+					  1, mode->vblank_def);
+
 	/* Auto/manual white balance */
 	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
 					   V4L2_CID_AUTO_WHITE_BALANCE,
-- 
2.35.0

[PATCH 12/21] media: ov5640: Fix durations to comply with FPS

[Jacopo Mondi]

Now that the frame duration can be controlled by tuning the VBLANK
duration, fix all modes to comply with the reported FPS.

All modes run at 30 FPS except for full-resolution mode 2592x1944
which runs at 15FPS.

Tested on a 2 data lanes setup in UYVY and RGB565 modes.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 6eeb50724195..2176fa0b8eae 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -611,8 +611,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 160,
 			.height	= 120,
 		},
-		.ppl		= 1896,
-		.vblank_def	= 864,
+		.ppl		= 1600,
+		.vblank_def	= 878,
 		.reg_data	= ov5640_setting_QQVGA_160_120,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
 		.max_fps	= OV5640_30_FPS
@@ -633,8 +633,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 176,
 			.height	= 144,
 		},
-		.ppl		= 1896,
-		.vblank_def	= 840,
+		.ppl		= 1600,
+		.vblank_def	= 854,
 		.reg_data	= ov5640_setting_QCIF_176_144,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
 		.max_fps	= OV5640_30_FPS
@@ -655,8 +655,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 320,
 			.height	= 240,
 		},
-		.ppl		= 1896,
-		.vblank_def	= 744,
+		.ppl		= 1600,
+		.vblank_def	= 760,
 		.reg_data	= ov5640_setting_QVGA_320_240,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
 		.max_fps	= OV5640_30_FPS
@@ -677,8 +677,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 640,
 			.height	= 480,
 		},
-		.ppl		= 1896,
-		.vblank_def	= 600,
+		.ppl		= 1600,
+		.vblank_def	= 520,
 		.reg_data	= ov5640_setting_VGA_640_480,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
 		.max_fps	= OV5640_60_FPS
@@ -700,7 +700,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.height	= 480,
 		},
 		.ppl		= 1896,
-		.vblank_def	= 504,
+		.vblank_def	= 1206,
 		.reg_data	= ov5640_setting_NTSC_720_480,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
 		.max_fps	= OV5640_30_FPS
@@ -722,7 +722,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.height	= 576,
 		},
 		.ppl		= 1896,
-		.vblank_def	= 408,
+		.vblank_def	= 1110,
 		.reg_data	= ov5640_setting_PAL_720_576,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
 		.max_fps	= OV5640_30_FPS
@@ -744,7 +744,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.height	= 768,
 		},
 		.ppl		= 1896,
-		.vblank_def	= 312,
+		.vblank_def	= 918,
 		.reg_data	= ov5640_setting_XGA_1024_768,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
 		.max_fps	= OV5640_30_FPS
@@ -765,8 +765,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 1280,
 			.height	= 720,
 		},
-		.ppl		= 1896,
-		.vblank_def	= 20,
+		.ppl		= 1600,
+		.vblank_def	= 560,
 		.reg_data	= ov5640_setting_720P_1280_720,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
 		.max_fps	= OV5640_30_FPS
@@ -787,8 +787,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
 			.width	= 1920,
 			.height	= 1080,
 		},
-		.ppl		= 2500,
-		.vblank_def	= 40,
+		.ppl		= 2234,
+		.vblank_def	= 24,
 		.reg_data	= ov5640_setting_1080P_1920_1080,
 		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
 		.max_fps	= OV5640_30_FPS
-- 
2.35.0

[PATCH 13/21] media: ov5640: Initialize try format

[Jacopo Mondi]

The TRY format is not initialized at device node open time.

Fix that by implementing the open() subdev internal function and
initialize the TRY format there with the default sensor format.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 38 ++++++++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 10 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 2176fa0b8eae..762bdca83aec 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -354,6 +354,18 @@ static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
  * over i2c.
  */
 /* YUV422 UYVY VGA@30fps */
+
+static struct v4l2_mbus_framefmt ov5640_default_fmt = {
+	.code = MEDIA_BUS_FMT_UYVY8_2X8,
+	.width = 640,
+	.height = 480,
+	.colorspace = V4L2_COLORSPACE_SRGB,
+	.ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(V4L2_COLORSPACE_SRGB),
+	.quantization = V4L2_QUANTIZATION_FULL_RANGE,
+	.xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(V4L2_COLORSPACE_SRGB),
+	.field = V4L2_FIELD_NONE,
+};
+
 static const struct reg_value ov5640_init_setting[] = {
 	{0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
 	{0x3103, 0x03, 0, 0}, {0x3630, 0x36, 0, 0},
@@ -3367,6 +3379,20 @@ static const struct v4l2_subdev_ops ov5640_subdev_ops = {
 	.pad = &ov5640_pad_ops,
 };
 
+static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+	struct v4l2_mbus_framefmt *try_fmt =
+		v4l2_subdev_get_try_format(sd, fh->state, 0);
+
+	*try_fmt = ov5640_default_fmt;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_internal_ops ov5640_internal_ops = {
+	.open = &ov5640_open,
+};
+
 static int ov5640_get_regulators(struct ov5640_dev *sensor)
 {
 	int i;
@@ -3412,7 +3438,6 @@ static int ov5640_probe(struct i2c_client *client)
 	struct device *dev = &client->dev;
 	struct fwnode_handle *endpoint;
 	struct ov5640_dev *sensor;
-	struct v4l2_mbus_framefmt *fmt;
 	u32 rotation;
 	int ret;
 
@@ -3426,15 +3451,7 @@ static int ov5640_probe(struct i2c_client *client)
 	 * default init sequence initialize sensor to
 	 * YUV422 UYVY VGA@30fps
 	 */
-	fmt = &sensor->fmt;
-	fmt->code = MEDIA_BUS_FMT_UYVY8_2X8;
-	fmt->colorspace = V4L2_COLORSPACE_SRGB;
-	fmt->ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(fmt->colorspace);
-	fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
-	fmt->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(fmt->colorspace);
-	fmt->width = 640;
-	fmt->height = 480;
-	fmt->field = V4L2_FIELD_NONE;
+	sensor->fmt = ov5640_default_fmt;
 	sensor->frame_interval.numerator = 1;
 	sensor->frame_interval.denominator = ov5640_framerates[OV5640_30_FPS];
 	sensor->current_fr = OV5640_30_FPS;
@@ -3510,6 +3527,7 @@ static int ov5640_probe(struct i2c_client *client)
 
 	v4l2_i2c_subdev_init(&sensor->sd, client, &ov5640_subdev_ops);
 
+	sensor->sd.internal_ops = &ov5640_internal_ops;
 	sensor->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
 			    V4L2_SUBDEV_FL_HAS_EVENTS;
 	sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
-- 
2.35.0

[PATCH 14/21] media: ov5640: Implement get_selection

[Jacopo Mondi]

Implement the get_selection pad operation for the OV5640 sensor driver.

The supported targets report the sensor's native size, the active pixel
array size and the analog crop rectangle from which the image is
produced.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 762bdca83aec..ae22300b9655 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -35,6 +35,13 @@
 #define OV5640_MIN_VBLANK	24
 #define OV5640_MAX_VTS		1968
 
+#define OV5640_NATIVE_WIDTH		2624
+#define OV5640_NATIVE_HEIGHT		1964
+#define OV5640_PIXEL_ARRAY_TOP		8
+#define OV5640_PIXEL_ARRAY_LEFT		16
+#define OV5640_PIXEL_ARRAY_WIDTH	2592
+#define OV5640_PIXEL_ARRAY_HEIGHT	1944
+
 #define OV5640_DEFAULT_SLAVE_ID 0x3c
 
 #define OV5640_REG_SYS_RESET02		0x3002
@@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
 	return ret;
 }
 
+static int ov5640_get_selection(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *sd_state,
+				struct v4l2_subdev_selection *sel)
+{
+	struct ov5640_dev *sensor = to_ov5640_dev(sd);
+	const struct ov5640_mode_info *mode = sensor->current_mode;
+	const struct v4l2_rect *analog_crop = &mode->analog_crop;
+	const struct v4l2_rect *crop = &mode->crop;
+
+	switch (sel->target) {
+	case V4L2_SEL_TGT_CROP: {
+		mutex_lock(&sensor->lock);
+
+		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
+		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
+		sel->r.width = analog_crop->width
+			     - analog_crop->left - crop->left;
+		sel->r.height = analog_crop->height
+			      - analog_crop->top - crop->top;
+
+		mutex_unlock(&sensor->lock);
+
+		return 0;
+	}
+
+	case V4L2_SEL_TGT_NATIVE_SIZE:
+		sel->r.top = 0;
+		sel->r.left = 0;
+		sel->r.width = OV5640_NATIVE_WIDTH;
+		sel->r.height = OV5640_NATIVE_HEIGHT;
+
+		return 0;
+
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
+		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
+		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
+		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;
+
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
 static int ov5640_set_framefmt(struct ov5640_dev *sensor,
 			       struct v4l2_mbus_framefmt *format)
 {
@@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
 	.enum_mbus_code = ov5640_enum_mbus_code,
 	.get_fmt = ov5640_get_fmt,
 	.set_fmt = ov5640_set_fmt,
+	.get_selection = ov5640_get_selection,
 	.enum_frame_size = ov5640_enum_frame_size,
 	.enum_frame_interval = ov5640_enum_frame_interval,
 };
@@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 {
 	struct v4l2_mbus_framefmt *try_fmt =
 		v4l2_subdev_get_try_format(sd, fh->state, 0);
+	struct v4l2_rect *try_crop =
+		v4l2_subdev_get_try_crop(sd, fh->state, 0);
 
 	*try_fmt = ov5640_default_fmt;
 
+	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
+	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
+	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
+	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
+
 	return 0;
 }
 
-- 
2.35.0

[PATCH 15/21] media: ov5640: Limit format to FPS in DVP mode only

[Jacopo Mondi]

In MIPI mode the frame rate control is performed by adjusting the
frame blankings and the s_frame_interval function is not used anymore.

Only check for the per-mode supported frame rate in DVP mode and do not
restrict MIPI mode.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index ae22300b9655..ec46e16223af 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -1845,8 +1845,13 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
 	     (mode->crop.width != width || mode->crop.height != height)))
 		return NULL;
 
-	/* Check to see if the current mode exceeds the max frame rate */
-	if (ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
+	/*
+	 * Check to see if the current mode exceeds the max frame rate.
+	 * Only DVP mode uses the frame rate set by s_frame_interval, MIPI
+	 * mode controls framerate by setting blankings.
+	 */
+	if (!ov5640_is_mipi(sensor) &&
+	    ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
 		return NULL;
 
 	return mode;
-- 
2.35.0

[PATCH 16/21] media: ov5640: Disable s_frame_interval in MIPI mode

[Jacopo Mondi]

When the sensor is operated in MIPI mode, the frame rate configuration
is performed by tuning the frame blanking times and not by the
s_frame_interval subdev operation.

Disallow enum/s/g_frame_interval if the chip is used in MIPI mode.

While at it re-indent one function which whose parameters were wrongly
aligned.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index ec46e16223af..28da0ddd2a06 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -3275,15 +3275,17 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static int ov5640_enum_frame_interval(
-	struct v4l2_subdev *sd,
-	struct v4l2_subdev_state *sd_state,
-	struct v4l2_subdev_frame_interval_enum *fie)
+static int ov5640_enum_frame_interval(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *sd_state,
+				      struct v4l2_subdev_frame_interval_enum *fie)
 {
 	struct ov5640_dev *sensor = to_ov5640_dev(sd);
 	struct v4l2_fract tpf;
 	int ret;
 
+	if (ov5640_is_mipi(sensor))
+		return -EINVAL;
+
 	if (fie->pad != 0)
 		return -EINVAL;
 	if (fie->index >= OV5640_NUM_FRAMERATES)
@@ -3306,6 +3308,9 @@ static int ov5640_g_frame_interval(struct v4l2_subdev *sd,
 {
 	struct ov5640_dev *sensor = to_ov5640_dev(sd);
 
+	if (ov5640_is_mipi(sensor))
+		return -EINVAL;
+
 	mutex_lock(&sensor->lock);
 	fi->interval = sensor->frame_interval;
 	mutex_unlock(&sensor->lock);
@@ -3320,6 +3325,9 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
 	const struct ov5640_mode_info *mode;
 	int frame_rate, ret = 0;
 
+	if (ov5640_is_mipi(sensor))
+		return -EINVAL;
+
 	if (fi->pad != 0)
 		return -EINVAL;
 
-- 
2.35.0

[PATCH 17/21] media: ov5640: Register device properties

[Jacopo Mondi]

Parse the device properties and register the rotation and orientation
V4L2 controls using v4l2_ctrl_new_fwnode_properties().

Remove the open-coded parsing of the rotation property and assume the
DTS is correct is providing either <0> or <180> as possible rotations.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 29 ++++++++++++-----------------
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 28da0ddd2a06..9d6eb3288b07 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -3164,6 +3164,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
 	struct ov5640_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+	struct v4l2_fwnode_device_properties props;
 	unsigned int max_vblank;
 	unsigned int hblank;
 	int ret;
@@ -3240,6 +3241,17 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
 		goto free_ctrls;
 	}
 
+	ret = v4l2_fwnode_device_parse(&sensor->i2c_client->dev, &props);
+	if (ret)
+		goto free_ctrls;
+
+	if (props.rotation == 180)
+		sensor->upside_down = true;
+
+	ret = v4l2_ctrl_new_fwnode_properties(hdl, ops, &props);
+	if (ret)
+		goto free_ctrls;
+
 	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 	ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
@@ -3512,7 +3524,6 @@ static int ov5640_probe(struct i2c_client *client)
 	struct device *dev = &client->dev;
 	struct fwnode_handle *endpoint;
 	struct ov5640_dev *sensor;
-	u32 rotation;
 	int ret;
 
 	sensor = devm_kzalloc(dev, sizeof(*sensor), GFP_KERNEL);
@@ -3535,22 +3546,6 @@ static int ov5640_probe(struct i2c_client *client)
 
 	sensor->ae_target = 52;
 
-	/* optional indication of physical rotation of sensor */
-	ret = fwnode_property_read_u32(dev_fwnode(&client->dev), "rotation",
-				       &rotation);
-	if (!ret) {
-		switch (rotation) {
-		case 180:
-			sensor->upside_down = true;
-			fallthrough;
-		case 0:
-			break;
-		default:
-			dev_warn(dev, "%u degrees rotation is not supported, ignoring...\n",
-				 rotation);
-		}
-	}
-
 	endpoint = fwnode_graph_get_next_endpoint(dev_fwnode(&client->dev),
 						  NULL);
 	if (!endpoint) {
-- 
2.35.0

[PATCH 18/21] media: ov5640: Add RGB565_1X16 format

[Jacopo Mondi]

The driver already supports the 2X8_[LE|BE] variants of RGB565
formats.

As for CSI-2 the 2X8 variants do not apply, add RGB565_1X16 variant
with little-endian ordering of components as required by the CSI-2
specifications.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 9d6eb3288b07..db77cdb8a958 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -197,6 +197,7 @@ static const struct ov5640_pixfmt {
 	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, 16,},
 	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
 	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_RGB565_1X16, V4L2_COLORSPACE_SRGB, 16},
 	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
 	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
 	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
@@ -2746,6 +2747,7 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
 		mux = OV5640_FMT_MUX_YUV422;
 		break;
 	case MEDIA_BUS_FMT_RGB565_2X8_LE:
+	case MEDIA_BUS_FMT_RGB565_1X16:
 		/* RGB565 {g[2:0],b[4:0]},{r[4:0],g[5:3]} */
 		fmt = 0x6F;
 		mux = OV5640_FMT_MUX_RGB;
-- 
2.35.0

[PATCH 19/21] media: ov5640: Add RGB888/BGR888 formats

[Jacopo Mondi]

Add support for RGB888/BGR888 image formats.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index db77cdb8a958..5891eaab05ee 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -198,6 +198,8 @@ static const struct ov5640_pixfmt {
 	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
 	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
 	{ MEDIA_BUS_FMT_RGB565_1X16, V4L2_COLORSPACE_SRGB, 16},
+	{ MEDIA_BUS_FMT_RGB888_1X24, V4L2_COLORSPACE_SRGB, 24},
+	{ MEDIA_BUS_FMT_BGR888_1X24, V4L2_COLORSPACE_SRGB, 24},
 	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
 	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
 	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
@@ -2757,6 +2759,16 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
 		fmt = 0x61;
 		mux = OV5640_FMT_MUX_RGB;
 		break;
+	case MEDIA_BUS_FMT_RGB888_1X24:
+		/* RGB888: BGR */
+		fmt = 0x22;
+		mux = OV5640_FMT_MUX_RGB;
+		break;
+	case MEDIA_BUS_FMT_BGR888_1X24:
+		/* BGR888: RGB */
+		fmt = 0x23;
+		mux = OV5640_FMT_MUX_RGB;
+		break;
 	case MEDIA_BUS_FMT_JPEG_1X8:
 		/* YUV422, YUYV */
 		fmt = 0x30;
-- 
2.35.0

[PATCH 20/21] media: ov5640: Restrict sizes to mbus code

[Jacopo Mondi]

The ov5640 driver supports different sizes for different mbus_codes.
In particular:

- 8bpp modes: high resolution sizes (>= 1280x720)
- 16bpp modes: all sizes
- 24bpp modes: low resolutions sizes (< 1280x720)

Restrict the frame sizes enumerations to the above constraints.

While at it, make sure the fse->mbus_code parameter is valid, and return
-EINVAL if it's not.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 5891eaab05ee..2978dabd1d54 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -3288,14 +3288,28 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_frame_size_enum *fse)
 {
+	u32 bpp = ov5640_code_to_bpp(fse->code);
+	unsigned int index = fse->index;
+
 	if (fse->pad != 0)
 		return -EINVAL;
-	if (fse->index >= OV5640_NUM_MODES)
+	if (!bpp)
+		return -EINVAL;
+
+	/* Only low-resolution modes are supported for 24bpp modes. */
+	if (bpp == 24 && index >= OV5640_MODE_720P_1280_720)
+		return -EINVAL;
+
+	/* Only high-resolutions modes are supported for 8bpp formats. */
+	if (bpp == 8)
+		index += OV5640_MODE_720P_1280_720;
+
+	if (index >= OV5640_NUM_MODES)
 		return -EINVAL;
 
-	fse->min_width = ov5640_mode_data[fse->index].crop.width;
+	fse->min_width = ov5640_mode_data[index].crop.width;
 	fse->max_width = fse->min_width;
-	fse->min_height = ov5640_mode_data[fse->index].crop.height;
+	fse->min_height = ov5640_mode_data[index].crop.height;
 	fse->max_height = fse->min_height;
 
 	return 0;
-- 
2.35.0

[PATCH 21/21] media: ov5640: Adjust format to bpp in s_fmt

[Jacopo Mondi]

The ov5640 driver supports different sizes for different mbus_codes.
In particular:

- 8bpp modes: high resolution sizes (>= 1280x720)
- 16bpp modes: all sizes
- 24bpp modes: low resolutions sizes (< 1280x720)

Adjust the image sizes according to the above constraints.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/ov5640.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 2978dabd1d54..49d0df80f71a 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -2529,6 +2529,7 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
 				   enum ov5640_frame_rate fr,
 				   const struct ov5640_mode_info **new_mode)
 {
+	unsigned int bpp = ov5640_code_to_bpp(fmt->code);
 	struct ov5640_dev *sensor = to_ov5640_dev(sd);
 	const struct ov5640_mode_info *mode;
 	int i;
@@ -2536,6 +2537,17 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
 	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
 	if (!mode)
 		return -EINVAL;
+
+	/*
+	 * Adjust mode according to bpp:
+	 * - 8bpp modes work for resolution >= 1280x720
+	 * - 24bpp modes work resolution < 1280x720
+	 */
+	if (bpp == 8 && mode->crop.width < 1280)
+		mode = &ov5640_mode_data[OV5640_MODE_720P_1280_720];
+	else if (bpp == 24 && mode->crop.width > 1024)
+		mode = &ov5640_mode_data[OV5640_MODE_XGA_1024_768];
+
 	fmt->width = mode->crop.width;
 	fmt->height = mode->crop.height;
 
-- 
2.35.0

Re: [PATCH 02/21] media: ov5604: Re-arrange modes definition

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:26PM +0100, Jacopo Mondi wrote:
> The array of supported modes is close to unreadable.
> Re-arrange it giving it some room to breath.
> 
> Cosmetic change only.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 141 +++++++++++++++++++++----------------
>  1 file changed, 81 insertions(+), 60 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index d915c9652302..7e7732f30486 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -615,66 +615,87 @@ static const struct ov5640_mode_info ov5640_mode_init_data = {
>  
>  static const struct ov5640_mode_info
>  ov5640_mode_data[OV5640_NUM_MODES] = {
> -	{OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_48M,
> -	 160, 1896, 120, 984,
> -	 ov5640_setting_QQVGA_160_120,
> -	 ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_48M,
> -	 176, 1896, 144, 984,
> -	 ov5640_setting_QCIF_176_144,
> -	 ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_48M,
> -	 320, 1896, 240, 984,
> -	 ov5640_setting_QVGA_320_240,
> -	 ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_VGA_640_480, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_48M,
> -	 640, 1896, 480, 1080,
> -	 ov5640_setting_VGA_640_480,
> -	 ARRAY_SIZE(ov5640_setting_VGA_640_480),
> -	 OV5640_60_FPS},
> -	{OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_96M,
> -	 720, 1896, 480, 984,
> -	 ov5640_setting_NTSC_720_480,
> -	 ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> -	OV5640_30_FPS},
> -	{OV5640_MODE_PAL_720_576, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_96M,
> -	 720, 1896, 576, 984,
> -	 ov5640_setting_PAL_720_576,
> -	 ARRAY_SIZE(ov5640_setting_PAL_720_576),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_96M,
> -	 1024, 1896, 768, 1080,
> -	 ov5640_setting_XGA_1024_768,
> -	 ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_720P_1280_720, SUBSAMPLING,
> -	 OV5640_PIXEL_RATE_124M,
> -	 1280, 1892, 720, 740,
> -	 ov5640_setting_720P_1280_720,
> -	 ARRAY_SIZE(ov5640_setting_720P_1280_720),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_1080P_1920_1080, SCALING,
> -	 OV5640_PIXEL_RATE_148M,
> -	 1920, 2500, 1080, 1120,
> -	 ov5640_setting_1080P_1920_1080,
> -	 ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> -	 OV5640_30_FPS},
> -	{OV5640_MODE_QSXGA_2592_1944, SCALING,
> -	 OV5640_PIXEL_RATE_168M,
> -	 2592, 2844, 1944, 1968,
> -	 ov5640_setting_QSXGA_2592_1944,
> -	 ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> -	 OV5640_15_FPS},
> +	{
> +		/* 160x120 */
> +		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_48M,
> +		160, 1896, 120, 984,
> +		ov5640_setting_QQVGA_160_120,
> +		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> +		OV5640_30_FPS

You can add a comma at the end here while at it.

That's definitely more readable. Should we go one step further?

		.id = OV5640_MODE_QQVGA_160_120,
		.dn_mode = SUBSAMPLING,
		.pixel_rate = OV5640_PIXEL_RATE_48M,
		.hact = 160,
		.htot = 1896,
		.vact = 120,
		.vtot = 984,
		.reg_data = ov5640_setting_QQVGA_160_120,
		.reg_data_szize = ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
		.max_fps = OV5640_30_FPS,

Up to you.

I trust that you've automated this change and that if the first entry is
correct, all the other ones are as well :-)

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	}, {
> +		/* 176x144 */
> +		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_48M,
> +		176, 1896, 144, 984,
> +		ov5640_setting_QCIF_176_144,
> +		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> +		OV5640_30_FPS
> +	}, {
> +		/* 320x240 */
> +		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_48M,
> +		320, 1896, 240, 984,
> +		ov5640_setting_QVGA_320_240,
> +		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> +		OV5640_30_FPS
> +	}, {
> +		/* 640x480 */
> +		OV5640_MODE_VGA_640_480, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_48M,
> +		640, 1896, 480, 1080,
> +		ov5640_setting_VGA_640_480,
> +		ARRAY_SIZE(ov5640_setting_VGA_640_480),
> +		OV5640_60_FPS
> +	}, {
> +		/* 720x480 */
> +		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_96M,
> +		720, 1896, 480, 984,
> +		ov5640_setting_NTSC_720_480,
> +		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> +		OV5640_30_FPS
> +	}, {
> +		/* 720x576 */
> +		OV5640_MODE_PAL_720_576, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_96M,
> +		720, 1896, 576, 984,
> +		ov5640_setting_PAL_720_576,
> +		ARRAY_SIZE(ov5640_setting_PAL_720_576),
> +		OV5640_30_FPS
> +	}, {
> +		/* 1024x768 */
> +		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_96M,
> +		1024, 1896, 768, 1080,
> +		ov5640_setting_XGA_1024_768,
> +		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> +		OV5640_30_FPS
> +	}, {
> +		/* 1280x720 */
> +		OV5640_MODE_720P_1280_720, SUBSAMPLING,
> +		OV5640_PIXEL_RATE_124M,
> +		1280, 1892, 720, 740,
> +		ov5640_setting_720P_1280_720,
> +		ARRAY_SIZE(ov5640_setting_720P_1280_720),
> +		OV5640_30_FPS
> +	}, {
> +		/* 1920x1080 */
> +		OV5640_MODE_1080P_1920_1080, SCALING,
> +		OV5640_PIXEL_RATE_148M,
> +		1920, 2500, 1080, 1120,
> +		ov5640_setting_1080P_1920_1080,
> +		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> +		OV5640_30_FPS
> +	}, {
> +		/* 2592x1944 */
> +		OV5640_MODE_QSXGA_2592_1944, SCALING,
> +		OV5640_PIXEL_RATE_168M,
> +		2592, 2844, 1944, 1968,
> +		ov5640_setting_QSXGA_2592_1944,
> +		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> +		OV5640_15_FPS
> +	},
>  };
>  
>  static int ov5640_init_slave_id(struct ov5640_dev *sensor)

-- 
Regards,

Laurent Pinchart

Re: [PATCH 03/21] media: ov5640: Add is_mipi() function

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

s/is_mipi/ov5640_is_mipi/

On Mon, Jan 31, 2022 at 03:32:27PM +0100, Jacopo Mondi wrote:
> Checking if the sensor is used in DVP or MIPI mode is a repeated
> pattern which is about to be repeated more often.
> 
> Provide an inline function to shortcut that.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 7e7732f30486..fc3e4f61709c 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -310,6 +310,11 @@ static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
>  			     ctrls.handler)->sd;
>  }
>  
> +static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)

I'd name is ov5640_is_csi2() as MIPI is more than just CSI-2. Up to you.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +{
> +	return sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY;
> +}
> +
>  /*
>   * FIXME: all of these register tables are likely filled with
>   * entries that set the register to their power-on default values,
> @@ -1224,7 +1229,7 @@ static int ov5640_load_regs(struct ov5640_dev *sensor,
>  		/* remain in power down mode for DVP */
>  		if (regs->reg_addr == OV5640_REG_SYS_CTRL0 &&
>  		    val == OV5640_REG_SYS_CTRL0_SW_PWUP &&
> -		    sensor->ep.bus_type != V4L2_MBUS_CSI2_DPHY)
> +		    !ov5640_is_mipi(sensor))
>  			continue;
>  
>  		if (mask)
> @@ -1859,7 +1864,7 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
>  	 * the same rate than YUV, so we can just use 16 bpp all the time.
>  	 */
>  	rate = ov5640_calc_pixel_rate(sensor) * 16;
> -	if (sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY) {
> +	if (ov5640_is_mipi(sensor)) {
>  		rate = rate / sensor->ep.bus.mipi_csi2.num_data_lanes;
>  		ret = ov5640_set_mipi_pclk(sensor, rate);
>  	} else {
> @@ -3042,7 +3047,7 @@ static int ov5640_s_stream(struct v4l2_subdev *sd, int enable)
>  			sensor->pending_fmt_change = false;
>  		}
>  
> -		if (sensor->ep.bus_type == V4L2_MBUS_CSI2_DPHY)
> +		if (ov5640_is_mipi(sensor))
>  			ret = ov5640_set_stream_mipi(sensor, enable);
>  		else
>  			ret = ov5640_set_stream_dvp(sensor, enable);

-- 
Regards,

Laurent Pinchart

Re: [PATCH 04/21] media: ov5640: Associate bpp with formats

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:28PM +0100, Jacopo Mondi wrote:
> Associate the bit depth to each format supported by the sensor.
> 
> The bpp will be used to calculate the line length.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 39 ++++++++++++++++++++++++--------------
>  1 file changed, 25 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index fc3e4f61709c..8322b99eb2b7 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -165,24 +165,35 @@ enum ov5640_format_mux {
>  	OV5640_FMT_MUX_RAW_CIP,
>  };
>  
> -struct ov5640_pixfmt {
> +static const struct ov5640_pixfmt {
>  	u32 code;
>  	u32 colorspace;
> +	u8 bpp;
> +} ov5640_formats[] = {
> +	{ MEDIA_BUS_FMT_JPEG_1X8, V4L2_COLORSPACE_JPEG, 16},

s/}/ }/

and possibly

	{
		.code = MEDIA_BUS_FMT_JPEG_1X8,
		.colorspace = V4L2_COLORSPACE_JPEG,
		.bpp = 16,
	},

One day we'll have centralized data...

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	{ MEDIA_BUS_FMT_UYVY8_2X8, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_UYVY8_1X16, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_YUYV8_2X8, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, 16,},
> +	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
> +	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
> +	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
> +	{ MEDIA_BUS_FMT_SRGGB8_1X8, V4L2_COLORSPACE_SRGB, 8},
>  };
>  
> -static const struct ov5640_pixfmt ov5640_formats[] = {
> -	{ MEDIA_BUS_FMT_JPEG_1X8, V4L2_COLORSPACE_JPEG, },
> -	{ MEDIA_BUS_FMT_UYVY8_2X8, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_UYVY8_1X16, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_YUYV8_2X8, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, },
> -	{ MEDIA_BUS_FMT_SRGGB8_1X8, V4L2_COLORSPACE_SRGB, },
> -};
> +static u32 ov5640_code_to_bpp(u32 code)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(ov5640_formats); ++i) {
> +		if (ov5640_formats[i].code == code)
> +			return ov5640_formats[i].bpp;
> +	}
> +
> +	return 0;
> +}
>  
>  /*
>   * FIXME: remove this when a subdev API becomes available

-- 
Regards,

Laurent Pinchart

Re: [PATCH 01/21] media: ov5640: Add pixel rate to modes

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:25PM +0100, Jacopo Mondi wrote:
> Add to each mode supported by the sensor the ideal pixel rate, as
> defined by Table 2.1 in the chip manual.
> 
> The ideal pixel rate will be used to compute the MIPI CSI-2 clock tree.
> 
> Create the V4L2_CID_LINK_FREQ control and define the enumeration
> of possible MIPI CSI-2 link frequencies from the list of supported
> pixel clocks.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 66 ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 63 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index db5a19babe67..d915c9652302 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -118,6 +118,44 @@ enum ov5640_frame_rate {
>  	OV5640_NUM_FRAMERATES,
>  };
>  
> +enum ov5640_pixel_rate_id {
> +	OV5640_PIXEL_RATE_168M,
> +	OV5640_PIXEL_RATE_148M,
> +	OV5640_PIXEL_RATE_124M,
> +	OV5640_PIXEL_RATE_96M,
> +	OV5640_PIXEL_RATE_48M,
> +	OV5640_NUM_PIXEL_RATES,
> +};
> +
> +/*
> + * The chip manual suggests 24/48/96/192 MHz pixel clocks.
> + *
> + * 192MHz exceeds the sysclk limits; use 168MHz as maximum pixel rate for
> + * full resolution mode @15 FPS.
> + */
> +static const u32 ov5640_pixel_rates[] = {
> +	[OV5640_PIXEL_RATE_168M] = 168000000,
> +	[OV5640_PIXEL_RATE_148M] = 148000000,
> +	[OV5640_PIXEL_RATE_124M] = 124000000,
> +	[OV5640_PIXEL_RATE_96M] = 96000000,
> +	[OV5640_PIXEL_RATE_48M] = 48000000,
> +};
> +
> +/*
> + * MIPI CSI-2 link frequencies.
> + *
> + * Derived from the above defined pixel rate for bpp = (8, 16, 24) and
> + * data_lanes = (1, 2)
> + *
> + * link_freq = (pixel_rate * bpp) / (2 * data_lanes)
> + */
> +static const s64 ov5640_csi2_link_freqs[] = {
> +	992000000, 888000000, 768000000, 744000000, 672000000, 672000000,
> +	592000000, 592000000, 576000000, 576000000, 496000000, 496000000,
> +	384000000, 384000000, 384000000, 336000000, 296000000, 288000000,
> +	248000000, 192000000, 192000000, 192000000, 96000000,
> +};
> +
>  enum ov5640_format_mux {
>  	OV5640_FMT_MUX_YUV422 = 0,
>  	OV5640_FMT_MUX_RGB,
> @@ -189,6 +227,7 @@ struct reg_value {
>  struct ov5640_mode_info {
>  	enum ov5640_mode_id id;
>  	enum ov5640_downsize_mode dn_mode;
> +	enum ov5640_pixel_rate_id pixel_rate;
>  	u32 hact;
>  	u32 htot;
>  	u32 vact;
> @@ -201,6 +240,7 @@ struct ov5640_mode_info {
>  struct ov5640_ctrls {
>  	struct v4l2_ctrl_handler handler;
>  	struct v4l2_ctrl *pixel_rate;
> +	struct v4l2_ctrl *link_freq;
>  	struct {
>  		struct v4l2_ctrl *auto_exp;
>  		struct v4l2_ctrl *exposure;
> @@ -565,7 +605,9 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
>  
>  /* power-on sensor init reg table */
>  static const struct ov5640_mode_info ov5640_mode_init_data = {
> -	0, SUBSAMPLING, 640, 1896, 480, 984,
> +	0, SUBSAMPLING,
> +	OV5640_PIXEL_RATE_96M,
> +	640, 1896, 480, 984,
>  	ov5640_init_setting_30fps_VGA,
>  	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
>  	OV5640_30_FPS,
> @@ -574,51 +616,61 @@ static const struct ov5640_mode_info ov5640_mode_init_data = {
>  static const struct ov5640_mode_info
>  ov5640_mode_data[OV5640_NUM_MODES] = {
>  	{OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_48M,

I would have moved this after 02/21 :-) No need to change that though,
the rebase conflict isn't worth it.

>  	 160, 1896, 120, 984,
>  	 ov5640_setting_QQVGA_160_120,
>  	 ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_48M,
>  	 176, 1896, 144, 984,
>  	 ov5640_setting_QCIF_176_144,
>  	 ARRAY_SIZE(ov5640_setting_QCIF_176_144),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_48M,
>  	 320, 1896, 240, 984,
>  	 ov5640_setting_QVGA_320_240,
>  	 ARRAY_SIZE(ov5640_setting_QVGA_320_240),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_VGA_640_480, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_48M,
>  	 640, 1896, 480, 1080,
>  	 ov5640_setting_VGA_640_480,
>  	 ARRAY_SIZE(ov5640_setting_VGA_640_480),
>  	 OV5640_60_FPS},
>  	{OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_96M,
>  	 720, 1896, 480, 984,
>  	 ov5640_setting_NTSC_720_480,
>  	 ARRAY_SIZE(ov5640_setting_NTSC_720_480),
>  	OV5640_30_FPS},
>  	{OV5640_MODE_PAL_720_576, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_96M,
>  	 720, 1896, 576, 984,
>  	 ov5640_setting_PAL_720_576,
>  	 ARRAY_SIZE(ov5640_setting_PAL_720_576),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_96M,
>  	 1024, 1896, 768, 1080,
>  	 ov5640_setting_XGA_1024_768,
>  	 ARRAY_SIZE(ov5640_setting_XGA_1024_768),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_720P_1280_720, SUBSAMPLING,
> +	 OV5640_PIXEL_RATE_124M,
>  	 1280, 1892, 720, 740,
>  	 ov5640_setting_720P_1280_720,
>  	 ARRAY_SIZE(ov5640_setting_720P_1280_720),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_1080P_1920_1080, SCALING,
> +	 OV5640_PIXEL_RATE_148M,
>  	 1920, 2500, 1080, 1120,
>  	 ov5640_setting_1080P_1920_1080,
>  	 ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
>  	 OV5640_30_FPS},
>  	{OV5640_MODE_QSXGA_2592_1944, SCALING,
> +	 OV5640_PIXEL_RATE_168M,
>  	 2592, 2844, 1944, 1968,
>  	 ov5640_setting_QSXGA_2592_1944,
>  	 ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> @@ -2743,6 +2795,7 @@ static const struct v4l2_ctrl_ops ov5640_ctrl_ops = {
>  
>  static int ov5640_init_controls(struct ov5640_dev *sensor)
>  {
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
>  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
>  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> @@ -2755,8 +2808,14 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  
>  	/* Clock related controls */
>  	ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_PIXEL_RATE,
> -					      0, INT_MAX, 1,
> -					      ov5640_calc_pixel_rate(sensor));
> +				      ov5640_pixel_rates[OV5640_NUM_PIXEL_RATES - 1],
> +				      ov5640_pixel_rates[0], 1,
> +				      ov5640_pixel_rates[mode->pixel_rate]);
> +
> +	ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, ops,
> +					V4L2_CID_LINK_FREQ,
> +					ARRAY_SIZE(ov5640_csi2_link_freqs) - 1,
> +					4, ov5640_csi2_link_freqs);

I'd add the control in patch 05/21, as it's not very usable until then.

>  
>  	/* Auto/manual white balance */
>  	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
> @@ -2806,6 +2865,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  	}
>  
>  	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;

While a read-only control isn't optimal, I think the series goes in the
right direction, so I'm OK with it.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  	ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
>  	ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/21] media: ov5640: Update pixel_rate and link_freq

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:29PM +0100, Jacopo Mondi wrote:
> After having set a new format re-calculate the pixel_rate and link_freq
> control values and update them when in MIPI mode.
> 
> Take into account the limitation of the link frequency having to be
> strictly smaller than 1GHz when computing the desired link_freq, and
> adjust the resulting pixel_rate acounting for the clock tree
> configuration.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 63 ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 61 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 8322b99eb2b7..457f76030163 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -2375,6 +2375,65 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> +static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> +{
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
> +	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
> +	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
> +	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;

As this is only valid for CSI-2, I'd move the initialization of the
variable after the !ov5640_is_mipi() check.

> +	unsigned int i = 0;
> +	u32 pixel_rate;
> +	s64 link_freq;
> +	u32 bpp;
> +
> +	/*
> +	 * Update the pixel rate control value.
> +	 *
> +	 * For DVP mode, maintain the pixel rate calculation using fixed FPS.
> +	 */
> +	if (!ov5640_is_mipi(sensor)) {
> +		__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate,
> +					 ov5640_calc_pixel_rate(sensor));
> +
> +		return 0;
> +	}
> +
> +	/*
> +	 * The MIPI CSI-2 link frequency should comply with the CSI-2
> +	 * specifications and be lower than 1GHz.

s/specifications/specification/

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	 *
> +	 * Start from the suggested pixel_rate for the current mode and
> +	 * progressively slow it down if it exceeds 1GHz.
> +	 */
> +	bpp = ov5640_code_to_bpp(fmt->code);
> +	do {
> +		pixel_rate = ov5640_pixel_rates[pixel_rate_id];
> +		link_freq = pixel_rate * bpp / (2 * num_lanes);
> +	} while (link_freq >= 1000000000U &&
> +		 ++pixel_rate_id < OV5640_NUM_PIXEL_RATES);
> +
> +	/*
> +	 * Higher link rates require the clock tree to be programmed with
> +	 * 'mipi_div' = 1; this has the effect of halving the actual output
> +	 * pixel rate in the MIPI domain.
> +	 *
> +	 * Adjust the pixel rate control value to report it correctly to
> +	 * userspace.
> +	 */
> +	if (link_freq > OV5640_LINK_RATE_MAX)
> +		pixel_rate /= 2;
> +
> +	for (i = 0; i < ARRAY_SIZE(ov5640_csi2_link_freqs); ++i) {
> +		if (ov5640_csi2_link_freqs[i] == link_freq)
> +			break;
> +	}
> +
> +	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate, pixel_rate);
> +	__v4l2_ctrl_s_ctrl(sensor->ctrls.link_freq, i);
> +
> +	return 0;
> +}
> +
>  static int ov5640_set_fmt(struct v4l2_subdev *sd,
>  			  struct v4l2_subdev_state *sd_state,
>  			  struct v4l2_subdev_format *format)
> @@ -2414,8 +2473,8 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
>  	/* update format even if code is unchanged, resolution might change */
>  	sensor->fmt = *mbus_fmt;
>  
> -	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate,
> -				 ov5640_calc_pixel_rate(sensor));
> +	ov5640_update_pixel_rate(sensor);
> +
>  out:
>  	mutex_unlock(&sensor->lock);
>  	return ret;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 06/21] media: ov5640: Rework CSI-2 clock tree

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:30PM +0100, Jacopo Mondi wrote:
> Re-work the ov5640_set_mipi_pclk() function to calculate the
> PLL configuration using the pixel_rate and link_freq values set at
> s_fmt time.
> 
> Rework the DVP clock mode settings to calculate the pixel clock
> internally and remove the assumption on the 16bpp format.
> 
> Tested in MIPI CSI-2 mode with 2 data lanes with:
> - all the sensor supported resolutions in UYVY and RGB565 formats.
> - resolutions >= 1280x720 in RAW Bayer format.
> - resolutions < 1280x720 in RGB888 format.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> 
> ---
> 
> Not tested with JPEG.
> Not tested with 1 data lane.
> Not tested in DVP mode.
> 
> If I get come Tested-by: tags for the above use cases, it would be
> great.
> ---
>  drivers/media/i2c/ov5640.c | 175 ++++++++++++++++++++-----------------
>  1 file changed, 94 insertions(+), 81 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 457f76030163..acc636500907 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -29,6 +29,8 @@
>  #define OV5640_XCLK_MIN  6000000
>  #define OV5640_XCLK_MAX 54000000
>  
> +#define OV5640_LINK_RATE_MAX	490000000U
> +
>  #define OV5640_DEFAULT_SLAVE_ID 0x3c
>  
>  #define OV5640_REG_SYS_RESET02		0x3002
> @@ -88,6 +90,7 @@
>  #define OV5640_REG_POLARITY_CTRL00	0x4740
>  #define OV5640_REG_MIPI_CTRL00		0x4800
>  #define OV5640_REG_DEBUG_MODE		0x4814
> +#define OV5640_REG_PCLK_PERIOD		0x4837
>  #define OV5640_REG_ISP_FORMAT_MUX_CTRL	0x501f
>  #define OV5640_REG_PRE_ISP_TEST_SET1	0x503d
>  #define OV5640_REG_SDE_CTRL0		0x5580
> @@ -1035,69 +1038,80 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor,
>   * ov5640_set_mipi_pclk() - Calculate the clock tree configuration values
>   *			    for the MIPI CSI-2 output.
>   *
> - * @rate: The requested bandwidth per lane in bytes per second.
> - *	  'Bandwidth Per Lane' is calculated as:
> - *	  bpl = HTOT * VTOT * FPS * bpp / num_lanes;
> - *
> - * This function use the requested bandwidth to calculate:
> - * - sample_rate = bpl / (bpp / num_lanes);
> - *	         = bpl / (PLL_RDIV * BIT_DIV * PCLK_DIV * MIPI_DIV / num_lanes);
> - *
> - * - mipi_sclk   = bpl / MIPI_DIV / 2; ( / 2 is for CSI-2 DDR)

It would be useful to have the update formula either here or in the code
below.

> - *
> - * with these fixed parameters:
> - *	PLL_RDIV	= 2;
> - *	BIT_DIVIDER	= 2; (MIPI_BIT_MODE == 8 ? 2 : 2,5);
> - *	PCLK_DIV	= 1;
> - *
> - * The MIPI clock generation differs for modes that use the scaler and modes
> - * that do not. In case the scaler is in use, the MIPI_SCLK generates the MIPI
> - * BIT CLk, and thus:
> - *
> - * - mipi_sclk = bpl / MIPI_DIV / 2;
> - *   MIPI_DIV = 1;
> - *
> - * For modes that do not go through the scaler, the MIPI BIT CLOCK is generated
> - * from the pixel clock, and thus:
> - *
> - * - sample_rate = bpl / (bpp / num_lanes);
> - *	         = bpl / (2 * 2 * 1 * MIPI_DIV / num_lanes);
> - *		 = bpl / (4 * MIPI_DIV / num_lanes);
> - * - MIPI_DIV	 = bpp / (4 * num_lanes);
> - *
> - * FIXME: this have been tested with 16bpp and 2 lanes setup only.
> - * MIPI_DIV is fixed to value 2, but it -might- be changed according to the
> - * above formula for setups with 1 lane or image formats with different bpp.
> - *
> - * FIXME: this deviates from the sensor manual documentation which is quite
> - * thin on the MIPI clock tree generation part.
> + * FIXME: tested with 2 lanes only.
>   */
> -static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
> -				unsigned long rate)
> +static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor)
>  {
> -	const struct ov5640_mode_info *mode = sensor->current_mode;
> +	u8 bit_div, mipi_div, pclk_div, sclk_div, sclk2x_div, root_div;
> +	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
>  	u8 prediv, mult, sysdiv;
> -	u8 mipi_div;
> +	unsigned long sysclk;
> +	unsigned long sample_rate;
> +	u8 pclk_period;
> +	s64 link_freq;
>  	int ret;
>  
> +	/* Use the link frequency computed at s_fmt time. */
> +	link_freq = ov5640_csi2_link_freqs[sensor->ctrls.link_freq->cur.val];
> +
>  	/*
> -	 * 1280x720 is reported to use 'SUBSAMPLING' only,
> -	 * but according to the sensor manual it goes through the
> -	 * scaler before subsampling.
> +	 * - mipi_div - Additional divider for the MIPI lane clock.
> +	 *
> +	 * Higher link frequencies would make sysclk > 1GHz.
> +	 * Keep the sysclk low and do not divide in the MIPI domain.
>  	 */
> -	if (mode->dn_mode == SCALING ||
> -	   (mode->id == OV5640_MODE_720P_1280_720))
> -		mipi_div = OV5640_MIPI_DIV_SCLK;
> +	if (link_freq > OV5640_LINK_RATE_MAX)
> +		mipi_div = 1;
>  	else
> -		mipi_div = OV5640_MIPI_DIV_PCLK;
> +		mipi_div = 2;
>  
> -	ov5640_calc_sys_clk(sensor, rate, &prediv, &mult, &sysdiv);
> +	sysclk = link_freq * mipi_div;
> +	ov5640_calc_sys_clk(sensor, sysclk, &prediv, &mult, &sysdiv);
>  
> -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0,
> -			     0x0f, OV5640_PLL_CTRL0_MIPI_MODE_8BIT);
> +	/*
> +	 * Adjust PLL parameters to maintain the MIPI_SCLK-to-PCLK ratio;
> +	 *
> +	 * - root_div = 2 (fixed)
> +	 * - bit_div : MIPI 8-bit = 2
> +	 *	       MIPI 10-bit = 2,5

s/2,5/2.5/

> +	 * - pclk_div = 1 (fixed)
> +	 * - pll_div  = (2 lanes ? mipi_div : 2 * mipi_div)

What's pll_div here ? There's no corresponding variable. It would be
nice to name it according to the clock tree diagram located above in the
driver.

> +	 *   2 lanes: MIPI_SCLK = (4 or 5) * PCLK
> +	 *   1 lanes: MIPI_SCLK = (8 or 10) * PCLK
> +	 *
> +	 * TODO: support 10-bit formats
> +	 * TODO: test with 1 data lane
> +	 */
> +	root_div = OV5640_PLL_CTRL3_PLL_ROOT_DIV_2;
> +	bit_div =  OV5640_PLL_CTRL0_MIPI_MODE_8BIT;
> +	pclk_div = OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS;
>  
> -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1,
> -			     0xff, sysdiv << 4 | mipi_div);
> +	/*
> +	 * Scaler clock:
> +	 * - YUV: PCLK >= 2 * SCLK
> +	 * - RAW or JPEG: PCLK >= SCLK
> +	 * - sclk2x_div = sclk_div / 2
> +	 *
> +	 * TODO: test with JPEG.
> +	 */
> +	sclk_div = ilog2(OV5640_SCLK_ROOT_DIV);
> +	sclk2x_div = ilog2(OV5640_SCLK2X_ROOT_DIV);
> +
> +	/*
> +	 * Set the sample period expressed in ns with 1-bit decimal
> +	 * (0x01=0.5ns).
> +	 */
> +	sample_rate = ov5640_pixel_rates[sensor->current_mode->pixel_rate]
> +		    * (ov5640_code_to_bpp(fmt->code) / 8);

Won't this cause rouding errors for bpp values that are not multiples of
8 ?

> +	pclk_period = 2000000000U / sample_rate;
> +
> +	/* Program the clock tree registers. */
> +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0, 0x0f, bit_div);
> +	if (ret)
> +		return ret;
> +
> +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1, 0xff,
> +			     sysdiv << 4 | mipi_div);

Here you don't use parentheses around (sysdiv << 4), and below you do
when writing OV5640_REG_SYS_ROOT_DIVIDER. They're not necessary, but it
could be clearer to add parentheses here.

>  	if (ret)
>  		return ret;
>  
> @@ -1105,13 +1119,27 @@ static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
>  	if (ret)
>  		return ret;
>  
> -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3,
> -			     0x1f, OV5640_PLL_CTRL3_PLL_ROOT_DIV_2 | prediv);
> +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3, 0x1f,
> +			     root_div | prediv);
>  	if (ret)
>  		return ret;
>  
> -	return ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER,
> -			      0x30, OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS);
> +	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
> +			     (pclk_div << 4) | (sclk2x_div << 2) | sclk_div);
> +	if (ret)
> +		return ret;
> +
> +	return ov5640_write_reg(sensor, OV5640_REG_PCLK_PERIOD, pclk_period);
> +}
> +
> +static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> +{
> +	u64 rate;
> +
> +	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> +	rate *= ov5640_framerates[sensor->current_fr];
> +
> +	return rate;
>  }
>  
>  static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
> @@ -1131,11 +1159,16 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
>  	return _rate / *pll_rdiv / *bit_div / *pclk_div;
>  }
>  
> -static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate)
> +static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor)
>  {
>  	u8 prediv, mult, sysdiv, pll_rdiv, bit_div, pclk_div;
> +	u64 rate;

Do you need a 64-bit integer here, or can the calculation fit in 32-bit?

>  	int ret;
>  
> +	rate = ov5640_calc_pixel_rate(sensor);
> +	rate *= ov5640_code_to_bpp(sensor->fmt.code);
> +	do_div(rate, sensor->ep.bus.parallel.bus_width);
> +
>  	ov5640_calc_pclk(sensor, rate, &prediv, &mult, &sysdiv, &pll_rdiv,
>  			 &bit_div, &pclk_div);
>  
> @@ -1660,16 +1693,6 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
>  	return mode;
>  }
>  
> -static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> -{
> -	u64 rate;
> -
> -	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> -	rate *= ov5640_framerates[sensor->current_fr];
> -
> -	return rate;
> -}
> -
>  /*
>   * sensor changes between scaling and subsampling, go through
>   * exposure calculation
> @@ -1851,7 +1874,6 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
>  	enum ov5640_downsize_mode dn_mode, orig_dn_mode;
>  	bool auto_gain = sensor->ctrls.auto_gain->val == 1;
>  	bool auto_exp =  sensor->ctrls.auto_exp->val == V4L2_EXPOSURE_AUTO;
> -	unsigned long rate;
>  	int ret;
>  
>  	dn_mode = mode->dn_mode;
> @@ -1870,19 +1892,10 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
>  			goto restore_auto_gain;
>  	}
>  
> -	/*
> -	 * All the formats we support have 16 bits per pixel, seems to require
> -	 * the same rate than YUV, so we can just use 16 bpp all the time.
> -	 */
> -	rate = ov5640_calc_pixel_rate(sensor) * 16;
> -	if (ov5640_is_mipi(sensor)) {
> -		rate = rate / sensor->ep.bus.mipi_csi2.num_data_lanes;
> -		ret = ov5640_set_mipi_pclk(sensor, rate);
> -	} else {
> -		rate = rate / sensor->ep.bus.parallel.bus_width;
> -		ret = ov5640_set_dvp_pclk(sensor, rate);
> -	}
> -
> +	if (ov5640_is_mipi(sensor))
> +		ret = ov5640_set_mipi_pclk(sensor);
> +	else
> +		ret = ov5640_set_dvp_pclk(sensor);
>  	if (ret < 0)
>  		return 0;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 07/21] media: ov5640: Rework timings programming

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:31PM +0100, Jacopo Mondi wrote:
> The current definition of a sensor mode defines timings as follows:
> 
> - hact, vact: Visible width and height
> - htot, vtot: Total sizes invluding blankings

s/invluding/including/

> This makes difficult to clearly separate the visible sizes from the
> blankings and to make the vertical blanking programmable.
> 
> Rework the sensor modes sizes definition to:
> - Report the analog crop sizes
> - Report the visible crop size
> - Report the total pixels per line as HBLANK is fixed
> - Report the VBLANK value to make it programmable
> 
> Also modify the ov5640_set_timings() function to program all the
> windowing registers are remove them from the per-mode register-value
> tables.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 451 +++++++++++++++++++++++++------------
>  1 file changed, 304 insertions(+), 147 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index acc636500907..bd14e2ad22f6 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -61,10 +61,16 @@
>  #define OV5640_REG_AEC_PK_MANUAL	0x3503
>  #define OV5640_REG_AEC_PK_REAL_GAIN	0x350a
>  #define OV5640_REG_AEC_PK_VTS		0x350c
> +#define OV5640_REG_TIMING_HS		0x3800
> +#define OV5640_REG_TIMING_VS		0x3802
> +#define OV5640_REG_TIMING_HW		0x3804
> +#define OV5640_REG_TIMING_VH		0x3806
>  #define OV5640_REG_TIMING_DVPHO		0x3808
>  #define OV5640_REG_TIMING_DVPVO		0x380a
>  #define OV5640_REG_TIMING_HTS		0x380c
>  #define OV5640_REG_TIMING_VTS		0x380e
> +#define OV5640_REG_TIMING_HOFFS		0x3810
> +#define OV5640_REG_TIMING_VOFFS		0x3812
>  #define OV5640_REG_TIMING_TC_REG20	0x3820
>  #define OV5640_REG_TIMING_TC_REG21	0x3821
>  #define OV5640_REG_AEC_CTRL00		0x3a00
> @@ -242,12 +248,17 @@ struct ov5640_mode_info {
>  	enum ov5640_mode_id id;
>  	enum ov5640_downsize_mode dn_mode;
>  	enum ov5640_pixel_rate_id pixel_rate;
> -	u32 hact;
> -	u32 htot;
> -	u32 vact;
> -	u32 vtot;
> +	/*  Analog crop rectangle. */
> +	struct v4l2_rect analog_crop;
> +	/* Visibile crop: from analog crop top-left corner. */
> +	struct v4l2_rect crop;
> +	/* Total pixels per line: crop.width + fixed hblank. */
> +	u32 ppl;

ppl is a fairly uncommon term when it comes to sensor configuration. I'd
rather keep hact, or use a hblank value.

> +	/* Total frame height = crop.height + vblank. */
> +	u32 vblank_def;

The comment doesn't seem to match the field.

>  	const struct reg_value *reg_data;
>  	u32 reg_data_size;
> +	/* DVP only; ignored in MIPI mode. */
>  	u32 max_fps;
>  };
>  
> @@ -353,11 +364,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
>  	{0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -425,11 +432,7 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -444,11 +447,7 @@ static const struct reg_value ov5640_setting_XGA_1024_768[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -463,11 +462,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -482,11 +477,7 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -500,11 +491,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -557,11 +536,7 @@ static const struct reg_value ov5640_setting_720P_1280_720[] = {
>  	{0x3c07, 0x07, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> +	{0x3815, 0x31, 0, 0},
>  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
>  	{0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
> @@ -576,11 +551,7 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x11, 0, 0},
> -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> +	{0x3815, 0x11, 0, 0},
>  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
>  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -591,9 +562,6 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
>  	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0},
>  	{0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> -	{0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
> -	{0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
> -	{0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0},
>  	{0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
>  	{0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
>  	{0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
> @@ -607,11 +575,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x11, 0, 0},
> -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> -	{0x3810, 0x00, 0, 0},
> -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> +	{0x3815, 0x11, 0, 0},
>  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
>  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
>  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> @@ -624,96 +588,250 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
>  
>  /* power-on sensor init reg table */
>  static const struct ov5640_mode_info ov5640_mode_init_data = {
> -	0, SUBSAMPLING,
> -	OV5640_PIXEL_RATE_96M,
> -	640, 1896, 480, 984,
> -	ov5640_init_setting_30fps_VGA,
> -	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> -	OV5640_30_FPS,
> +		.id		= 0,

Ah there we go, named field initializers :-) It would be nice to move
them to patch 02/21 to minimize the changes here.

> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,

I don't think that's right. The TIMING_HW and TIMING_VH registers store
the end X and Y values respectively, so here width should be 2624 and
height 1944. Same below, and you'll need to update the register writes
accordingly (left + width - 1 and top + height - 1).

> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 640,
> +			.height	= 480,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 504,
> +		.reg_data	= ov5640_init_setting_30fps_VGA,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> +		.max_fps	= OV5640_30_FPS
>  };
>  
>  static const struct ov5640_mode_info
>  ov5640_mode_data[OV5640_NUM_MODES] = {
>  	{
>  		/* 160x120 */
> -		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_48M,
> -		160, 1896, 120, 984,
> -		ov5640_setting_QQVGA_160_120,
> -		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_QQVGA_160_120,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 160,
> +			.height	= 120,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 864,
> +		.reg_data	= ov5640_setting_QQVGA_160_120,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 176x144 */
> -		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_48M,
> -		176, 1896, 144, 984,
> -		ov5640_setting_QCIF_176_144,
> -		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_QCIF_176_144,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 176,
> +			.height	= 144,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 840,
> +		.reg_data	= ov5640_setting_QCIF_176_144,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 320x240 */
> -		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_48M,
> -		320, 1896, 240, 984,
> -		ov5640_setting_QVGA_320_240,
> -		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_QVGA_320_240,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 320,
> +			.height	= 240,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 744,
> +		.reg_data	= ov5640_setting_QVGA_320_240,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 640x480 */
> -		OV5640_MODE_VGA_640_480, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_48M,
> -		640, 1896, 480, 1080,
> -		ov5640_setting_VGA_640_480,
> -		ARRAY_SIZE(ov5640_setting_VGA_640_480),
> -		OV5640_60_FPS
> +		.id		= OV5640_MODE_VGA_640_480,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 640,
> +			.height	= 480,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 600,
> +		.reg_data	= ov5640_setting_VGA_640_480,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
> +		.max_fps	= OV5640_60_FPS
>  	}, {
>  		/* 720x480 */
> -		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_96M,
> -		720, 1896, 480, 984,
> -		ov5640_setting_NTSC_720_480,
> -		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_NTSC_720_480,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 56,

I think this one should be 16.

> +			.top	= 60,
> +			.width	= 720,
> +			.height	= 480,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 504,
> +		.reg_data	= ov5640_setting_NTSC_720_480,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 720x576 */
> -		OV5640_MODE_PAL_720_576, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_96M,
> -		720, 1896, 576, 984,
> -		ov5640_setting_PAL_720_576,
> -		ARRAY_SIZE(ov5640_setting_PAL_720_576),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_PAL_720_576,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 56,
> +			.top	= 60,

And this should be 6.

> +			.width	= 720,
> +			.height	= 576,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 408,
> +		.reg_data	= ov5640_setting_PAL_720_576,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 1024x768 */
> -		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_96M,
> -		1024, 1896, 768, 1080,
> -		ov5640_setting_XGA_1024_768,
> -		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_XGA_1024_768,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 4,
> +			.width	= 2623,
> +			.height	= 1947,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 6,
> +			.width	= 1024,
> +			.height	= 768,
> +		},
> +		.ppl		= 1896,
> +		.vblank_def	= 312,
> +		.reg_data	= ov5640_setting_XGA_1024_768,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 1280x720 */
> -		OV5640_MODE_720P_1280_720, SUBSAMPLING,
> -		OV5640_PIXEL_RATE_124M,
> -		1280, 1892, 720, 740,
> -		ov5640_setting_720P_1280_720,
> -		ARRAY_SIZE(ov5640_setting_720P_1280_720),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_720P_1280_720,
> +		.dn_mode	= SUBSAMPLING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_124M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 250,
> +			.width	= 2623,
> +			.height	= 1705,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 4,
> +			.width	= 1280,
> +			.height	= 720,
> +		},
> +		.ppl		= 1896,

1892 ?

> +		.vblank_def	= 20,
> +		.reg_data	= ov5640_setting_720P_1280_720,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 1920x1080 */
> -		OV5640_MODE_1080P_1920_1080, SCALING,
> -		OV5640_PIXEL_RATE_148M,
> -		1920, 2500, 1080, 1120,
> -		ov5640_setting_1080P_1920_1080,
> -		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> -		OV5640_30_FPS
> +		.id		= OV5640_MODE_1080P_1920_1080,
> +		.dn_mode	= SCALING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_148M,
> +		.analog_crop = {
> +			.left	= 336,
> +			.top	= 434,
> +			.width	= 2287,
> +			.height	= 1521,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 4,
> +			.width	= 1920,
> +			.height	= 1080,
> +		},
> +		.ppl		= 2500,
> +		.vblank_def	= 40,
> +		.reg_data	= ov5640_setting_1080P_1920_1080,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> +		.max_fps	= OV5640_30_FPS
>  	}, {
>  		/* 2592x1944 */
> -		OV5640_MODE_QSXGA_2592_1944, SCALING,
> -		OV5640_PIXEL_RATE_168M,
> -		2592, 2844, 1944, 1968,
> -		ov5640_setting_QSXGA_2592_1944,
> -		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> -		OV5640_15_FPS
> +		.id		= OV5640_MODE_QSXGA_2592_1944,
> +		.dn_mode	= SCALING,
> +		.pixel_rate	= OV5640_PIXEL_RATE_168M,
> +		.analog_crop = {
> +			.left	= 0,
> +			.top	= 0,
> +			.width	= 2623,
> +			.height	= 1951,
> +		},
> +		.crop = {
> +			.left	= 16,
> +			.top	= 4,
> +			.width	= 2592,
> +			.height	= 1944,
> +		},
> +		.ppl		= 2844,
> +		.vblank_def	= 24,
> +		.reg_data	= ov5640_setting_QSXGA_2592_1944,
> +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> +		.max_fps	= OV5640_15_FPS
>  	},
>  };
>  
> @@ -1136,7 +1254,8 @@ static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
>  {
>  	u64 rate;
>  
> -	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> +	rate = sensor->current_mode->ppl
> +	     * (sensor->current_mode->crop.height + sensor->current_mode->vblank_def);
>  	rate *= ov5640_framerates[sensor->current_fr];
>  
>  	return rate;
> @@ -1220,17 +1339,21 @@ static int ov5640_set_jpeg_timings(struct ov5640_dev *sensor,
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE, mode->hact);
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE,
> +				 mode->crop.width);
>  	if (ret < 0)
>  		return ret;
>  
> -	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE, mode->vact);
> +	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE,
> +				  mode->crop.height);
>  }
>  
>  /* download ov5640 settings to sensor through i2c */
>  static int ov5640_set_timings(struct ov5640_dev *sensor,
>  			      const struct ov5640_mode_info *mode)
>  {
> +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> +	const struct v4l2_rect *crop = &mode->crop;
>  	int ret;
>  
>  	if (sensor->fmt.code == MEDIA_BUS_FMT_JPEG_1X8) {
> @@ -1239,19 +1362,54 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
>  			return ret;
>  	}
>  
> -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HS,
> +				 analog_crop->left);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VS,
> +				 analog_crop->top);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HW,
> +				 analog_crop->width);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VH,
> +				 analog_crop->height);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, crop->width);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, crop->height);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->ppl);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> +				 crop->height + mode->vblank_def);
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HOFFS,
> +				 crop->left);
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
> +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VOFFS,
> +				 crop->top);
>  	if (ret < 0)
>  		return ret;

Maybe you could order the crop writes as the analog crop writes, with
left, top, width and height ?

>  
> -	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
> +	return 0;
>  }
>  
>  static int ov5640_load_regs(struct ov5640_dev *sensor,
> @@ -1679,11 +1837,11 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
>  
>  	mode = v4l2_find_nearest_size(ov5640_mode_data,
>  				      ARRAY_SIZE(ov5640_mode_data),
> -				      hact, vact,
> -				      width, height);
> +				      crop.width, crop.height, width, height);
>  
>  	if (!mode ||
> -	    (!nearest && (mode->hact != width || mode->vact != height)))
> +	    (!nearest &&
> +	     (mode->crop.width != width || mode->crop.height != height)))
>  		return NULL;
>  
>  	/* Check to see if the current mode exceeds the max frame rate */
> @@ -2367,8 +2525,8 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
>  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
>  	if (!mode)
>  		return -EINVAL;
> -	fmt->width = mode->hact;
> -	fmt->height = mode->vact;
> +	fmt->width = mode->crop.width;
> +	fmt->height = mode->crop.height;
>  
>  	if (new_mode)
>  		*new_mode = mode;
> @@ -2999,11 +3157,9 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
>  	if (fse->index >= OV5640_NUM_MODES)
>  		return -EINVAL;
>  
> -	fse->min_width =
> -		ov5640_mode_data[fse->index].hact;
> +	fse->min_width = ov5640_mode_data[fse->index].crop.width;
>  	fse->max_width = fse->min_width;
> -	fse->min_height =
> -		ov5640_mode_data[fse->index].vact;
> +	fse->min_height = ov5640_mode_data[fse->index].crop.height;
>  	fse->max_height = fse->min_height;
>  
>  	return 0;
> @@ -3067,15 +3223,16 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
>  	mode = sensor->current_mode;
>  
>  	frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
> -					       mode->hact, mode->vact);
> +					       mode->crop.width,
> +					       mode->crop.height);
>  	if (frame_rate < 0) {
>  		/* Always return a valid frame interval value */
>  		fi->interval = sensor->frame_interval;
>  		goto out;
>  	}
>  
> -	mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
> -				mode->vact, true);
> +	mode = ov5640_find_mode(sensor, frame_rate, mode->crop.width,
> +				mode->crop.height, true);
>  	if (!mode) {
>  		ret = -EINVAL;
>  		goto out;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 08/21] media: ov5640: Re-sort per-mode register tables

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:32PM +0100, Jacopo Mondi wrote:
> The per-mode register tables are not sorted by size. Fix it.
> 
> Cosmetic change only.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index bd14e2ad22f6..d966cca78e92 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -428,7 +428,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
>  	{0x3a1f, 0x14, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3c00, 0x04, 0, 300},
>  };
>  
> -static const struct reg_value ov5640_setting_VGA_640_480[] = {
> +static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> @@ -439,11 +439,10 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
>  	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
>  	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
>  	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
> -	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
> -	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> +	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
>  };
>  
> -static const struct reg_value ov5640_setting_XGA_1024_768[] = {
> +static const struct reg_value ov5640_setting_QCIF_176_144[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> @@ -473,7 +472,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
>  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
>  };
>  
> -static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> +static const struct reg_value ov5640_setting_VGA_640_480[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> @@ -484,10 +483,11 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
>  	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
>  	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
>  	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
> -	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
> +	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
> +	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
>  };
>  
> -static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> +static const struct reg_value ov5640_setting_NTSC_720_480[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> @@ -502,7 +502,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
>  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
>  };
>  
> -static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> +static const struct reg_value ov5640_setting_PAL_720_576[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},
> @@ -517,7 +517,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
>  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
>  };
>  
> -static const struct reg_value ov5640_setting_PAL_720_576[] = {
> +static const struct reg_value ov5640_setting_XGA_1024_768[] = {
>  	{0x3c07, 0x08, 0, 0},
>  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
>  	{0x3814, 0x31, 0, 0},

Is it me, or does this mean that ov5640_setting_XGA_1024_768,
ov5640_setting_QCIF_176_144, ov5640_setting_PAL_720_576 and
ov5640_setting_NTSC_720_480 are all identical ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH 09/21] media: ov5640: Remove ov5640_mode_init_data

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:33PM +0100, Jacopo Mondi wrote:
> The ov5640_mode_init_data is a fictional sensor more which is used to
> program the initial sensor settings.
> 
> It is only used to initialize the sensor and can be replaced
> it with a throw-away mode which just wraps the register table.
> 
> Also rename the register table to drop the format from the name to make
> it clear an actual sensor mode has to be applied after the initial
> programming.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 32 +++++---------------------------
>  1 file changed, 5 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index d966cca78e92..1e2f37c93f0d 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -348,7 +348,7 @@ static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
>   * over i2c.
>   */
>  /* YUV422 UYVY VGA@30fps */
> -static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
> +static const struct reg_value ov5640_init_setting[] = {
>  	{0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
>  	{0x3103, 0x03, 0, 0}, {0x3630, 0x36, 0, 0},
>  	{0x3631, 0x0e, 0, 0}, {0x3632, 0xe2, 0, 0}, {0x3633, 0x12, 0, 0},
> @@ -586,30 +586,6 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
>  	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
>  };
>  
> -/* power-on sensor init reg table */
> -static const struct ov5640_mode_info ov5640_mode_init_data = {
> -		.id		= 0,
> -		.dn_mode	= SUBSAMPLING,
> -		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> -		.analog_crop = {
> -			.left	= 0,
> -			.top	= 4,
> -			.width	= 2623,
> -			.height	= 1947,
> -		},
> -		.crop = {
> -			.left	= 16,
> -			.top	= 6,
> -			.width	= 640,
> -			.height	= 480,
> -		},
> -		.ppl		= 1896,
> -		.vblank_def	= 504,
> -		.reg_data	= ov5640_init_setting_30fps_VGA,
> -		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> -		.max_fps	= OV5640_30_FPS
> -};
> -
>  static const struct ov5640_mode_info
>  ov5640_mode_data[OV5640_NUM_MODES] = {
>  	{
> @@ -2117,13 +2093,15 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
>  /* restore the last set video mode after chip power-on */
>  static int ov5640_restore_mode(struct ov5640_dev *sensor)
>  {
> +	struct ov5640_mode_info init_data = {};
>  	int ret;
>  
>  	/* first load the initial register values */
> -	ret = ov5640_load_regs(sensor, &ov5640_mode_init_data);
> +	init_data.reg_data = ov5640_init_setting;
> +	init_data.reg_data_size = ARRAY_SIZE(ov5640_init_setting);
> +	ret = ov5640_load_regs(sensor, &init_data);

Could we change ov5640_load_regs() to take a reg_value array and size as
parameters, to avoid the local init_data variable ?

>  	if (ret < 0)
>  		return ret;
> -	sensor->last_mode = &ov5640_mode_init_data;
>  
>  	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
>  			     (ilog2(OV5640_SCLK2X_ROOT_DIV) << 2) |

-- 
Regards,

Laurent Pinchart

Re: [PATCH 10/21] media: ov5640: Add HBLANK control

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:34PM +0100, Jacopo Mondi wrote:
> Add the HBLANK control as read-only.
> 
> The hblank value is fixed in the mode definition and is updated
> everytime a new format is applied.

s/everytime/every time/

> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 1e2f37c93f0d..b2961e8e07c3 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -266,6 +266,7 @@ struct ov5640_ctrls {
>  	struct v4l2_ctrl_handler handler;
>  	struct v4l2_ctrl *pixel_rate;
>  	struct v4l2_ctrl *link_freq;
> +	struct v4l2_ctrl *hblank;
>  	struct {
>  		struct v4l2_ctrl *auto_exp;
>  		struct v4l2_ctrl *exposure;
> @@ -2530,6 +2531,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>  	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
>  	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
>  	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> +	unsigned int hblank;
>  	unsigned int i = 0;
>  	u32 pixel_rate;
>  	s64 link_freq;
> @@ -2580,6 +2582,10 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>  	__v4l2_ctrl_s_ctrl_int64(sensor->ctrls.pixel_rate, pixel_rate);
>  	__v4l2_ctrl_s_ctrl(sensor->ctrls.link_freq, i);
>  
> +	hblank = mode->ppl - mode->crop.width;
> +	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
> +				 hblank, hblank, 1, hblank);
> +
>  	return 0;
>  }
>  
> @@ -3044,6 +3050,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
>  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> +	unsigned int hblank;
>  	int ret;
>  
>  	v4l2_ctrl_handler_init(hdl, 32);
> @@ -3062,6 +3069,10 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  					ARRAY_SIZE(ov5640_csi2_link_freqs) - 1,
>  					4, ov5640_csi2_link_freqs);
>  
> +	hblank = mode->ppl - mode->crop.width;
> +	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
> +					  hblank, 1, hblank);

I was going to say that you should increase the number of controls
passed to v4l2_ctrl_handler_init(), but it's already 32, higher than the
actual number of controls. Maybe you could reduce it to the right number
on top of this series ?

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +
>  	/* Auto/manual white balance */
>  	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
>  					   V4L2_CID_AUTO_WHITE_BALANCE,
> @@ -3111,6 +3122,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  
>  	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>  	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +	ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>  	ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
>  	ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 11/21] media: ov5640: Add VBLANK control

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:32:35PM +0100, Jacopo Mondi wrote:
> Add the VBLANK control which allows to select the duration of the
> frame vertical blankings and allows to control the framerate.
> 
> The VBLANK control also modifies the exposure time range, which cannot
> exceed the maximum frame length.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index b2961e8e07c3..6eeb50724195 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -31,6 +31,10 @@
>  
>  #define OV5640_LINK_RATE_MAX	490000000U
>  
> +/* FIXME: not documented. */
> +#define OV5640_MIN_VBLANK	24
> +#define OV5640_MAX_VTS		1968
> +
>  #define OV5640_DEFAULT_SLAVE_ID 0x3c
>  
>  #define OV5640_REG_SYS_RESET02		0x3002
> @@ -267,6 +271,7 @@ struct ov5640_ctrls {
>  	struct v4l2_ctrl *pixel_rate;
>  	struct v4l2_ctrl *link_freq;
>  	struct v4l2_ctrl *hblank;
> +	struct v4l2_ctrl *vblank;
>  	struct {
>  		struct v4l2_ctrl *auto_exp;
>  		struct v4l2_ctrl *exposure;
> @@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>  	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
>  	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
>  	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> +	s64 exposure_val, exposure_max;
>  	unsigned int hblank;
>  	unsigned int i = 0;
>  	u32 pixel_rate;
> @@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>  	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
>  				 hblank, hblank, 1, hblank);
>  
> +	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
> +				 OV5640_MIN_VBLANK,
> +				 OV5640_MAX_VTS - mode->crop.height, 1,
> +				 mode->vblank_def);
> +	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
> +
> +	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;

There's typically a fixed margin of a few lines between the maximum
exposure and the total vertical size, but that's usually smaller than
the minimum vblank value. Are you sure OV5640_MIN_VBLANK is right here ?

> +	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
> +			     (s64)sensor->ctrls.exposure->minimum,
> +			     (s64)exposure_max);
> +	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> +				 sensor->ctrls.exposure->minimum,
> +				 exposure_max, 1, exposure_val);
> +
>  	return 0;
>  }
>  
> @@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
>  			      (BIT(2) | BIT(1)) : 0);
>  }
>  
> +static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
> +{
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
> +
> +	/* Update the VTOT timing register value. */
> +	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> +				  mode->crop.height + value);
> +}
> +
>  static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
>  {
>  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> @@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
>  {
>  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
>  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
>  	int ret;
>  
>  	/* v4l2_ctrl_lock() locks our own mutex */
>  
> +	switch (ctrl->id) {
> +	case V4L2_CID_VBLANK:
> +		/* Update the exposure range to the newly programmed vblank. */
> +		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
> +
> +		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> +					 sensor->ctrls.exposure->minimum,
> +					 max, sensor->ctrls.exposure->step, max);

Should the default be set to mode->vblank_def ?

> +		break;
> +	}
> +
>  	/*
>  	 * If the device is not powered up by the host driver do
>  	 * not apply any controls to H/W at this time. Instead
> @@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
>  	case V4L2_CID_VFLIP:
>  		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
>  		break;
> +	case V4L2_CID_VBLANK:
> +		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
> +		break;
>  	default:
>  		ret = -EINVAL;
>  		break;
> @@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
>  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> +	unsigned int max_vblank;
>  	unsigned int hblank;
>  	int ret;
>  
> @@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
>  					  hblank, 1, hblank);
>  
> +	max_vblank = OV5640_MAX_VTS - mode->crop.height;
> +	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
> +					  OV5640_MIN_VBLANK, max_vblank,
> +					  1, mode->vblank_def);
> +
>  	/* Auto/manual white balance */
>  	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
>  					   V4L2_CID_AUTO_WHITE_BALANCE,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 12/21] media: ov5640: Fix durations to comply with FPS

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:44:40PM +0100, Jacopo Mondi wrote:
> Now that the frame duration can be controlled by tuning the VBLANK
> duration, fix all modes to comply with the reported FPS.
> 
> All modes run at 30 FPS except for full-resolution mode 2592x1944
> which runs at 15FPS.
> 
> Tested on a 2 data lanes setup in UYVY and RGB565 modes.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 30 +++++++++++++++---------------
>  1 file changed, 15 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 6eeb50724195..2176fa0b8eae 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -611,8 +611,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 160,
>  			.height	= 120,
>  		},
> -		.ppl		= 1896,
> -		.vblank_def	= 864,
> +		.ppl		= 1600,
> +		.vblank_def	= 878,

The total number of pixels is changed from 1896 * (120+864) = 1865664 to
1600 * (120+878) = 1596800. The latter produces 30.06fps with a 48 Mp/s
rate, which certainly seems better than the 25.73fps produced by the
format. I wonder, though, if this has always been wrong, or if the
incorrect frame rate got introduced earlier in this series.

Also, how did you pick a htot value of 1600 ?

>  		.reg_data	= ov5640_setting_QQVGA_160_120,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
>  		.max_fps	= OV5640_30_FPS
> @@ -633,8 +633,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 176,
>  			.height	= 144,
>  		},
> -		.ppl		= 1896,
> -		.vblank_def	= 840,
> +		.ppl		= 1600,
> +		.vblank_def	= 854,
>  		.reg_data	= ov5640_setting_QCIF_176_144,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
>  		.max_fps	= OV5640_30_FPS
> @@ -655,8 +655,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 320,
>  			.height	= 240,
>  		},
> -		.ppl		= 1896,
> -		.vblank_def	= 744,
> +		.ppl		= 1600,
> +		.vblank_def	= 760,
>  		.reg_data	= ov5640_setting_QVGA_320_240,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
>  		.max_fps	= OV5640_30_FPS
> @@ -677,8 +677,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 640,
>  			.height	= 480,
>  		},
> -		.ppl		= 1896,
> -		.vblank_def	= 600,
> +		.ppl		= 1600,
> +		.vblank_def	= 520,
>  		.reg_data	= ov5640_setting_VGA_640_480,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
>  		.max_fps	= OV5640_60_FPS
> @@ -700,7 +700,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.height	= 480,
>  		},
>  		.ppl		= 1896,
> -		.vblank_def	= 504,
> +		.vblank_def	= 1206,
>  		.reg_data	= ov5640_setting_NTSC_720_480,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
>  		.max_fps	= OV5640_30_FPS
> @@ -722,7 +722,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.height	= 576,
>  		},
>  		.ppl		= 1896,
> -		.vblank_def	= 408,
> +		.vblank_def	= 1110,
>  		.reg_data	= ov5640_setting_PAL_720_576,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
>  		.max_fps	= OV5640_30_FPS
> @@ -744,7 +744,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.height	= 768,
>  		},
>  		.ppl		= 1896,
> -		.vblank_def	= 312,
> +		.vblank_def	= 918,
>  		.reg_data	= ov5640_setting_XGA_1024_768,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
>  		.max_fps	= OV5640_30_FPS
> @@ -765,8 +765,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 1280,
>  			.height	= 720,
>  		},
> -		.ppl		= 1896,
> -		.vblank_def	= 20,
> +		.ppl		= 1600,
> +		.vblank_def	= 560,
>  		.reg_data	= ov5640_setting_720P_1280_720,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
>  		.max_fps	= OV5640_30_FPS
> @@ -787,8 +787,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
>  			.width	= 1920,
>  			.height	= 1080,
>  		},
> -		.ppl		= 2500,
> -		.vblank_def	= 40,
> +		.ppl		= 2234,
> +		.vblank_def	= 24,
>  		.reg_data	= ov5640_setting_1080P_1920_1080,
>  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
>  		.max_fps	= OV5640_30_FPS

-- 
Regards,

Laurent Pinchart

Re: [PATCH 13/21] media: ov5640: Initialize try format

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:44:41PM +0100, Jacopo Mondi wrote:
> The TRY format is not initialized at device node open time.
> 
> Fix that by implementing the open() subdev internal function and
> initialize the TRY format there with the default sensor format.

The .open() operation is deprecated for this purpose. Could you please
use .init_cfg() instead ?

> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 38 ++++++++++++++++++++++++++++----------
>  1 file changed, 28 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 2176fa0b8eae..762bdca83aec 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -354,6 +354,18 @@ static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
>   * over i2c.
>   */
>  /* YUV422 UYVY VGA@30fps */
> +
> +static struct v4l2_mbus_framefmt ov5640_default_fmt = {

static const

> +	.code = MEDIA_BUS_FMT_UYVY8_2X8,
> +	.width = 640,
> +	.height = 480,
> +	.colorspace = V4L2_COLORSPACE_SRGB,
> +	.ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(V4L2_COLORSPACE_SRGB),
> +	.quantization = V4L2_QUANTIZATION_FULL_RANGE,
> +	.xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(V4L2_COLORSPACE_SRGB),
> +	.field = V4L2_FIELD_NONE,
> +};
> +
>  static const struct reg_value ov5640_init_setting[] = {
>  	{0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
>  	{0x3103, 0x03, 0, 0}, {0x3630, 0x36, 0, 0},
> @@ -3367,6 +3379,20 @@ static const struct v4l2_subdev_ops ov5640_subdev_ops = {
>  	.pad = &ov5640_pad_ops,
>  };
>  
> +static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> +{
> +	struct v4l2_mbus_framefmt *try_fmt =

s/try_fmt/fmt/ (we should really drop the try_ prefix now that we have a
v4l2_subdev_state)

With these small issues fixed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +		v4l2_subdev_get_try_format(sd, fh->state, 0);
> +
> +	*try_fmt = ov5640_default_fmt;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_internal_ops ov5640_internal_ops = {
> +	.open = &ov5640_open,
> +};
> +
>  static int ov5640_get_regulators(struct ov5640_dev *sensor)
>  {
>  	int i;
> @@ -3412,7 +3438,6 @@ static int ov5640_probe(struct i2c_client *client)
>  	struct device *dev = &client->dev;
>  	struct fwnode_handle *endpoint;
>  	struct ov5640_dev *sensor;
> -	struct v4l2_mbus_framefmt *fmt;
>  	u32 rotation;
>  	int ret;
>  
> @@ -3426,15 +3451,7 @@ static int ov5640_probe(struct i2c_client *client)
>  	 * default init sequence initialize sensor to
>  	 * YUV422 UYVY VGA@30fps
>  	 */
> -	fmt = &sensor->fmt;
> -	fmt->code = MEDIA_BUS_FMT_UYVY8_2X8;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
> -	fmt->ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(fmt->colorspace);
> -	fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
> -	fmt->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(fmt->colorspace);
> -	fmt->width = 640;
> -	fmt->height = 480;
> -	fmt->field = V4L2_FIELD_NONE;
> +	sensor->fmt = ov5640_default_fmt;
>  	sensor->frame_interval.numerator = 1;
>  	sensor->frame_interval.denominator = ov5640_framerates[OV5640_30_FPS];
>  	sensor->current_fr = OV5640_30_FPS;
> @@ -3510,6 +3527,7 @@ static int ov5640_probe(struct i2c_client *client)
>  
>  	v4l2_i2c_subdev_init(&sensor->sd, client, &ov5640_subdev_ops);
>  
> +	sensor->sd.internal_ops = &ov5640_internal_ops;
>  	sensor->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
>  			    V4L2_SUBDEV_FL_HAS_EVENTS;
>  	sensor->pad.flags = MEDIA_PAD_FL_SOURCE;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 14/21] media: ov5640: Implement get_selection

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:44:42PM +0100, Jacopo Mondi wrote:
> Implement the get_selection pad operation for the OV5640 sensor driver.
> 
> The supported targets report the sensor's native size, the active pixel
> array size and the analog crop rectangle from which the image is
> produced.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 61 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 762bdca83aec..ae22300b9655 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -35,6 +35,13 @@
>  #define OV5640_MIN_VBLANK	24
>  #define OV5640_MAX_VTS		1968
>  
> +#define OV5640_NATIVE_WIDTH		2624
> +#define OV5640_NATIVE_HEIGHT		1964
> +#define OV5640_PIXEL_ARRAY_TOP		8
> +#define OV5640_PIXEL_ARRAY_LEFT		16
> +#define OV5640_PIXEL_ARRAY_WIDTH	2592
> +#define OV5640_PIXEL_ARRAY_HEIGHT	1944

According to the datasheet, the sensor has 8 black lines, 6 dummy lines,
1944 active lines and 6 dummy lines. Horizontally, it has 16 dummy
columns, 2592 active columns, and 16 dummy columns. If "pixel array" is
meant to refer to the active area (I dislike the "active" name here, as
the dummy lines and columns are typically "active" too, but that's a
digression), then top should be 14.

> +
>  #define OV5640_DEFAULT_SLAVE_ID 0x3c
>  
>  #define OV5640_REG_SYS_RESET02		0x3002
> @@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
>  	return ret;
>  }
>  
> +static int ov5640_get_selection(struct v4l2_subdev *sd,
> +				struct v4l2_subdev_state *sd_state,
> +				struct v4l2_subdev_selection *sel)
> +{
> +	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
> +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> +	const struct v4l2_rect *crop = &mode->crop;
> +
> +	switch (sel->target) {
> +	case V4L2_SEL_TGT_CROP: {
> +		mutex_lock(&sensor->lock);
> +
> +		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> +		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> +		sel->r.width = analog_crop->width
> +			     - analog_crop->left - crop->left;

Why do you subtract the left coordinates here ?

> +		sel->r.height = analog_crop->height
> +			      - analog_crop->top - crop->top;
> +
> +		mutex_unlock(&sensor->lock);
> +
> +		return 0;
> +	}
> +
> +	case V4L2_SEL_TGT_NATIVE_SIZE:
> +		sel->r.top = 0;
> +		sel->r.left = 0;
> +		sel->r.width = OV5640_NATIVE_WIDTH;
> +		sel->r.height = OV5640_NATIVE_HEIGHT;
> +
> +		return 0;
> +
> +	case V4L2_SEL_TGT_CROP_DEFAULT:
> +	case V4L2_SEL_TGT_CROP_BOUNDS:
> +		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
> +		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
> +		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
> +		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;

In libcamera we use V4L2_SEL_TGT_CROP_BOUNDS to set PixelArraySize,
ignoring the left and top coordinates, and V4L2_SEL_TGT_CROP_DEFAULT to
to set the PixelArrayActiveAreas property relative to PixelArraySize.
This means that non-zero values for the left and top coordinates of
V4L2_SEL_TGT_CROP_BOUNDS will cause issues. Is this an issue in
libcamera, or should V4L2_SEL_TGT_CROP_BOUNDS be changed here ?

The related question is, can we read the optical black lines and the
dummy lines and columns ?

> +
> +		return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +
>  static int ov5640_set_framefmt(struct ov5640_dev *sensor,
>  			       struct v4l2_mbus_framefmt *format)
>  {
> @@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
>  	.enum_mbus_code = ov5640_enum_mbus_code,
>  	.get_fmt = ov5640_get_fmt,
>  	.set_fmt = ov5640_set_fmt,
> +	.get_selection = ov5640_get_selection,
>  	.enum_frame_size = ov5640_enum_frame_size,
>  	.enum_frame_interval = ov5640_enum_frame_interval,
>  };
> @@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
>  {
>  	struct v4l2_mbus_framefmt *try_fmt =
>  		v4l2_subdev_get_try_format(sd, fh->state, 0);
> +	struct v4l2_rect *try_crop =
> +		v4l2_subdev_get_try_crop(sd, fh->state, 0);
>  
>  	*try_fmt = ov5640_default_fmt;
>  
> +	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
> +	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
> +	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
> +	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
> +
>  	return 0;
>  }
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 15/21] media: ov5640: Limit format to FPS in DVP mode only

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:44:43PM +0100, Jacopo Mondi wrote:
> In MIPI mode the frame rate control is performed by adjusting the
> frame blankings and the s_frame_interval function is not used anymore.
> 
> Only check for the per-mode supported frame rate in DVP mode and do not
> restrict MIPI mode.

This certainly aligns better with how the sensor driver is supposed to
operate. I however wonder why you don't do so in DVP mode too. Is it for
backward-compatibility ? If so a comment would be useful.

> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/i2c/ov5640.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index ae22300b9655..ec46e16223af 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -1845,8 +1845,13 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
>  	     (mode->crop.width != width || mode->crop.height != height)))
>  		return NULL;
>  
> -	/* Check to see if the current mode exceeds the max frame rate */
> -	if (ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
> +	/*
> +	 * Check to see if the current mode exceeds the max frame rate.
> +	 * Only DVP mode uses the frame rate set by s_frame_interval, MIPI
> +	 * mode controls framerate by setting blankings.
> +	 */
> +	if (!ov5640_is_mipi(sensor) &&
> +	    ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
>  		return NULL;
>  
>  	return mode;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 16/21] media: ov5640: Disable s_frame_interval in MIPI mode

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:44:44PM +0100, Jacopo Mondi wrote:
> When the sensor is operated in MIPI mode, the frame rate configuration
> is performed by tuning the frame blanking times and not by the
> s_frame_interval subdev operation.
> 
> Disallow enum/s/g_frame_interval if the chip is used in MIPI mode.
> 
> While at it re-indent one function which whose parameters were wrongly
> aligned.

I would squash this with 15/21.

> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index ec46e16223af..28da0ddd2a06 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -3275,15 +3275,17 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static int ov5640_enum_frame_interval(
> -	struct v4l2_subdev *sd,
> -	struct v4l2_subdev_state *sd_state,
> -	struct v4l2_subdev_frame_interval_enum *fie)
> +static int ov5640_enum_frame_interval(struct v4l2_subdev *sd,
> +				      struct v4l2_subdev_state *sd_state,
> +				      struct v4l2_subdev_frame_interval_enum *fie)
>  {
>  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
>  	struct v4l2_fract tpf;
>  	int ret;
>  
> +	if (ov5640_is_mipi(sensor))
> +		return -EINVAL;

Should we return -ENOIOCTLCMD instead of -EINVAL, to indicate the
operation isn't implemented in CSI-2 mode ? Same below.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +
>  	if (fie->pad != 0)
>  		return -EINVAL;
>  	if (fie->index >= OV5640_NUM_FRAMERATES)
> @@ -3306,6 +3308,9 @@ static int ov5640_g_frame_interval(struct v4l2_subdev *sd,
>  {
>  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
>  
> +	if (ov5640_is_mipi(sensor))
> +		return -EINVAL;
> +
>  	mutex_lock(&sensor->lock);
>  	fi->interval = sensor->frame_interval;
>  	mutex_unlock(&sensor->lock);
> @@ -3320,6 +3325,9 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
>  	const struct ov5640_mode_info *mode;
>  	int frame_rate, ret = 0;
>  
> +	if (ov5640_is_mipi(sensor))
> +		return -EINVAL;
> +
>  	if (fi->pad != 0)
>  		return -EINVAL;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 17/21] media: ov5640: Register device properties

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:45:27PM +0100, Jacopo Mondi wrote:
> Parse the device properties and register the rotation and orientation
> V4L2 controls using v4l2_ctrl_new_fwnode_properties().
> 
> Remove the open-coded parsing of the rotation property and assume the
> DTS is correct is providing either <0> or <180> as possible rotations.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/i2c/ov5640.c | 29 ++++++++++++-----------------
>  1 file changed, 12 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 28da0ddd2a06..9d6eb3288b07 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -3164,6 +3164,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
>  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> +	struct v4l2_fwnode_device_properties props;
>  	unsigned int max_vblank;
>  	unsigned int hblank;
>  	int ret;
> @@ -3240,6 +3241,17 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>  		goto free_ctrls;
>  	}
>  
> +	ret = v4l2_fwnode_device_parse(&sensor->i2c_client->dev, &props);
> +	if (ret)
> +		goto free_ctrls;
> +
> +	if (props.rotation == 180)
> +		sensor->upside_down = true;
> +
> +	ret = v4l2_ctrl_new_fwnode_properties(hdl, ops, &props);
> +	if (ret)
> +		goto free_ctrls;
> +
>  	ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>  	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>  	ctrls->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> @@ -3512,7 +3524,6 @@ static int ov5640_probe(struct i2c_client *client)
>  	struct device *dev = &client->dev;
>  	struct fwnode_handle *endpoint;
>  	struct ov5640_dev *sensor;
> -	u32 rotation;
>  	int ret;
>  
>  	sensor = devm_kzalloc(dev, sizeof(*sensor), GFP_KERNEL);
> @@ -3535,22 +3546,6 @@ static int ov5640_probe(struct i2c_client *client)
>  
>  	sensor->ae_target = 52;
>  
> -	/* optional indication of physical rotation of sensor */
> -	ret = fwnode_property_read_u32(dev_fwnode(&client->dev), "rotation",
> -				       &rotation);
> -	if (!ret) {
> -		switch (rotation) {
> -		case 180:
> -			sensor->upside_down = true;
> -			fallthrough;
> -		case 0:
> -			break;
> -		default:
> -			dev_warn(dev, "%u degrees rotation is not supported, ignoring...\n",
> -				 rotation);
> -		}
> -	}
> -
>  	endpoint = fwnode_graph_get_next_endpoint(dev_fwnode(&client->dev),
>  						  NULL);
>  	if (!endpoint) {

-- 
Regards,

Laurent Pinchart

Re: [PATCH 18/21] media: ov5640: Add RGB565_1X16 format

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:45:28PM +0100, Jacopo Mondi wrote:
> The driver already supports the 2X8_[LE|BE] variants of RGB565
> formats.
> 
> As for CSI-2 the 2X8 variants do not apply, add RGB565_1X16 variant
> with little-endian ordering of components as required by the CSI-2
> specifications.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 9d6eb3288b07..db77cdb8a958 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -197,6 +197,7 @@ static const struct ov5640_pixfmt {
>  	{ MEDIA_BUS_FMT_YUYV8_1X16, V4L2_COLORSPACE_SRGB, 16,},
>  	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
>  	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_RGB565_1X16, V4L2_COLORSPACE_SRGB, 16},

Should/could we limit this format to CSI-2 mode ?

>  	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
>  	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
>  	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
> @@ -2746,6 +2747,7 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
>  		mux = OV5640_FMT_MUX_YUV422;
>  		break;
>  	case MEDIA_BUS_FMT_RGB565_2X8_LE:
> +	case MEDIA_BUS_FMT_RGB565_1X16:
>  		/* RGB565 {g[2:0],b[4:0]},{r[4:0],g[5:3]} */
>  		fmt = 0x6F;
>  		mux = OV5640_FMT_MUX_RGB;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 19/21] media: ov5640: Add RGB888/BGR888 formats

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:45:29PM +0100, Jacopo Mondi wrote:
> Add support for RGB888/BGR888 image formats.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index db77cdb8a958..5891eaab05ee 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -198,6 +198,8 @@ static const struct ov5640_pixfmt {
>  	{ MEDIA_BUS_FMT_RGB565_2X8_LE, V4L2_COLORSPACE_SRGB, 16},
>  	{ MEDIA_BUS_FMT_RGB565_2X8_BE, V4L2_COLORSPACE_SRGB, 16},
>  	{ MEDIA_BUS_FMT_RGB565_1X16, V4L2_COLORSPACE_SRGB, 16},
> +	{ MEDIA_BUS_FMT_RGB888_1X24, V4L2_COLORSPACE_SRGB, 24},
> +	{ MEDIA_BUS_FMT_BGR888_1X24, V4L2_COLORSPACE_SRGB, 24},

If we can limit MEDIA_BUS_FMT_RGB565_1X16 to CSI-2 mode in the previous
patch, then this should be limited too. The rest of the patch looks
fine,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  	{ MEDIA_BUS_FMT_SBGGR8_1X8, V4L2_COLORSPACE_SRGB, 8},
>  	{ MEDIA_BUS_FMT_SGBRG8_1X8, V4L2_COLORSPACE_SRGB, 8},
>  	{ MEDIA_BUS_FMT_SGRBG8_1X8, V4L2_COLORSPACE_SRGB, 8},
> @@ -2757,6 +2759,16 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
>  		fmt = 0x61;
>  		mux = OV5640_FMT_MUX_RGB;
>  		break;
> +	case MEDIA_BUS_FMT_RGB888_1X24:
> +		/* RGB888: BGR */
> +		fmt = 0x22;
> +		mux = OV5640_FMT_MUX_RGB;
> +		break;
> +	case MEDIA_BUS_FMT_BGR888_1X24:
> +		/* BGR888: RGB */
> +		fmt = 0x23;
> +		mux = OV5640_FMT_MUX_RGB;
> +		break;
>  	case MEDIA_BUS_FMT_JPEG_1X8:
>  		/* YUV422, YUYV */
>  		fmt = 0x30;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 20/21] media: ov5640: Restrict sizes to mbus code

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:45:55PM +0100, Jacopo Mondi wrote:
> The ov5640 driver supports different sizes for different mbus_codes.
> In particular:
> 
> - 8bpp modes: high resolution sizes (>= 1280x720)

I'm curious as to why the raw modes can't support lower resolutions.

> - 16bpp modes: all sizes
> - 24bpp modes: low resolutions sizes (< 1280x720)
> 
> Restrict the frame sizes enumerations to the above constraints.
> 
> While at it, make sure the fse->mbus_code parameter is valid, and return
> -EINVAL if it's not.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 20 +++++++++++++++++---
>  1 file changed, 17 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 5891eaab05ee..2978dabd1d54 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -3288,14 +3288,28 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
>  				  struct v4l2_subdev_state *sd_state,
>  				  struct v4l2_subdev_frame_size_enum *fse)
>  {
> +	u32 bpp = ov5640_code_to_bpp(fse->code);
> +	unsigned int index = fse->index;
> +
>  	if (fse->pad != 0)
>  		return -EINVAL;
> -	if (fse->index >= OV5640_NUM_MODES)
> +	if (!bpp)
> +		return -EINVAL;
> +
> +	/* Only low-resolution modes are supported for 24bpp modes. */
> +	if (bpp == 24 && index >= OV5640_MODE_720P_1280_720)
> +		return -EINVAL;
> +
> +	/* Only high-resolutions modes are supported for 8bpp formats. */
> +	if (bpp == 8)
> +		index += OV5640_MODE_720P_1280_720;
> +
> +	if (index >= OV5640_NUM_MODES)
>  		return -EINVAL;
>  
> -	fse->min_width = ov5640_mode_data[fse->index].crop.width;
> +	fse->min_width = ov5640_mode_data[index].crop.width;
>  	fse->max_width = fse->min_width;
> -	fse->min_height = ov5640_mode_data[fse->index].crop.height;
> +	fse->min_height = ov5640_mode_data[index].crop.height;
>  	fse->max_height = fse->min_height;
>  
>  	return 0;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 21/21] media: ov5640: Adjust format to bpp in s_fmt

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Mon, Jan 31, 2022 at 03:45:56PM +0100, Jacopo Mondi wrote:
> The ov5640 driver supports different sizes for different mbus_codes.
> In particular:
> 
> - 8bpp modes: high resolution sizes (>= 1280x720)
> - 16bpp modes: all sizes
> - 24bpp modes: low resolutions sizes (< 1280x720)
> 
> Adjust the image sizes according to the above constraints.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>  drivers/media/i2c/ov5640.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 2978dabd1d54..49d0df80f71a 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -2529,6 +2529,7 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
>  				   enum ov5640_frame_rate fr,
>  				   const struct ov5640_mode_info **new_mode)
>  {
> +	unsigned int bpp = ov5640_code_to_bpp(fmt->code);
>  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
>  	const struct ov5640_mode_info *mode;
>  	int i;
> @@ -2536,6 +2537,17 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
>  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
>  	if (!mode)
>  		return -EINVAL;
> +
> +	/*
> +	 * Adjust mode according to bpp:
> +	 * - 8bpp modes work for resolution >= 1280x720
> +	 * - 24bpp modes work resolution < 1280x720
> +	 */
> +	if (bpp == 8 && mode->crop.width < 1280)
> +		mode = &ov5640_mode_data[OV5640_MODE_720P_1280_720];
> +	else if (bpp == 24 && mode->crop.width > 1024)
> +		mode = &ov5640_mode_data[OV5640_MODE_XGA_1024_768];

This is in line with patch 20/21, so

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

but I'm still not sure about the limitation for 8bpp.

> +
>  	fmt->width = mode->crop.width;
>  	fmt->height = mode->crop.height;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 11/21] media: ov5640: Add VBLANK control

[Xavier Roumegue (OSS)]

Hello Jacopo,

On 1/31/22 15:32, Jacopo Mondi wrote:
> Add the VBLANK control which allows to select the duration of the
> frame vertical blankings and allows to control the framerate.
> 
> The VBLANK control also modifies the exposure time range, which cannot
> exceed the maximum frame length.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>   drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
>   1 file changed, 50 insertions(+)
> 
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index b2961e8e07c3..6eeb50724195 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -31,6 +31,10 @@
>   
>   #define OV5640_LINK_RATE_MAX	490000000U
>   
> +/* FIXME: not documented. */
> +#define OV5640_MIN_VBLANK	24
> +#define OV5640_MAX_VTS		1968
> +
>   #define OV5640_DEFAULT_SLAVE_ID 0x3c
>   
>   #define OV5640_REG_SYS_RESET02		0x3002
> @@ -267,6 +271,7 @@ struct ov5640_ctrls {
>   	struct v4l2_ctrl *pixel_rate;
>   	struct v4l2_ctrl *link_freq;
>   	struct v4l2_ctrl *hblank;
> +	struct v4l2_ctrl *vblank;
>   	struct {
>   		struct v4l2_ctrl *auto_exp;
>   		struct v4l2_ctrl *exposure;
> @@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>   	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
>   	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
>   	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> +	s64 exposure_val, exposure_max;
>   	unsigned int hblank;
>   	unsigned int i = 0;
>   	u32 pixel_rate;
> @@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
>   	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
>   				 hblank, hblank, 1, hblank);
>   
> +	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
> +				 OV5640_MIN_VBLANK,
> +				 OV5640_MAX_VTS - mode->crop.height, 1,
> +				 mode->vblank_def);
> +	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
> +
> +	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;
> +	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
> +			     (s64)sensor->ctrls.exposure->minimum,
> +			     (s64)exposure_max);
> +	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> +				 sensor->ctrls.exposure->minimum,
> +				 exposure_max, 1, exposure_val);
> +
>   	return 0;
>   }
>   
> @@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
>   			      (BIT(2) | BIT(1)) : 0);
>   }
>   
> +static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
> +{
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
> +
> +	/* Update the VTOT timing register value. */
> +	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> +				  mode->crop.height + value);
> +}
> +
>   static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
>   {
>   	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> @@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
>   {
>   	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
>   	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> +	const struct ov5640_mode_info *mode = sensor->current_mode;
>   	int ret;
>   
>   	/* v4l2_ctrl_lock() locks our own mutex */
>   
> +	switch (ctrl->id) {
> +	case V4L2_CID_VBLANK:
> +		/* Update the exposure range to the newly programmed vblank. */
> +		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
> +
> +		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> +					 sensor->ctrls.exposure->minimum,
> +					 max, sensor->ctrls.exposure->step, max);
> +		break;
> +	}The kernel build reports the errors below with clang toolchain (LLVM=1):

drivers/media/i2c/ov5640.c:3124:3: error: expected expression
                 unsigned int max = mode->crop.height + ctrl->val - 
OV5640_MIN_VBLANK;
                 ^
drivers/media/i2c/ov5640.c:3128:7: error: use of undeclared identifier 'max'
                                          max, 
sensor->ctrls.exposure->step, max);
                                          ^
drivers/media/i2c/ov5640.c:3128:42: error: use of undeclared identifier 
'max'
                                          max, 
sensor->ctrls.exposure->step, max);
 
      ^
3 errors generated.
make[3]: *** [scripts/Makefile.build:288: drivers/media/i2c/ov5640.o] 
Error 1

Declaring the  variable within a compound statement following the label 
(case), or moving the declaration after the switch fix the build.
The former construct might be more aligned with kernel coding use (?).

switch(...) {
case ...: {
	unsigned int max = ...;
	...	
}
}

or

switch(...) {
	unsigned int max = 0;
	case ...:
		...
		break;
}

> +
>   	/*
>   	 * If the device is not powered up by the host driver do
>   	 * not apply any controls to H/W at this time. Instead
> @@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
>   	case V4L2_CID_VFLIP:
>   		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
>   		break;
> +	case V4L2_CID_VBLANK:
> +		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
> +		break;
>   	default:
>   		ret = -EINVAL;
>   		break;
> @@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>   	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
>   	struct ov5640_ctrls *ctrls = &sensor->ctrls;
>   	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> +	unsigned int max_vblank;
>   	unsigned int hblank;
>   	int ret;
>   
> @@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
>   	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
>   					  hblank, 1, hblank);
>   
> +	max_vblank = OV5640_MAX_VTS - mode->crop.height;
> +	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
> +					  OV5640_MIN_VBLANK, max_vblank,
> +					  1, mode->vblank_def);
> +
>   	/* Auto/manual white balance */
>   	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
>   					   V4L2_CID_AUTO_WHITE_BALANCE,
Regards,
  Xavier

Re: [PATCH 06/21] media: ov5640: Rework CSI-2 clock tree

[Jacopo Mondi]

Hi Laurent,

On Tue, Feb 01, 2022 at 07:27:31PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:32:30PM +0100, Jacopo Mondi wrote:
> > Re-work the ov5640_set_mipi_pclk() function to calculate the
> > PLL configuration using the pixel_rate and link_freq values set at
> > s_fmt time.
> >
> > Rework the DVP clock mode settings to calculate the pixel clock
> > internally and remove the assumption on the 16bpp format.
> >
> > Tested in MIPI CSI-2 mode with 2 data lanes with:
> > - all the sensor supported resolutions in UYVY and RGB565 formats.
> > - resolutions >= 1280x720 in RAW Bayer format.
> > - resolutions < 1280x720 in RGB888 format.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> >
> > ---
> >
> > Not tested with JPEG.
> > Not tested with 1 data lane.
> > Not tested in DVP mode.
> >
> > If I get come Tested-by: tags for the above use cases, it would be
> > great.
> > ---
> >  drivers/media/i2c/ov5640.c | 175 ++++++++++++++++++++-----------------
> >  1 file changed, 94 insertions(+), 81 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index 457f76030163..acc636500907 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -29,6 +29,8 @@
> >  #define OV5640_XCLK_MIN  6000000
> >  #define OV5640_XCLK_MAX 54000000
> >
> > +#define OV5640_LINK_RATE_MAX	490000000U
> > +
> >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> >
> >  #define OV5640_REG_SYS_RESET02		0x3002
> > @@ -88,6 +90,7 @@
> >  #define OV5640_REG_POLARITY_CTRL00	0x4740
> >  #define OV5640_REG_MIPI_CTRL00		0x4800
> >  #define OV5640_REG_DEBUG_MODE		0x4814
> > +#define OV5640_REG_PCLK_PERIOD		0x4837
> >  #define OV5640_REG_ISP_FORMAT_MUX_CTRL	0x501f
> >  #define OV5640_REG_PRE_ISP_TEST_SET1	0x503d
> >  #define OV5640_REG_SDE_CTRL0		0x5580
> > @@ -1035,69 +1038,80 @@ static unsigned long ov5640_calc_sys_clk(struct ov5640_dev *sensor,
> >   * ov5640_set_mipi_pclk() - Calculate the clock tree configuration values
> >   *			    for the MIPI CSI-2 output.
> >   *
> > - * @rate: The requested bandwidth per lane in bytes per second.
> > - *	  'Bandwidth Per Lane' is calculated as:
> > - *	  bpl = HTOT * VTOT * FPS * bpp / num_lanes;
> > - *
> > - * This function use the requested bandwidth to calculate:
> > - * - sample_rate = bpl / (bpp / num_lanes);
> > - *	         = bpl / (PLL_RDIV * BIT_DIV * PCLK_DIV * MIPI_DIV / num_lanes);
> > - *
> > - * - mipi_sclk   = bpl / MIPI_DIV / 2; ( / 2 is for CSI-2 DDR)
>
> It would be useful to have the update formula either here or in the code
> below.
>

I added the above two lines in the comment block as 'rate' parameters
was not clear (as it was the rate _per-lane_ and not the whole bus
rate). Having removed that and since I now only rely on pixel_rate and
link_freq I thought it was not worth reporting the usual formula to
get from one to the other, but I can surely add it back.

> > - *
> > - * with these fixed parameters:
> > - *	PLL_RDIV	= 2;
> > - *	BIT_DIVIDER	= 2; (MIPI_BIT_MODE == 8 ? 2 : 2,5);
> > - *	PCLK_DIV	= 1;
> > - *
> > - * The MIPI clock generation differs for modes that use the scaler and modes
> > - * that do not. In case the scaler is in use, the MIPI_SCLK generates the MIPI
> > - * BIT CLk, and thus:
> > - *
> > - * - mipi_sclk = bpl / MIPI_DIV / 2;
> > - *   MIPI_DIV = 1;
> > - *
> > - * For modes that do not go through the scaler, the MIPI BIT CLOCK is generated
> > - * from the pixel clock, and thus:
> > - *
> > - * - sample_rate = bpl / (bpp / num_lanes);
> > - *	         = bpl / (2 * 2 * 1 * MIPI_DIV / num_lanes);
> > - *		 = bpl / (4 * MIPI_DIV / num_lanes);
> > - * - MIPI_DIV	 = bpp / (4 * num_lanes);
> > - *
> > - * FIXME: this have been tested with 16bpp and 2 lanes setup only.
> > - * MIPI_DIV is fixed to value 2, but it -might- be changed according to the
> > - * above formula for setups with 1 lane or image formats with different bpp.
> > - *
> > - * FIXME: this deviates from the sensor manual documentation which is quite
> > - * thin on the MIPI clock tree generation part.
> > + * FIXME: tested with 2 lanes only.
> >   */
> > -static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
> > -				unsigned long rate)
> > +static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor)
> >  {
> > -	const struct ov5640_mode_info *mode = sensor->current_mode;
> > +	u8 bit_div, mipi_div, pclk_div, sclk_div, sclk2x_div, root_div;
> > +	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
> >  	u8 prediv, mult, sysdiv;
> > -	u8 mipi_div;
> > +	unsigned long sysclk;
> > +	unsigned long sample_rate;
> > +	u8 pclk_period;
> > +	s64 link_freq;
> >  	int ret;
> >
> > +	/* Use the link frequency computed at s_fmt time. */
> > +	link_freq = ov5640_csi2_link_freqs[sensor->ctrls.link_freq->cur.val];
> > +
> >  	/*
> > -	 * 1280x720 is reported to use 'SUBSAMPLING' only,
> > -	 * but according to the sensor manual it goes through the
> > -	 * scaler before subsampling.
> > +	 * - mipi_div - Additional divider for the MIPI lane clock.
> > +	 *
> > +	 * Higher link frequencies would make sysclk > 1GHz.
> > +	 * Keep the sysclk low and do not divide in the MIPI domain.
> >  	 */
> > -	if (mode->dn_mode == SCALING ||
> > -	   (mode->id == OV5640_MODE_720P_1280_720))
> > -		mipi_div = OV5640_MIPI_DIV_SCLK;
> > +	if (link_freq > OV5640_LINK_RATE_MAX)
> > +		mipi_div = 1;
> >  	else
> > -		mipi_div = OV5640_MIPI_DIV_PCLK;
> > +		mipi_div = 2;
> >
> > -	ov5640_calc_sys_clk(sensor, rate, &prediv, &mult, &sysdiv);
> > +	sysclk = link_freq * mipi_div;
> > +	ov5640_calc_sys_clk(sensor, sysclk, &prediv, &mult, &sysdiv);
> >
> > -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0,
> > -			     0x0f, OV5640_PLL_CTRL0_MIPI_MODE_8BIT);
> > +	/*
> > +	 * Adjust PLL parameters to maintain the MIPI_SCLK-to-PCLK ratio;
> > +	 *
> > +	 * - root_div = 2 (fixed)
> > +	 * - bit_div : MIPI 8-bit = 2
> > +	 *	       MIPI 10-bit = 2,5
>
> s/2,5/2.5/
>
> > +	 * - pclk_div = 1 (fixed)
> > +	 * - pll_div  = (2 lanes ? mipi_div : 2 * mipi_div)
>
> What's pll_div here ? There's no corresponding variable. It would be
> nice to name it according to the clock tree diagram located above in the
> driver.
>

Yes, in the diagram above (which I should indeed rework in v2 as it
reports a few things that do not apply anymore) it is called P_DIV. In
the drawing of the clock tree which I used as reference and which is
not part of the sensor dataheet it is said that divider automatically
takes the value of:

- 1 data lane = MIPI_DIV (0x3035[3:0])
- 2 data lane = 2 * MIPI_DIV


> > +	 *   2 lanes: MIPI_SCLK = (4 or 5) * PCLK
> > +	 *   1 lanes: MIPI_SCLK = (8 or 10) * PCLK
> > +	 *
> > +	 * TODO: support 10-bit formats
> > +	 * TODO: test with 1 data lane
> > +	 */
> > +	root_div = OV5640_PLL_CTRL3_PLL_ROOT_DIV_2;
> > +	bit_div =  OV5640_PLL_CTRL0_MIPI_MODE_8BIT;
> > +	pclk_div = OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS;
> >
> > -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1,
> > -			     0xff, sysdiv << 4 | mipi_div);
> > +	/*
> > +	 * Scaler clock:
> > +	 * - YUV: PCLK >= 2 * SCLK
> > +	 * - RAW or JPEG: PCLK >= SCLK
> > +	 * - sclk2x_div = sclk_div / 2
> > +	 *
> > +	 * TODO: test with JPEG.
> > +	 */
> > +	sclk_div = ilog2(OV5640_SCLK_ROOT_DIV);
> > +	sclk2x_div = ilog2(OV5640_SCLK2X_ROOT_DIV);
> > +
> > +	/*
> > +	 * Set the sample period expressed in ns with 1-bit decimal
> > +	 * (0x01=0.5ns).
> > +	 */
> > +	sample_rate = ov5640_pixel_rates[sensor->current_mode->pixel_rate]
> > +		    * (ov5640_code_to_bpp(fmt->code) / 8);
>
> Won't this cause rouding errors for bpp values that are not multiples of
> 8 ?
>

The driver only supports formats with a bpp multiple of 8.
This should probably be the

		    * (ov5640_code_to_bpp(fmt->code) / bps);

with bps = bits per sample
but as far as I can tell it is not needed.

> > +	pclk_period = 2000000000U / sample_rate;
> > +
> > +	/* Program the clock tree registers. */
> > +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL0, 0x0f, bit_div);
> > +	if (ret)
> > +		return ret;
> > +
> > +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL1, 0xff,
> > +			     sysdiv << 4 | mipi_div);
>
> Here you don't use parentheses around (sysdiv << 4), and below you do
> when writing OV5640_REG_SYS_ROOT_DIVIDER. They're not necessary, but it
> could be clearer to add parentheses here.
>

Sure!

> >  	if (ret)
> >  		return ret;
> >
> > @@ -1105,13 +1119,27 @@ static int ov5640_set_mipi_pclk(struct ov5640_dev *sensor,
> >  	if (ret)
> >  		return ret;
> >
> > -	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3,
> > -			     0x1f, OV5640_PLL_CTRL3_PLL_ROOT_DIV_2 | prediv);
> > +	ret = ov5640_mod_reg(sensor, OV5640_REG_SC_PLL_CTRL3, 0x1f,
> > +			     root_div | prediv);
> >  	if (ret)
> >  		return ret;
> >
> > -	return ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER,
> > -			      0x30, OV5640_PLL_SYS_ROOT_DIVIDER_BYPASS);
> > +	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
> > +			     (pclk_div << 4) | (sclk2x_div << 2) | sclk_div);
> > +	if (ret)
> > +		return ret;
> > +
> > +	return ov5640_write_reg(sensor, OV5640_REG_PCLK_PERIOD, pclk_period);
> > +}
> > +
> > +static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> > +{
> > +	u64 rate;
> > +
> > +	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> > +	rate *= ov5640_framerates[sensor->current_fr];
> > +
> > +	return rate;
> >  }
> >
> >  static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
> > @@ -1131,11 +1159,16 @@ static unsigned long ov5640_calc_pclk(struct ov5640_dev *sensor,
> >  	return _rate / *pll_rdiv / *bit_div / *pclk_div;
> >  }
> >
> > -static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor, unsigned long rate)
> > +static int ov5640_set_dvp_pclk(struct ov5640_dev *sensor)
> >  {
> >  	u8 prediv, mult, sysdiv, pll_rdiv, bit_div, pclk_div;
> > +	u64 rate;
>
> Do you need a 64-bit integer here, or can the calculation fit in 32-bit?
>

32bits should be fine and could save me the do_div() which I had to
add to avoid an error on 32-bit ARMv7 caused by dividing a 64bits
value.

Thanks
  j

> >  	int ret;
> >
> > +	rate = ov5640_calc_pixel_rate(sensor);
> > +	rate *= ov5640_code_to_bpp(sensor->fmt.code);
> > +	do_div(rate, sensor->ep.bus.parallel.bus_width);
> > +
> >  	ov5640_calc_pclk(sensor, rate, &prediv, &mult, &sysdiv, &pll_rdiv,
> >  			 &bit_div, &pclk_div);
> >
> > @@ -1660,16 +1693,6 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
> >  	return mode;
> >  }
> >
> > -static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> > -{
> > -	u64 rate;
> > -
> > -	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> > -	rate *= ov5640_framerates[sensor->current_fr];
> > -
> > -	return rate;
> > -}
> > -
> >  /*
> >   * sensor changes between scaling and subsampling, go through
> >   * exposure calculation
> > @@ -1851,7 +1874,6 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
> >  	enum ov5640_downsize_mode dn_mode, orig_dn_mode;
> >  	bool auto_gain = sensor->ctrls.auto_gain->val == 1;
> >  	bool auto_exp =  sensor->ctrls.auto_exp->val == V4L2_EXPOSURE_AUTO;
> > -	unsigned long rate;
> >  	int ret;
> >
> >  	dn_mode = mode->dn_mode;
> > @@ -1870,19 +1892,10 @@ static int ov5640_set_mode(struct ov5640_dev *sensor)
> >  			goto restore_auto_gain;
> >  	}
> >
> > -	/*
> > -	 * All the formats we support have 16 bits per pixel, seems to require
> > -	 * the same rate than YUV, so we can just use 16 bpp all the time.
> > -	 */
> > -	rate = ov5640_calc_pixel_rate(sensor) * 16;
> > -	if (ov5640_is_mipi(sensor)) {
> > -		rate = rate / sensor->ep.bus.mipi_csi2.num_data_lanes;
> > -		ret = ov5640_set_mipi_pclk(sensor, rate);
> > -	} else {
> > -		rate = rate / sensor->ep.bus.parallel.bus_width;
> > -		ret = ov5640_set_dvp_pclk(sensor, rate);
> > -	}
> > -
> > +	if (ov5640_is_mipi(sensor))
> > +		ret = ov5640_set_mipi_pclk(sensor);
> > +	else
> > +		ret = ov5640_set_dvp_pclk(sensor);
> >  	if (ret < 0)
> >  		return 0;
> >
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 07/21] media: ov5640: Rework timings programming

[Jacopo Mondi]

Hi Laurent

On Tue, Feb 01, 2022 at 09:03:25PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:32:31PM +0100, Jacopo Mondi wrote:
> > The current definition of a sensor mode defines timings as follows:
> >
> > - hact, vact: Visible width and height
> > - htot, vtot: Total sizes invluding blankings
>
> s/invluding/including/
>
> > This makes difficult to clearly separate the visible sizes from the
> > blankings and to make the vertical blanking programmable.
> >
> > Rework the sensor modes sizes definition to:
> > - Report the analog crop sizes
> > - Report the visible crop size
> > - Report the total pixels per line as HBLANK is fixed
> > - Report the VBLANK value to make it programmable
> >
> > Also modify the ov5640_set_timings() function to program all the
> > windowing registers are remove them from the per-mode register-value
> > tables.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 451 +++++++++++++++++++++++++------------
> >  1 file changed, 304 insertions(+), 147 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index acc636500907..bd14e2ad22f6 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -61,10 +61,16 @@
> >  #define OV5640_REG_AEC_PK_MANUAL	0x3503
> >  #define OV5640_REG_AEC_PK_REAL_GAIN	0x350a
> >  #define OV5640_REG_AEC_PK_VTS		0x350c
> > +#define OV5640_REG_TIMING_HS		0x3800
> > +#define OV5640_REG_TIMING_VS		0x3802
> > +#define OV5640_REG_TIMING_HW		0x3804
> > +#define OV5640_REG_TIMING_VH		0x3806
> >  #define OV5640_REG_TIMING_DVPHO		0x3808
> >  #define OV5640_REG_TIMING_DVPVO		0x380a
> >  #define OV5640_REG_TIMING_HTS		0x380c
> >  #define OV5640_REG_TIMING_VTS		0x380e
> > +#define OV5640_REG_TIMING_HOFFS		0x3810
> > +#define OV5640_REG_TIMING_VOFFS		0x3812
> >  #define OV5640_REG_TIMING_TC_REG20	0x3820
> >  #define OV5640_REG_TIMING_TC_REG21	0x3821
> >  #define OV5640_REG_AEC_CTRL00		0x3a00
> > @@ -242,12 +248,17 @@ struct ov5640_mode_info {
> >  	enum ov5640_mode_id id;
> >  	enum ov5640_downsize_mode dn_mode;
> >  	enum ov5640_pixel_rate_id pixel_rate;
> > -	u32 hact;
> > -	u32 htot;
> > -	u32 vact;
> > -	u32 vtot;
> > +	/*  Analog crop rectangle. */
> > +	struct v4l2_rect analog_crop;
> > +	/* Visibile crop: from analog crop top-left corner. */
> > +	struct v4l2_rect crop;
> > +	/* Total pixels per line: crop.width + fixed hblank. */
> > +	u32 ppl;
>
> ppl is a fairly uncommon term when it comes to sensor configuration. I'd
> rather keep hact, or use a hblank value.
>

I found ppl (pixels per line) nicer that hact (horizontal act... ?)

> > +	/* Total frame height = crop.height + vblank. */
> > +	u32 vblank_def;
>
> The comment doesn't seem to match the field.
>

Ah yes, I meant to pair it with the above one and make clear that
vblank doesn't include the visible pixels. But yes I get your comment
and I'll change this

> >  	const struct reg_value *reg_data;
> >  	u32 reg_data_size;
> > +	/* DVP only; ignored in MIPI mode. */
> >  	u32 max_fps;
> >  };
> >
> > @@ -353,11 +364,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
> >  	{0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -425,11 +432,7 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -444,11 +447,7 @@ static const struct reg_value ov5640_setting_XGA_1024_768[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -463,11 +462,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -482,11 +477,7 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -500,11 +491,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -557,11 +536,7 @@ static const struct reg_value ov5640_setting_720P_1280_720[] = {
> >  	{0x3c07, 0x07, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > +	{0x3815, 0x31, 0, 0},
> >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
> >  	{0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
> > @@ -576,11 +551,7 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x11, 0, 0},
> > -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > +	{0x3815, 0x11, 0, 0},
> >  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
> >  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -591,9 +562,6 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
> >  	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0},
> >  	{0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > -	{0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
> > -	{0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
> > -	{0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0},
> >  	{0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
> >  	{0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
> >  	{0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
> > @@ -607,11 +575,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x11, 0, 0},
> > -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > +	{0x3815, 0x11, 0, 0},
> >  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
> >  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
> >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > @@ -624,96 +588,250 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
> >
> >  /* power-on sensor init reg table */
> >  static const struct ov5640_mode_info ov5640_mode_init_data = {
> > -	0, SUBSAMPLING,
> > -	OV5640_PIXEL_RATE_96M,
> > -	640, 1896, 480, 984,
> > -	ov5640_init_setting_30fps_VGA,
> > -	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> > -	OV5640_30_FPS,
> > +		.id		= 0,
>
> Ah there we go, named field initializers :-) It would be nice to move
> them to patch 02/21 to minimize the changes here.
>

Rebasing might be rather painful, but I'll see how it goes

> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
>
> I don't think that's right. The TIMING_HW and TIMING_VH registers store
> the end X and Y values respectively, so here width should be 2624 and
> height 1944. Same below, and you'll need to update the register writes
> accordingly (left + width - 1 and top + height - 1).
>

These values come from what was hardcoded in the register tables
In example:

{0x3804, 0x0a, 0, 0}, {0x3805, 0x3f, 0, 0} 0xa3f = 2623
{0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0} 0x79b = 1947

I wouldn't change them for sake for bisectability, but I can adjust
them on top as they smell fishy to me as well

> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 640,
> > +			.height	= 480,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 504,
> > +		.reg_data	= ov5640_init_setting_30fps_VGA,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> > +		.max_fps	= OV5640_30_FPS
> >  };
> >
> >  static const struct ov5640_mode_info
> >  ov5640_mode_data[OV5640_NUM_MODES] = {
> >  	{
> >  		/* 160x120 */
> > -		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_48M,
> > -		160, 1896, 120, 984,
> > -		ov5640_setting_QQVGA_160_120,
> > -		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_QQVGA_160_120,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 160,
> > +			.height	= 120,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 864,
> > +		.reg_data	= ov5640_setting_QQVGA_160_120,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 176x144 */
> > -		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_48M,
> > -		176, 1896, 144, 984,
> > -		ov5640_setting_QCIF_176_144,
> > -		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_QCIF_176_144,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 176,
> > +			.height	= 144,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 840,
> > +		.reg_data	= ov5640_setting_QCIF_176_144,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 320x240 */
> > -		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_48M,
> > -		320, 1896, 240, 984,
> > -		ov5640_setting_QVGA_320_240,
> > -		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_QVGA_320_240,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 320,
> > +			.height	= 240,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 744,
> > +		.reg_data	= ov5640_setting_QVGA_320_240,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 640x480 */
> > -		OV5640_MODE_VGA_640_480, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_48M,
> > -		640, 1896, 480, 1080,
> > -		ov5640_setting_VGA_640_480,
> > -		ARRAY_SIZE(ov5640_setting_VGA_640_480),
> > -		OV5640_60_FPS
> > +		.id		= OV5640_MODE_VGA_640_480,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 640,
> > +			.height	= 480,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 600,
> > +		.reg_data	= ov5640_setting_VGA_640_480,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
> > +		.max_fps	= OV5640_60_FPS
> >  	}, {
> >  		/* 720x480 */
> > -		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_96M,
> > -		720, 1896, 480, 984,
> > -		ov5640_setting_NTSC_720_480,
> > -		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_NTSC_720_480,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 56,
>
> I think this one should be 16.
>

Uuuups

> > @@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},

        .left = 0x10
        .top = 0x3c = 60d

> > +			.top	= 60,
> > +			.width	= 720,
> > +			.height	= 480,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 504,
> > +		.reg_data	= ov5640_setting_NTSC_720_480,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 720x576 */
> > -		OV5640_MODE_PAL_720_576, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_96M,
> > -		720, 1896, 576, 984,
> > -		ov5640_setting_PAL_720_576,
> > -		ARRAY_SIZE(ov5640_setting_PAL_720_576),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_PAL_720_576,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 56,
> > +			.top	= 60,
>
> And this should be 6.
>

Ups x2

> > @@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
> > -	{0x3810, 0x00, 0, 0},
> > -	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},

        .left = 0x38 ( = 56d)
        .top = 0x06

Thanks for spotting!

Now I'm puzzled as I recal I tried to algin these modes to use
        .left = 16
        .top = 6

As all the other modes and they broke, so I don't recall if I've mixed
up values or changed them on purpose... I will re-test


> > +			.width	= 720,
> > +			.height	= 576,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 408,
> > +		.reg_data	= ov5640_setting_PAL_720_576,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 1024x768 */
> > -		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_96M,
> > -		1024, 1896, 768, 1080,
> > -		ov5640_setting_XGA_1024_768,
> > -		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_XGA_1024_768,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 4,
> > +			.width	= 2623,
> > +			.height	= 1947,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 6,
> > +			.width	= 1024,
> > +			.height	= 768,
> > +		},
> > +		.ppl		= 1896,
> > +		.vblank_def	= 312,
> > +		.reg_data	= ov5640_setting_XGA_1024_768,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 1280x720 */
> > -		OV5640_MODE_720P_1280_720, SUBSAMPLING,
> > -		OV5640_PIXEL_RATE_124M,
> > -		1280, 1892, 720, 740,
> > -		ov5640_setting_720P_1280_720,
> > -		ARRAY_SIZE(ov5640_setting_720P_1280_720),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_720P_1280_720,
> > +		.dn_mode	= SUBSAMPLING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_124M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 250,
> > +			.width	= 2623,
> > +			.height	= 1705,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 4,
> > +			.width	= 1280,
> > +			.height	= 720,
> > +		},
> > +		.ppl		= 1896,
>
> 1892 ?
>
> > +		.vblank_def	= 20,
> > +		.reg_data	= ov5640_setting_720P_1280_720,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 1920x1080 */
> > -		OV5640_MODE_1080P_1920_1080, SCALING,
> > -		OV5640_PIXEL_RATE_148M,
> > -		1920, 2500, 1080, 1120,
> > -		ov5640_setting_1080P_1920_1080,
> > -		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> > -		OV5640_30_FPS
> > +		.id		= OV5640_MODE_1080P_1920_1080,
> > +		.dn_mode	= SCALING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_148M,
> > +		.analog_crop = {
> > +			.left	= 336,
> > +			.top	= 434,
> > +			.width	= 2287,
> > +			.height	= 1521,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 4,
> > +			.width	= 1920,
> > +			.height	= 1080,
> > +		},
> > +		.ppl		= 2500,
> > +		.vblank_def	= 40,
> > +		.reg_data	= ov5640_setting_1080P_1920_1080,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> > +		.max_fps	= OV5640_30_FPS
> >  	}, {
> >  		/* 2592x1944 */
> > -		OV5640_MODE_QSXGA_2592_1944, SCALING,
> > -		OV5640_PIXEL_RATE_168M,
> > -		2592, 2844, 1944, 1968,
> > -		ov5640_setting_QSXGA_2592_1944,
> > -		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> > -		OV5640_15_FPS
> > +		.id		= OV5640_MODE_QSXGA_2592_1944,
> > +		.dn_mode	= SCALING,
> > +		.pixel_rate	= OV5640_PIXEL_RATE_168M,
> > +		.analog_crop = {
> > +			.left	= 0,
> > +			.top	= 0,
> > +			.width	= 2623,
> > +			.height	= 1951,
> > +		},
> > +		.crop = {
> > +			.left	= 16,
> > +			.top	= 4,
> > +			.width	= 2592,
> > +			.height	= 1944,
> > +		},
> > +		.ppl		= 2844,
> > +		.vblank_def	= 24,
> > +		.reg_data	= ov5640_setting_QSXGA_2592_1944,
> > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> > +		.max_fps	= OV5640_15_FPS
> >  	},
> >  };
> >
> > @@ -1136,7 +1254,8 @@ static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> >  {
> >  	u64 rate;
> >
> > -	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> > +	rate = sensor->current_mode->ppl
> > +	     * (sensor->current_mode->crop.height + sensor->current_mode->vblank_def);
> >  	rate *= ov5640_framerates[sensor->current_fr];
> >
> >  	return rate;
> > @@ -1220,17 +1339,21 @@ static int ov5640_set_jpeg_timings(struct ov5640_dev *sensor,
> >  	if (ret < 0)
> >  		return ret;
> >
> > -	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE, mode->hact);
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE,
> > +				 mode->crop.width);
> >  	if (ret < 0)
> >  		return ret;
> >
> > -	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE, mode->vact);
> > +	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE,
> > +				  mode->crop.height);
> >  }
> >
> >  /* download ov5640 settings to sensor through i2c */
> >  static int ov5640_set_timings(struct ov5640_dev *sensor,
> >  			      const struct ov5640_mode_info *mode)
> >  {
> > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > +	const struct v4l2_rect *crop = &mode->crop;
> >  	int ret;
> >
> >  	if (sensor->fmt.code == MEDIA_BUS_FMT_JPEG_1X8) {
> > @@ -1239,19 +1362,54 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
> >  			return ret;
> >  	}
> >
> > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HS,
> > +				 analog_crop->left);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VS,
> > +				 analog_crop->top);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HW,
> > +				 analog_crop->width);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VH,
> > +				 analog_crop->height);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, crop->width);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, crop->height);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->ppl);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> > +				 crop->height + mode->vblank_def);
> >  	if (ret < 0)
> >  		return ret;
> >
> > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HOFFS,
> > +				 crop->left);
> >  	if (ret < 0)
> >  		return ret;
> >
> > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
> > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VOFFS,
> > +				 crop->top);
> >  	if (ret < 0)
> >  		return ret;
>
> Maybe you could order the crop writes as the analog crop writes, with
> left, top, width and height ?
>

I can, yes

Thanks for review

> >
> > -	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
> > +	return 0;
> >  }
> >
> >  static int ov5640_load_regs(struct ov5640_dev *sensor,
> > @@ -1679,11 +1837,11 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
> >
> >  	mode = v4l2_find_nearest_size(ov5640_mode_data,
> >  				      ARRAY_SIZE(ov5640_mode_data),
> > -				      hact, vact,
> > -				      width, height);
> > +				      crop.width, crop.height, width, height);
> >
> >  	if (!mode ||
> > -	    (!nearest && (mode->hact != width || mode->vact != height)))
> > +	    (!nearest &&
> > +	     (mode->crop.width != width || mode->crop.height != height)))
> >  		return NULL;
> >
> >  	/* Check to see if the current mode exceeds the max frame rate */
> > @@ -2367,8 +2525,8 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> >  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
> >  	if (!mode)
> >  		return -EINVAL;
> > -	fmt->width = mode->hact;
> > -	fmt->height = mode->vact;
> > +	fmt->width = mode->crop.width;
> > +	fmt->height = mode->crop.height;
> >
> >  	if (new_mode)
> >  		*new_mode = mode;
> > @@ -2999,11 +3157,9 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
> >  	if (fse->index >= OV5640_NUM_MODES)
> >  		return -EINVAL;
> >
> > -	fse->min_width =
> > -		ov5640_mode_data[fse->index].hact;
> > +	fse->min_width = ov5640_mode_data[fse->index].crop.width;
> >  	fse->max_width = fse->min_width;
> > -	fse->min_height =
> > -		ov5640_mode_data[fse->index].vact;
> > +	fse->min_height = ov5640_mode_data[fse->index].crop.height;
> >  	fse->max_height = fse->min_height;
> >
> >  	return 0;
> > @@ -3067,15 +3223,16 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
> >  	mode = sensor->current_mode;
> >
> >  	frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
> > -					       mode->hact, mode->vact);
> > +					       mode->crop.width,
> > +					       mode->crop.height);
> >  	if (frame_rate < 0) {
> >  		/* Always return a valid frame interval value */
> >  		fi->interval = sensor->frame_interval;
> >  		goto out;
> >  	}
> >
> > -	mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
> > -				mode->vact, true);
> > +	mode = ov5640_find_mode(sensor, frame_rate, mode->crop.width,
> > +				mode->crop.height, true);
> >  	if (!mode) {
> >  		ret = -EINVAL;
> >  		goto out;
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 08/21] media: ov5640: Re-sort per-mode register tables

[Jacopo Mondi]

Hi Laurent

On Tue, Feb 01, 2022 at 09:05:55PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:32:32PM +0100, Jacopo Mondi wrote:
> > The per-mode register tables are not sorted by size. Fix it.
> >
> > Cosmetic change only.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 18 +++++++++---------
> >  1 file changed, 9 insertions(+), 9 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index bd14e2ad22f6..d966cca78e92 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -428,7 +428,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
> >  	{0x3a1f, 0x14, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3c00, 0x04, 0, 300},
> >  };
> >
> > -static const struct reg_value ov5640_setting_VGA_640_480[] = {
> > +static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > @@ -439,11 +439,10 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
> >  	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
> >  	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
> >  	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
> > -	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
> > -	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> > +	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
> >  };
> >
> > -static const struct reg_value ov5640_setting_XGA_1024_768[] = {
> > +static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > @@ -473,7 +472,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
> >  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> >  };
> >
> > -static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> > +static const struct reg_value ov5640_setting_VGA_640_480[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > @@ -484,10 +483,11 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> >  	{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
> >  	{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
> >  	{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0},
> > -	{0x4407, 0x04, 0, 0}, {0x5001, 0xa3, 0, 0},
> > +	{0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
> > +	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> >  };
> >
> > -static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> > +static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > @@ -502,7 +502,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> >  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> >  };
> >
> > -static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> > +static const struct reg_value ov5640_setting_PAL_720_576[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
> > @@ -517,7 +517,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> >  	{0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
> >  };
> >
> > -static const struct reg_value ov5640_setting_PAL_720_576[] = {
> > +static const struct reg_value ov5640_setting_XGA_1024_768[] = {
> >  	{0x3c07, 0x08, 0, 0},
> >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> >  	{0x3814, 0x31, 0, 0},
>
> Is it me, or does this mean that ov5640_setting_XGA_1024_768,
> ov5640_setting_QCIF_176_144, ov5640_setting_PAL_720_576 and
> ov5640_setting_NTSC_720_480 are all identical ?
>

Ah! Fun, I didn't realize!
meaning I've not re-looked at this patch after I formatted it :D

yes they are!
There likely are more opportunities to remove more black magic tables
then. I wonder in what the other modes differ now. I'll investigate, I
wish we could remove the ov5640_setting_XXX tables completely, that
would be great if we can almost fully control the sensor programming
sequence and not rely on opaque tables!

> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 09/21] media: ov5640: Remove ov5640_mode_init_data

[Jacopo Mondi]

Hi Laurent,

On Tue, Feb 01, 2022 at 09:07:35PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:32:33PM +0100, Jacopo Mondi wrote:
> > The ov5640_mode_init_data is a fictional sensor more which is used to
> > program the initial sensor settings.
> >
> > It is only used to initialize the sensor and can be replaced
> > it with a throw-away mode which just wraps the register table.
> >
> > Also rename the register table to drop the format from the name to make
> > it clear an actual sensor mode has to be applied after the initial
> > programming.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 32 +++++---------------------------
> >  1 file changed, 5 insertions(+), 27 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index d966cca78e92..1e2f37c93f0d 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -348,7 +348,7 @@ static inline bool ov5640_is_mipi(struct ov5640_dev *sensor)
> >   * over i2c.
> >   */
> >  /* YUV422 UYVY VGA@30fps */
> > -static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
> > +static const struct reg_value ov5640_init_setting[] = {
> >  	{0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
> >  	{0x3103, 0x03, 0, 0}, {0x3630, 0x36, 0, 0},
> >  	{0x3631, 0x0e, 0, 0}, {0x3632, 0xe2, 0, 0}, {0x3633, 0x12, 0, 0},
> > @@ -586,30 +586,6 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
> >  	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
> >  };
> >
> > -/* power-on sensor init reg table */
> > -static const struct ov5640_mode_info ov5640_mode_init_data = {
> > -		.id		= 0,
> > -		.dn_mode	= SUBSAMPLING,
> > -		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > -		.analog_crop = {
> > -			.left	= 0,
> > -			.top	= 4,
> > -			.width	= 2623,
> > -			.height	= 1947,
> > -		},
> > -		.crop = {
> > -			.left	= 16,
> > -			.top	= 6,
> > -			.width	= 640,
> > -			.height	= 480,
> > -		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 504,
> > -		.reg_data	= ov5640_init_setting_30fps_VGA,
> > -		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> > -		.max_fps	= OV5640_30_FPS
> > -};
> > -
> >  static const struct ov5640_mode_info
> >  ov5640_mode_data[OV5640_NUM_MODES] = {
> >  	{
> > @@ -2117,13 +2093,15 @@ static int ov5640_set_framefmt(struct ov5640_dev *sensor,
> >  /* restore the last set video mode after chip power-on */
> >  static int ov5640_restore_mode(struct ov5640_dev *sensor)
> >  {
> > +	struct ov5640_mode_info init_data = {};
> >  	int ret;
> >
> >  	/* first load the initial register values */
> > -	ret = ov5640_load_regs(sensor, &ov5640_mode_init_data);
> > +	init_data.reg_data = ov5640_init_setting;
> > +	init_data.reg_data_size = ARRAY_SIZE(ov5640_init_setting);
> > +	ret = ov5640_load_regs(sensor, &init_data);
>
> Could we change ov5640_load_regs() to take a reg_value array and size as
> parameters, to avoid the local init_data variable ?
>

ov5640_load_regs() calls ov5640_set_timings() which wants a mode.
I can de-couple the two, even more so because here I'm passing in a
mode with invalid timings, which is not an issue as the actual ones
will be programmed at s_stream() time, but might be confusing when
reading the code in future.

> >  	if (ret < 0)
> >  		return ret;
> > -	sensor->last_mode = &ov5640_mode_init_data;
> >
> >  	ret = ov5640_mod_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, 0x3f,
> >  			     (ilog2(OV5640_SCLK2X_ROOT_DIV) << 2) |
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 11/21] media: ov5640: Add VBLANK control

[Jacopo Mondi]

Hi Laurent,

On Wed, Feb 02, 2022 at 11:35:31PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:32:35PM +0100, Jacopo Mondi wrote:
> > Add the VBLANK control which allows to select the duration of the
> > frame vertical blankings and allows to control the framerate.
> >
> > The VBLANK control also modifies the exposure time range, which cannot
> > exceed the maximum frame length.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 50 insertions(+)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index b2961e8e07c3..6eeb50724195 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -31,6 +31,10 @@
> >
> >  #define OV5640_LINK_RATE_MAX	490000000U
> >
> > +/* FIXME: not documented. */
> > +#define OV5640_MIN_VBLANK	24
> > +#define OV5640_MAX_VTS		1968
> > +
> >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> >
> >  #define OV5640_REG_SYS_RESET02		0x3002
> > @@ -267,6 +271,7 @@ struct ov5640_ctrls {
> >  	struct v4l2_ctrl *pixel_rate;
> >  	struct v4l2_ctrl *link_freq;
> >  	struct v4l2_ctrl *hblank;
> > +	struct v4l2_ctrl *vblank;
> >  	struct {
> >  		struct v4l2_ctrl *auto_exp;
> >  		struct v4l2_ctrl *exposure;
> > @@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> >  	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
> >  	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
> >  	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> > +	s64 exposure_val, exposure_max;
> >  	unsigned int hblank;
> >  	unsigned int i = 0;
> >  	u32 pixel_rate;
> > @@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> >  	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
> >  				 hblank, hblank, 1, hblank);
> >
> > +	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
> > +				 OV5640_MIN_VBLANK,
> > +				 OV5640_MAX_VTS - mode->crop.height, 1,
> > +				 mode->vblank_def);
> > +	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
> > +
> > +	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;
>
> There's typically a fixed margin of a few lines between the maximum
> exposure and the total vertical size, but that's usually smaller than
> the minimum vblank value. Are you sure OV5640_MIN_VBLANK is right here ?
>

Section "4.6.2 manual exposure control" of the datasheet reports
some hints about how to enlarge the total frame size to accomodate the
desired exposure.

It is not totally clear to me how though. The maximum exposure time
seems to be limited not only by VTS, but by
(VTS + [0x350Cd,0x350d]).

The 0x350c/d registers are said to:

In auto exposure mode, the extra exposure values (larger than 1 frame)
in registers 0x350C/0x350D automatically change. In manual exposure
mode, these registers will not automatically change. The manually set
exposure in registers 0x3500~0x3502 must be less than the maximum
exposure value in {0x380E, 0x380F} + {0x350C,0x350D}


with:
- 0x350c,0x350d = PK_VTS
- 0x380e,0x380f = VTS

I have no idea how manual exposure works with this sensor, as I see
0x350c only being read and never programmed.

This anyway does not suggest any margin between VTS+PK_VTS but rahter
that the exposure time can be larger than VTS which doesn't make much
sense to me (maybe I should just consider 'real VTS' = VTS + PK_VTS).

Anyway, being aware of such margins being usually required, I used the
min VBLANK value. Which is also not documented but I experimentally
tried to progressively shrink until capture did not break.

Should I remove it from the above calculation ?

> > +	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
> > +			     (s64)sensor->ctrls.exposure->minimum,
> > +			     (s64)exposure_max);
> > +	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > +				 sensor->ctrls.exposure->minimum,
> > +				 exposure_max, 1, exposure_val);
> > +
> >  	return 0;
> >  }
> >
> > @@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
> >  			      (BIT(2) | BIT(1)) : 0);
> >  }
> >
> > +static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
> > +{
> > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > +
> > +	/* Update the VTOT timing register value. */
> > +	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> > +				  mode->crop.height + value);
> > +}
> > +
> >  static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
> >  {
> >  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> > @@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> >  {
> >  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> >  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> >  	int ret;
> >
> >  	/* v4l2_ctrl_lock() locks our own mutex */
> >
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_VBLANK:
> > +		/* Update the exposure range to the newly programmed vblank. */
> > +		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
> > +
> > +		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > +					 sensor->ctrls.exposure->minimum,
> > +					 max, sensor->ctrls.exposure->step, max);
>
> Should the default be set to mode->vblank_def ?
>

It should!

> > +		break;
> > +	}
> > +
> >  	/*
> >  	 * If the device is not powered up by the host driver do
> >  	 * not apply any controls to H/W at this time. Instead
> > @@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> >  	case V4L2_CID_VFLIP:
> >  		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
> >  		break;
> > +	case V4L2_CID_VBLANK:
> > +		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
> > +		break;
> >  	default:
> >  		ret = -EINVAL;
> >  		break;
> > @@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> >  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
> >  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
> >  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> > +	unsigned int max_vblank;
> >  	unsigned int hblank;
> >  	int ret;
> >
> > @@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> >  	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
> >  					  hblank, 1, hblank);
> >
> > +	max_vblank = OV5640_MAX_VTS - mode->crop.height;
> > +	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
> > +					  OV5640_MIN_VBLANK, max_vblank,
> > +					  1, mode->vblank_def);
> > +
> >  	/* Auto/manual white balance */
> >  	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
> >  					   V4L2_CID_AUTO_WHITE_BALANCE,
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 11/21] media: ov5640: Add VBLANK control

[Jacopo Mondi]

Hello Xavier

On Thu, Feb 03, 2022 at 08:58:37AM +0100, Xavier Roumegue (OSS) wrote:
> Hello Jacopo,
>
> On 1/31/22 15:32, Jacopo Mondi wrote:
> > Add the VBLANK control which allows to select the duration of the
> > frame vertical blankings and allows to control the framerate.
> >
> > The VBLANK control also modifies the exposure time range, which cannot
> > exceed the maximum frame length.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >   drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 50 insertions(+)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index b2961e8e07c3..6eeb50724195 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -31,6 +31,10 @@
> >   #define OV5640_LINK_RATE_MAX	490000000U
> > +/* FIXME: not documented. */
> > +#define OV5640_MIN_VBLANK	24
> > +#define OV5640_MAX_VTS		1968
> > +
> >   #define OV5640_DEFAULT_SLAVE_ID 0x3c
> >   #define OV5640_REG_SYS_RESET02		0x3002
> > @@ -267,6 +271,7 @@ struct ov5640_ctrls {
> >   	struct v4l2_ctrl *pixel_rate;
> >   	struct v4l2_ctrl *link_freq;
> >   	struct v4l2_ctrl *hblank;
> > +	struct v4l2_ctrl *vblank;
> >   	struct {
> >   		struct v4l2_ctrl *auto_exp;
> >   		struct v4l2_ctrl *exposure;
> > @@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> >   	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
> >   	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
> >   	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> > +	s64 exposure_val, exposure_max;
> >   	unsigned int hblank;
> >   	unsigned int i = 0;
> >   	u32 pixel_rate;
> > @@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> >   	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
> >   				 hblank, hblank, 1, hblank);
> > +	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
> > +				 OV5640_MIN_VBLANK,
> > +				 OV5640_MAX_VTS - mode->crop.height, 1,
> > +				 mode->vblank_def);
> > +	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
> > +
> > +	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;
> > +	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
> > +			     (s64)sensor->ctrls.exposure->minimum,
> > +			     (s64)exposure_max);
> > +	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > +				 sensor->ctrls.exposure->minimum,
> > +				 exposure_max, 1, exposure_val);
> > +
> >   	return 0;
> >   }
> > @@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
> >   			      (BIT(2) | BIT(1)) : 0);
> >   }
> > +static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
> > +{
> > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > +
> > +	/* Update the VTOT timing register value. */
> > +	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> > +				  mode->crop.height + value);
> > +}
> > +
> >   static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
> >   {
> >   	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> > @@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> >   {
> >   	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> >   	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> >   	int ret;
> >   	/* v4l2_ctrl_lock() locks our own mutex */
> > +	switch (ctrl->id) {
> > +	case V4L2_CID_VBLANK:
> > +		/* Update the exposure range to the newly programmed vblank. */
> > +		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
> > +
> > +		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > +					 sensor->ctrls.exposure->minimum,
> > +					 max, sensor->ctrls.exposure->step, max);
> > +		break;
> > +	}The kernel build reports the errors below with clang toolchain (LLVM=1):
>
> drivers/media/i2c/ov5640.c:3124:3: error: expected expression
>                 unsigned int max = mode->crop.height + ctrl->val -
> OV5640_MIN_VBLANK;
>                 ^
> drivers/media/i2c/ov5640.c:3128:7: error: use of undeclared identifier 'max'
>                                          max, sensor->ctrls.exposure->step,
> max);
>                                          ^
> drivers/media/i2c/ov5640.c:3128:42: error: use of undeclared identifier
> 'max'
>                                          max, sensor->ctrls.exposure->step,
> max);
>
>      ^
> 3 errors generated.
> make[3]: *** [scripts/Makefile.build:288: drivers/media/i2c/ov5640.o] Error
> 1
>
> Declaring the  variable within a compound statement following the label
> (case), or moving the declaration after the switch fix the build.
> The former construct might be more aligned with kernel coding use (?).
>
> switch(...) {
> case ...: {
> 	unsigned int max = ...;
> 	...
> }
> }
>
> or
>
> switch(...) {
> 	unsigned int max = 0;
> 	case ...:
> 		...
> 		break;

Thanks, I didn't compile with clang :)

I'll go for the former, as you say it is more aligned with common
practices as far as I can tell.

Thanks
  j

> }
>
> > +
> >   	/*
> >   	 * If the device is not powered up by the host driver do
> >   	 * not apply any controls to H/W at this time. Instead
> > @@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> >   	case V4L2_CID_VFLIP:
> >   		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
> >   		break;
> > +	case V4L2_CID_VBLANK:
> > +		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
> > +		break;
> >   	default:
> >   		ret = -EINVAL;
> >   		break;
> > @@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> >   	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
> >   	struct ov5640_ctrls *ctrls = &sensor->ctrls;
> >   	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> > +	unsigned int max_vblank;
> >   	unsigned int hblank;
> >   	int ret;
> > @@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> >   	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
> >   					  hblank, 1, hblank);
> > +	max_vblank = OV5640_MAX_VTS - mode->crop.height;
> > +	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
> > +					  OV5640_MIN_VBLANK, max_vblank,
> > +					  1, mode->vblank_def);
> > +
> >   	/* Auto/manual white balance */
> >   	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
> >   					   V4L2_CID_AUTO_WHITE_BALANCE,
> Regards,
>  Xavier

Re: [PATCH 14/21] media: ov5640: Implement get_selection

[Jacopo Mondi]

Hi Laurent

On Thu, Feb 03, 2022 at 12:29:47AM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:44:42PM +0100, Jacopo Mondi wrote:
> > Implement the get_selection pad operation for the OV5640 sensor driver.
> >
> > The supported targets report the sensor's native size, the active pixel
> > array size and the analog crop rectangle from which the image is
> > produced.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 61 insertions(+)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index 762bdca83aec..ae22300b9655 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -35,6 +35,13 @@
> >  #define OV5640_MIN_VBLANK	24
> >  #define OV5640_MAX_VTS		1968
> >
> > +#define OV5640_NATIVE_WIDTH		2624
> > +#define OV5640_NATIVE_HEIGHT		1964
> > +#define OV5640_PIXEL_ARRAY_TOP		8
> > +#define OV5640_PIXEL_ARRAY_LEFT		16
> > +#define OV5640_PIXEL_ARRAY_WIDTH	2592
> > +#define OV5640_PIXEL_ARRAY_HEIGHT	1944
>
> According to the datasheet, the sensor has 8 black lines, 6 dummy lines,
> 1944 active lines and 6 dummy lines. Horizontally, it has 16 dummy
> columns, 2592 active columns, and 16 dummy columns. If "pixel array" is
> meant to refer to the active area (I dislike the "active" name here, as
> the dummy lines and columns are typically "active" too, but that's a
> digression), then top should be 14.
>

Corret, I have only considered the 8 black lines, but dummy too are
'active but not valid' ones.

I'll change to 14

> > +
> >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> >
> >  #define OV5640_REG_SYS_RESET02		0x3002
> > @@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> >  	return ret;
> >  }
> >
> > +static int ov5640_get_selection(struct v4l2_subdev *sd,
> > +				struct v4l2_subdev_state *sd_state,
> > +				struct v4l2_subdev_selection *sel)
> > +{
> > +	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > +	const struct v4l2_rect *crop = &mode->crop;
> > +
> > +	switch (sel->target) {
> > +	case V4L2_SEL_TGT_CROP: {
> > +		mutex_lock(&sensor->lock);
> > +
> > +		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> > +		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> > +		sel->r.width = analog_crop->width
> > +			     - analog_crop->left - crop->left;
>
> Why do you subtract the left coordinates here ?
>

As the analog_crop->width is defined from the full pixel array size
(black + dummy + active). The TGT_CROP rectangle width and height
should instead report the dimensions of the portion of the pixel array
size which is processed to obtain the final image, and thus the
vertical and horizontal offsets should be subtracted


    full                            analog_crop.width
     |                              (0x3804, 0x3805) = 2624
     |                                   |
     V                                   V
  -> +-----------------------------------+
     |                                   |
     x-----------------------------------|
     |  x----  TGT_CROP width  ----------|
     |  |                                |
     |  |                                |
     |  |                                |
     |  |                                |
     |  |                                |
     |  |                                |
     |  |                                |
     +--|--------------------------------+
     |  |
     |  |-> crop.left = 16
     |
     |-> analog_crop.left = 0



> > +		sel->r.height = analog_crop->height
> > +			      - analog_crop->top - crop->top;
> > +
> > +		mutex_unlock(&sensor->lock);
> > +
> > +		return 0;
> > +	}
> > +
> > +	case V4L2_SEL_TGT_NATIVE_SIZE:
> > +		sel->r.top = 0;
> > +		sel->r.left = 0;
> > +		sel->r.width = OV5640_NATIVE_WIDTH;
> > +		sel->r.height = OV5640_NATIVE_HEIGHT;
> > +
> > +		return 0;
> > +
> > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > +		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
> > +		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
> > +		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
> > +		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;
>
> In libcamera we use V4L2_SEL_TGT_CROP_BOUNDS to set PixelArraySize,
> ignoring the left and top coordinates, and V4L2_SEL_TGT_CROP_DEFAULT to
> to set the PixelArrayActiveAreas property relative to PixelArraySize.
> This means that non-zero values for the left and top coordinates of
> V4L2_SEL_TGT_CROP_BOUNDS will cause issues. Is this an issue in
> libcamera, or should V4L2_SEL_TGT_CROP_BOUNDS be changed here ?
>
> The related question is, can we read the optical black lines and the
> dummy lines and columns ?
>
> > +
> > +		return 0;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> >  static int ov5640_set_framefmt(struct ov5640_dev *sensor,
> >  			       struct v4l2_mbus_framefmt *format)
> >  {
> > @@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
> >  	.enum_mbus_code = ov5640_enum_mbus_code,
> >  	.get_fmt = ov5640_get_fmt,
> >  	.set_fmt = ov5640_set_fmt,
> > +	.get_selection = ov5640_get_selection,
> >  	.enum_frame_size = ov5640_enum_frame_size,
> >  	.enum_frame_interval = ov5640_enum_frame_interval,
> >  };
> > @@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> >  {
> >  	struct v4l2_mbus_framefmt *try_fmt =
> >  		v4l2_subdev_get_try_format(sd, fh->state, 0);
> > +	struct v4l2_rect *try_crop =
> > +		v4l2_subdev_get_try_crop(sd, fh->state, 0);
> >
> >  	*try_fmt = ov5640_default_fmt;
> >
> > +	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
> > +	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
> > +	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
> > +	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
> > +
> >  	return 0;
> >  }
> >
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 15/21] media: ov5640: Limit format to FPS in DVP mode only

[Jacopo Mondi]

On Thu, Feb 03, 2022 at 12:38:12AM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:44:43PM +0100, Jacopo Mondi wrote:
> > In MIPI mode the frame rate control is performed by adjusting the
> > frame blankings and the s_frame_interval function is not used anymore.
> >
> > Only check for the per-mode supported frame rate in DVP mode and do not
> > restrict MIPI mode.
>
> This certainly aligns better with how the sensor driver is supposed to
> operate. I however wonder why you don't do so in DVP mode too. Is it for
> backward-compatibility ? If so a comment would be useful.

yes, and mostly because DVP mode seems to work well and I didn't want
to change the way subdev is operated for DVP users.

I would be more than happy to remove frame_interval completely.

>
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> > ---
> >  drivers/media/i2c/ov5640.c | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index ae22300b9655..ec46e16223af 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -1845,8 +1845,13 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
> >  	     (mode->crop.width != width || mode->crop.height != height)))
> >  		return NULL;
> >
> > -	/* Check to see if the current mode exceeds the max frame rate */
> > -	if (ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
> > +	/*
> > +	 * Check to see if the current mode exceeds the max frame rate.
> > +	 * Only DVP mode uses the frame rate set by s_frame_interval, MIPI
> > +	 * mode controls framerate by setting blankings.
> > +	 */
> > +	if (!ov5640_is_mipi(sensor) &&
> > +	    ov5640_framerates[fr] > ov5640_framerates[mode->max_fps])
> >  		return NULL;
> >
> >  	return mode;
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 12/21] media: ov5640: Fix durations to comply with FPS

[Jacopo Mondi]

Hi Laurent,

On Wed, Feb 02, 2022 at 11:48:52PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:44:40PM +0100, Jacopo Mondi wrote:
> > Now that the frame duration can be controlled by tuning the VBLANK
> > duration, fix all modes to comply with the reported FPS.
> >
> > All modes run at 30 FPS except for full-resolution mode 2592x1944
> > which runs at 15FPS.
> >
> > Tested on a 2 data lanes setup in UYVY and RGB565 modes.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 30 +++++++++++++++---------------
> >  1 file changed, 15 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index 6eeb50724195..2176fa0b8eae 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -611,8 +611,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 160,
> >  			.height	= 120,
> >  		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 864,
> > +		.ppl		= 1600,
> > +		.vblank_def	= 878,
>
> The total number of pixels is changed from 1896 * (120+864) = 1865664 to
> 1600 * (120+878) = 1596800. The latter produces 30.06fps with a 48 Mp/s
> rate, which certainly seems better than the 25.73fps produced by the
> format. I wonder, though, if this has always been wrong, or if the
> incorrect frame rate got introduced earlier in this series.
>

Ideally, the values there come from the existing tables. I could have
introduced mistakes of course, but not changed those numbers
internally. With the current implementation I recall the framerate was
not accurate I can't tell if that's because the PLL is wrongly
programmed or because of wrong timings.

> Also, how did you pick a htot value of 1600 ?
>

mmm, it was a value that allowed me to range in a decent interval of
vblank values. Found by experiments I have to admit, not planned.
It could prbably be better estimated...


> >  		.reg_data	= ov5640_setting_QQVGA_160_120,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -633,8 +633,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 176,
> >  			.height	= 144,
> >  		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 840,
> > +		.ppl		= 1600,
> > +		.vblank_def	= 854,
> >  		.reg_data	= ov5640_setting_QCIF_176_144,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -655,8 +655,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 320,
> >  			.height	= 240,
> >  		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 744,
> > +		.ppl		= 1600,
> > +		.vblank_def	= 760,
> >  		.reg_data	= ov5640_setting_QVGA_320_240,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -677,8 +677,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 640,
> >  			.height	= 480,
> >  		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 600,
> > +		.ppl		= 1600,
> > +		.vblank_def	= 520,
> >  		.reg_data	= ov5640_setting_VGA_640_480,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
> >  		.max_fps	= OV5640_60_FPS
> > @@ -700,7 +700,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.height	= 480,
> >  		},
> >  		.ppl		= 1896,
> > -		.vblank_def	= 504,
> > +		.vblank_def	= 1206,
> >  		.reg_data	= ov5640_setting_NTSC_720_480,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -722,7 +722,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.height	= 576,
> >  		},
> >  		.ppl		= 1896,
> > -		.vblank_def	= 408,
> > +		.vblank_def	= 1110,
> >  		.reg_data	= ov5640_setting_PAL_720_576,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -744,7 +744,7 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.height	= 768,
> >  		},
> >  		.ppl		= 1896,
> > -		.vblank_def	= 312,
> > +		.vblank_def	= 918,
> >  		.reg_data	= ov5640_setting_XGA_1024_768,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -765,8 +765,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 1280,
> >  			.height	= 720,
> >  		},
> > -		.ppl		= 1896,
> > -		.vblank_def	= 20,
> > +		.ppl		= 1600,
> > +		.vblank_def	= 560,
> >  		.reg_data	= ov5640_setting_720P_1280_720,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
> >  		.max_fps	= OV5640_30_FPS
> > @@ -787,8 +787,8 @@ ov5640_mode_data[OV5640_NUM_MODES] = {
> >  			.width	= 1920,
> >  			.height	= 1080,
> >  		},
> > -		.ppl		= 2500,
> > -		.vblank_def	= 40,
> > +		.ppl		= 2234,
> > +		.vblank_def	= 24,
> >  		.reg_data	= ov5640_setting_1080P_1920_1080,
> >  		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> >  		.max_fps	= OV5640_30_FPS
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 21/21] media: ov5640: Adjust format to bpp in s_fmt

[Jacopo Mondi]

Hi Laurent

On Thu, Feb 03, 2022 at 01:03:50AM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Mon, Jan 31, 2022 at 03:45:56PM +0100, Jacopo Mondi wrote:
> > The ov5640 driver supports different sizes for different mbus_codes.
> > In particular:
> >
> > - 8bpp modes: high resolution sizes (>= 1280x720)
> > - 16bpp modes: all sizes
> > - 24bpp modes: low resolutions sizes (< 1280x720)
> >
> > Adjust the image sizes according to the above constraints.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> >  drivers/media/i2c/ov5640.c | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index 2978dabd1d54..49d0df80f71a 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -2529,6 +2529,7 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> >  				   enum ov5640_frame_rate fr,
> >  				   const struct ov5640_mode_info **new_mode)
> >  {
> > +	unsigned int bpp = ov5640_code_to_bpp(fmt->code);
> >  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> >  	const struct ov5640_mode_info *mode;
> >  	int i;
> > @@ -2536,6 +2537,17 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> >  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
> >  	if (!mode)
> >  		return -EINVAL;
> > +
> > +	/*
> > +	 * Adjust mode according to bpp:
> > +	 * - 8bpp modes work for resolution >= 1280x720
> > +	 * - 24bpp modes work resolution < 1280x720
> > +	 */
> > +	if (bpp == 8 && mode->crop.width < 1280)
> > +		mode = &ov5640_mode_data[OV5640_MODE_720P_1280_720];
> > +	else if (bpp == 24 && mode->crop.width > 1024)
> > +		mode = &ov5640_mode_data[OV5640_MODE_XGA_1024_768];
>
> This is in line with patch 20/21, so
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> but I'm still not sure about the limitation for 8bpp.
>

Me neither. I had on my todo to test 8bpp with different pixel clock
rates to see if it makes any difference. If only 1080p and full res
were working I would have guessed it was because those are the only
modes obtained by cropping the full pixel array without any
subsampling being applied, but as 1280x720 works too....

One thinking I had was that the pixel_rate assigned to each mode where
too slow for an 8bpp mode and that the resulting CSI-2 frequencies
were consequentially below the minimum required ones, but I would have
to re-calculate to see if that's the case.

Anyway, as said in the cover letter, I have now assigned a pixel_rate
to each mode, regardless of the image format. As the same 'mode' in
24bpp or 8bpp requires different pixel rates, I think this first
approach is good, but to fully exploit all the modes the pixel_rate
should be controlled by userspace too, in function of the bpp (and the
number of data lanes). My understanding is that it should happen not
by changing PIXEL_RATE but rather LINK_FREQ. But with the latter being
a menu control (something I still don't get the reason for) there's a
limited number of combination that could be supported there...

We'll see, I would have been happy enough if 16bpp gets actually
fixed when it comes to frame durations, with 8bpp/24bpp being usable.

We can improve on top of an hopefully now more solid base.

> > +
> >  	fmt->width = mode->crop.width;
> >  	fmt->height = mode->crop.height;
> >
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 07/21] media: ov5640: Rework timings programming

[Laurent Pinchart]

Hi Jacopo,

On Mon, Feb 07, 2022 at 03:37:39PM +0100, Jacopo Mondi wrote:
> On Tue, Feb 01, 2022 at 09:03:25PM +0200, Laurent Pinchart wrote:
> > On Mon, Jan 31, 2022 at 03:32:31PM +0100, Jacopo Mondi wrote:
> > > The current definition of a sensor mode defines timings as follows:
> > >
> > > - hact, vact: Visible width and height
> > > - htot, vtot: Total sizes invluding blankings
> >
> > s/invluding/including/
> >
> > > This makes difficult to clearly separate the visible sizes from the
> > > blankings and to make the vertical blanking programmable.
> > >
> > > Rework the sensor modes sizes definition to:
> > > - Report the analog crop sizes
> > > - Report the visible crop size
> > > - Report the total pixels per line as HBLANK is fixed
> > > - Report the VBLANK value to make it programmable
> > >
> > > Also modify the ov5640_set_timings() function to program all the
> > > windowing registers are remove them from the per-mode register-value
> > > tables.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > >  drivers/media/i2c/ov5640.c | 451 +++++++++++++++++++++++++------------
> > >  1 file changed, 304 insertions(+), 147 deletions(-)
> > >
> > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > index acc636500907..bd14e2ad22f6 100644
> > > --- a/drivers/media/i2c/ov5640.c
> > > +++ b/drivers/media/i2c/ov5640.c
> > > @@ -61,10 +61,16 @@
> > >  #define OV5640_REG_AEC_PK_MANUAL	0x3503
> > >  #define OV5640_REG_AEC_PK_REAL_GAIN	0x350a
> > >  #define OV5640_REG_AEC_PK_VTS		0x350c
> > > +#define OV5640_REG_TIMING_HS		0x3800
> > > +#define OV5640_REG_TIMING_VS		0x3802
> > > +#define OV5640_REG_TIMING_HW		0x3804
> > > +#define OV5640_REG_TIMING_VH		0x3806
> > >  #define OV5640_REG_TIMING_DVPHO		0x3808
> > >  #define OV5640_REG_TIMING_DVPVO		0x380a
> > >  #define OV5640_REG_TIMING_HTS		0x380c
> > >  #define OV5640_REG_TIMING_VTS		0x380e
> > > +#define OV5640_REG_TIMING_HOFFS		0x3810
> > > +#define OV5640_REG_TIMING_VOFFS		0x3812
> > >  #define OV5640_REG_TIMING_TC_REG20	0x3820
> > >  #define OV5640_REG_TIMING_TC_REG21	0x3821
> > >  #define OV5640_REG_AEC_CTRL00		0x3a00
> > > @@ -242,12 +248,17 @@ struct ov5640_mode_info {
> > >  	enum ov5640_mode_id id;
> > >  	enum ov5640_downsize_mode dn_mode;
> > >  	enum ov5640_pixel_rate_id pixel_rate;
> > > -	u32 hact;
> > > -	u32 htot;
> > > -	u32 vact;
> > > -	u32 vtot;
> > > +	/*  Analog crop rectangle. */
> > > +	struct v4l2_rect analog_crop;
> > > +	/* Visibile crop: from analog crop top-left corner. */
> > > +	struct v4l2_rect crop;
> > > +	/* Total pixels per line: crop.width + fixed hblank. */
> > > +	u32 ppl;
> >
> > ppl is a fairly uncommon term when it comes to sensor configuration. I'd
> > rather keep hact, or use a hblank value.
> 
> I found ppl (pixels per line) nicer that hact (horizontal act... ?)
> 
> > > +	/* Total frame height = crop.height + vblank. */
> > > +	u32 vblank_def;
> >
> > The comment doesn't seem to match the field.
> 
> Ah yes, I meant to pair it with the above one and make clear that
> vblank doesn't include the visible pixels. But yes I get your comment
> and I'll change this
> 
> > >  	const struct reg_value *reg_data;
> > >  	u32 reg_data_size;
> > > +	/* DVP only; ignored in MIPI mode. */
> > >  	u32 max_fps;
> > >  };
> > >
> > > @@ -353,11 +364,7 @@ static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
> > >  	{0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -425,11 +432,7 @@ static const struct reg_value ov5640_setting_VGA_640_480[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -444,11 +447,7 @@ static const struct reg_value ov5640_setting_XGA_1024_768[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -463,11 +462,7 @@ static const struct reg_value ov5640_setting_QVGA_320_240[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -482,11 +477,7 @@ static const struct reg_value ov5640_setting_QQVGA_160_120[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -500,11 +491,7 @@ static const struct reg_value ov5640_setting_QCIF_176_144[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -557,11 +536,7 @@ static const struct reg_value ov5640_setting_720P_1280_720[] = {
> > >  	{0x3c07, 0x07, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x31, 0, 0},
> > > -	{0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > > +	{0x3815, 0x31, 0, 0},
> > >  	{0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
> > >  	{0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
> > > @@ -576,11 +551,7 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x11, 0, 0},
> > > -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > > +	{0x3815, 0x11, 0, 0},
> > >  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
> > >  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -591,9 +562,6 @@ static const struct reg_value ov5640_setting_1080P_1920_1080[] = {
> > >  	{0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0},
> > >  	{0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > > -	{0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
> > > -	{0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
> > > -	{0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0},
> > >  	{0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
> > >  	{0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
> > >  	{0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
> > > @@ -607,11 +575,7 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
> > >  	{0x3c07, 0x08, 0, 0},
> > >  	{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
> > >  	{0x3814, 0x11, 0, 0},
> > > -	{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
> > > -	{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
> > > -	{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
> > > +	{0x3815, 0x11, 0, 0},
> > >  	{0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
> > >  	{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
> > >  	{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
> > > @@ -624,96 +588,250 @@ static const struct reg_value ov5640_setting_QSXGA_2592_1944[] = {
> > >
> > >  /* power-on sensor init reg table */
> > >  static const struct ov5640_mode_info ov5640_mode_init_data = {
> > > -	0, SUBSAMPLING,
> > > -	OV5640_PIXEL_RATE_96M,
> > > -	640, 1896, 480, 984,
> > > -	ov5640_init_setting_30fps_VGA,
> > > -	ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> > > -	OV5640_30_FPS,
> > > +		.id		= 0,
> >
> > Ah there we go, named field initializers :-) It would be nice to move
> > them to patch 02/21 to minimize the changes here.
> 
> Rebasing might be rather painful, but I'll see how it goes
> 
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> >
> > I don't think that's right. The TIMING_HW and TIMING_VH registers store
> > the end X and Y values respectively, so here width should be 2624 and
> > height 1944. Same below, and you'll need to update the register writes
> > accordingly (left + width - 1 and top + height - 1).
> >
> 
> These values come from what was hardcoded in the register tables
> In example:
> 
> {0x3804, 0x0a, 0, 0}, {0x3805, 0x3f, 0, 0} 0xa3f = 2623
> {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0} 0x79b = 1947
> 
> I wouldn't change them for sake for bisectability, but I can adjust
> them on top as they smell fishy to me as well

You're right that register 0x3806 contains 1947, but that register
doesn't store the height of the analog crop rectangle, it stores the
y_end coordinate. width = y_end - y_start + 1 = 1944 in this case.

> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 640,
> > > +			.height	= 480,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 504,
> > > +		.reg_data	= ov5640_init_setting_30fps_VGA,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
> > > +		.max_fps	= OV5640_30_FPS
> > >  };
> > >
> > >  static const struct ov5640_mode_info
> > >  ov5640_mode_data[OV5640_NUM_MODES] = {
> > >  	{
> > >  		/* 160x120 */
> > > -		OV5640_MODE_QQVGA_160_120, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_48M,
> > > -		160, 1896, 120, 984,
> > > -		ov5640_setting_QQVGA_160_120,
> > > -		ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_QQVGA_160_120,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 160,
> > > +			.height	= 120,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 864,
> > > +		.reg_data	= ov5640_setting_QQVGA_160_120,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QQVGA_160_120),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 176x144 */
> > > -		OV5640_MODE_QCIF_176_144, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_48M,
> > > -		176, 1896, 144, 984,
> > > -		ov5640_setting_QCIF_176_144,
> > > -		ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_QCIF_176_144,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 176,
> > > +			.height	= 144,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 840,
> > > +		.reg_data	= ov5640_setting_QCIF_176_144,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QCIF_176_144),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 320x240 */
> > > -		OV5640_MODE_QVGA_320_240, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_48M,
> > > -		320, 1896, 240, 984,
> > > -		ov5640_setting_QVGA_320_240,
> > > -		ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_QVGA_320_240,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 320,
> > > +			.height	= 240,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 744,
> > > +		.reg_data	= ov5640_setting_QVGA_320_240,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QVGA_320_240),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 640x480 */
> > > -		OV5640_MODE_VGA_640_480, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_48M,
> > > -		640, 1896, 480, 1080,
> > > -		ov5640_setting_VGA_640_480,
> > > -		ARRAY_SIZE(ov5640_setting_VGA_640_480),
> > > -		OV5640_60_FPS
> > > +		.id		= OV5640_MODE_VGA_640_480,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_48M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 640,
> > > +			.height	= 480,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 600,
> > > +		.reg_data	= ov5640_setting_VGA_640_480,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_VGA_640_480),
> > > +		.max_fps	= OV5640_60_FPS
> > >  	}, {
> > >  		/* 720x480 */
> > > -		OV5640_MODE_NTSC_720_480, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_96M,
> > > -		720, 1896, 480, 984,
> > > -		ov5640_setting_NTSC_720_480,
> > > -		ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_NTSC_720_480,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 56,
> >
> > I think this one should be 16.
> 
> Uuuups
> 
> > > @@ -519,11 +506,7 @@ static const struct reg_value ov5640_setting_NTSC_720_480[] = {
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
> 
>         .left = 0x10
>         .top = 0x3c = 60d
> 
> > > +			.top	= 60,
> > > +			.width	= 720,
> > > +			.height	= 480,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 504,
> > > +		.reg_data	= ov5640_setting_NTSC_720_480,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_NTSC_720_480),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 720x576 */
> > > -		OV5640_MODE_PAL_720_576, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_96M,
> > > -		720, 1896, 576, 984,
> > > -		ov5640_setting_PAL_720_576,
> > > -		ARRAY_SIZE(ov5640_setting_PAL_720_576),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_PAL_720_576,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 56,
> > > +			.top	= 60,
> >
> > And this should be 6.
> 
> Ups x2
> 
> > > @@ -538,11 +521,7 @@ static const struct reg_value ov5640_setting_PAL_720_576[] = {
> > > -	{0x3810, 0x00, 0, 0},
> > > -	{0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
> 
>         .left = 0x38 ( = 56d)
>         .top = 0x06
> 
> Thanks for spotting!
> 
> Now I'm puzzled as I recal I tried to algin these modes to use
>         .left = 16
>         .top = 6
> 
> As all the other modes and they broke, so I don't recall if I've mixed
> up values or changed them on purpose... I will re-test
> 
> > > +			.width	= 720,
> > > +			.height	= 576,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 408,
> > > +		.reg_data	= ov5640_setting_PAL_720_576,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_PAL_720_576),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 1024x768 */
> > > -		OV5640_MODE_XGA_1024_768, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_96M,
> > > -		1024, 1896, 768, 1080,
> > > -		ov5640_setting_XGA_1024_768,
> > > -		ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_XGA_1024_768,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_96M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 4,
> > > +			.width	= 2623,
> > > +			.height	= 1947,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 6,
> > > +			.width	= 1024,
> > > +			.height	= 768,
> > > +		},
> > > +		.ppl		= 1896,
> > > +		.vblank_def	= 312,
> > > +		.reg_data	= ov5640_setting_XGA_1024_768,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_XGA_1024_768),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 1280x720 */
> > > -		OV5640_MODE_720P_1280_720, SUBSAMPLING,
> > > -		OV5640_PIXEL_RATE_124M,
> > > -		1280, 1892, 720, 740,
> > > -		ov5640_setting_720P_1280_720,
> > > -		ARRAY_SIZE(ov5640_setting_720P_1280_720),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_720P_1280_720,
> > > +		.dn_mode	= SUBSAMPLING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_124M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 250,
> > > +			.width	= 2623,
> > > +			.height	= 1705,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 4,
> > > +			.width	= 1280,
> > > +			.height	= 720,
> > > +		},
> > > +		.ppl		= 1896,
> >
> > 1892 ?
> >
> > > +		.vblank_def	= 20,
> > > +		.reg_data	= ov5640_setting_720P_1280_720,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_720P_1280_720),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 1920x1080 */
> > > -		OV5640_MODE_1080P_1920_1080, SCALING,
> > > -		OV5640_PIXEL_RATE_148M,
> > > -		1920, 2500, 1080, 1120,
> > > -		ov5640_setting_1080P_1920_1080,
> > > -		ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> > > -		OV5640_30_FPS
> > > +		.id		= OV5640_MODE_1080P_1920_1080,
> > > +		.dn_mode	= SCALING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_148M,
> > > +		.analog_crop = {
> > > +			.left	= 336,
> > > +			.top	= 434,
> > > +			.width	= 2287,
> > > +			.height	= 1521,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 4,
> > > +			.width	= 1920,
> > > +			.height	= 1080,
> > > +		},
> > > +		.ppl		= 2500,
> > > +		.vblank_def	= 40,
> > > +		.reg_data	= ov5640_setting_1080P_1920_1080,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_1080P_1920_1080),
> > > +		.max_fps	= OV5640_30_FPS
> > >  	}, {
> > >  		/* 2592x1944 */
> > > -		OV5640_MODE_QSXGA_2592_1944, SCALING,
> > > -		OV5640_PIXEL_RATE_168M,
> > > -		2592, 2844, 1944, 1968,
> > > -		ov5640_setting_QSXGA_2592_1944,
> > > -		ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> > > -		OV5640_15_FPS
> > > +		.id		= OV5640_MODE_QSXGA_2592_1944,
> > > +		.dn_mode	= SCALING,
> > > +		.pixel_rate	= OV5640_PIXEL_RATE_168M,
> > > +		.analog_crop = {
> > > +			.left	= 0,
> > > +			.top	= 0,
> > > +			.width	= 2623,
> > > +			.height	= 1951,
> > > +		},
> > > +		.crop = {
> > > +			.left	= 16,
> > > +			.top	= 4,
> > > +			.width	= 2592,
> > > +			.height	= 1944,
> > > +		},
> > > +		.ppl		= 2844,
> > > +		.vblank_def	= 24,
> > > +		.reg_data	= ov5640_setting_QSXGA_2592_1944,
> > > +		.reg_data_size	= ARRAY_SIZE(ov5640_setting_QSXGA_2592_1944),
> > > +		.max_fps	= OV5640_15_FPS
> > >  	},
> > >  };
> > >
> > > @@ -1136,7 +1254,8 @@ static u64 ov5640_calc_pixel_rate(struct ov5640_dev *sensor)
> > >  {
> > >  	u64 rate;
> > >
> > > -	rate = sensor->current_mode->vtot * sensor->current_mode->htot;
> > > +	rate = sensor->current_mode->ppl
> > > +	     * (sensor->current_mode->crop.height + sensor->current_mode->vblank_def);
> > >  	rate *= ov5640_framerates[sensor->current_fr];
> > >
> > >  	return rate;
> > > @@ -1220,17 +1339,21 @@ static int ov5640_set_jpeg_timings(struct ov5640_dev *sensor,
> > >  	if (ret < 0)
> > >  		return ret;
> > >
> > > -	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE, mode->hact);
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_VFIFO_HSIZE,
> > > +				 mode->crop.width);
> > >  	if (ret < 0)
> > >  		return ret;
> > >
> > > -	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE, mode->vact);
> > > +	return ov5640_write_reg16(sensor, OV5640_REG_VFIFO_VSIZE,
> > > +				  mode->crop.height);
> > >  }
> > >
> > >  /* download ov5640 settings to sensor through i2c */
> > >  static int ov5640_set_timings(struct ov5640_dev *sensor,
> > >  			      const struct ov5640_mode_info *mode)
> > >  {
> > > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > > +	const struct v4l2_rect *crop = &mode->crop;
> > >  	int ret;
> > >
> > >  	if (sensor->fmt.code == MEDIA_BUS_FMT_JPEG_1X8) {
> > > @@ -1239,19 +1362,54 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
> > >  			return ret;
> > >  	}
> > >
> > > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HS,
> > > +				 analog_crop->left);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VS,
> > > +				 analog_crop->top);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HW,
> > > +				 analog_crop->width);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VH,
> > > +				 analog_crop->height);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, crop->width);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, crop->height);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->ppl);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> > > +				 crop->height + mode->vblank_def);
> > >  	if (ret < 0)
> > >  		return ret;
> > >
> > > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HOFFS,
> > > +				 crop->left);
> > >  	if (ret < 0)
> > >  		return ret;
> > >
> > > -	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
> > > +	ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VOFFS,
> > > +				 crop->top);
> > >  	if (ret < 0)
> > >  		return ret;
> >
> > Maybe you could order the crop writes as the analog crop writes, with
> > left, top, width and height ?
> 
> I can, yes
> 
> Thanks for review
> 
> > >
> > > -	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
> > > +	return 0;
> > >  }
> > >
> > >  static int ov5640_load_regs(struct ov5640_dev *sensor,
> > > @@ -1679,11 +1837,11 @@ ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
> > >
> > >  	mode = v4l2_find_nearest_size(ov5640_mode_data,
> > >  				      ARRAY_SIZE(ov5640_mode_data),
> > > -				      hact, vact,
> > > -				      width, height);
> > > +				      crop.width, crop.height, width, height);
> > >
> > >  	if (!mode ||
> > > -	    (!nearest && (mode->hact != width || mode->vact != height)))
> > > +	    (!nearest &&
> > > +	     (mode->crop.width != width || mode->crop.height != height)))
> > >  		return NULL;
> > >
> > >  	/* Check to see if the current mode exceeds the max frame rate */
> > > @@ -2367,8 +2525,8 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> > >  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
> > >  	if (!mode)
> > >  		return -EINVAL;
> > > -	fmt->width = mode->hact;
> > > -	fmt->height = mode->vact;
> > > +	fmt->width = mode->crop.width;
> > > +	fmt->height = mode->crop.height;
> > >
> > >  	if (new_mode)
> > >  		*new_mode = mode;
> > > @@ -2999,11 +3157,9 @@ static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
> > >  	if (fse->index >= OV5640_NUM_MODES)
> > >  		return -EINVAL;
> > >
> > > -	fse->min_width =
> > > -		ov5640_mode_data[fse->index].hact;
> > > +	fse->min_width = ov5640_mode_data[fse->index].crop.width;
> > >  	fse->max_width = fse->min_width;
> > > -	fse->min_height =
> > > -		ov5640_mode_data[fse->index].vact;
> > > +	fse->min_height = ov5640_mode_data[fse->index].crop.height;
> > >  	fse->max_height = fse->min_height;
> > >
> > >  	return 0;
> > > @@ -3067,15 +3223,16 @@ static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
> > >  	mode = sensor->current_mode;
> > >
> > >  	frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
> > > -					       mode->hact, mode->vact);
> > > +					       mode->crop.width,
> > > +					       mode->crop.height);
> > >  	if (frame_rate < 0) {
> > >  		/* Always return a valid frame interval value */
> > >  		fi->interval = sensor->frame_interval;
> > >  		goto out;
> > >  	}
> > >
> > > -	mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
> > > -				mode->vact, true);
> > > +	mode = ov5640_find_mode(sensor, frame_rate, mode->crop.width,
> > > +				mode->crop.height, true);
> > >  	if (!mode) {
> > >  		ret = -EINVAL;
> > >  		goto out;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 21/21] media: ov5640: Adjust format to bpp in s_fmt

[Laurent Pinchart]

Hi Jacopo,

On Mon, Feb 07, 2022 at 05:07:07PM +0100, Jacopo Mondi wrote:
> On Thu, Feb 03, 2022 at 01:03:50AM +0200, Laurent Pinchart wrote:
> > On Mon, Jan 31, 2022 at 03:45:56PM +0100, Jacopo Mondi wrote:
> > > The ov5640 driver supports different sizes for different mbus_codes.
> > > In particular:
> > >
> > > - 8bpp modes: high resolution sizes (>= 1280x720)
> > > - 16bpp modes: all sizes
> > > - 24bpp modes: low resolutions sizes (< 1280x720)
> > >
> > > Adjust the image sizes according to the above constraints.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > >  drivers/media/i2c/ov5640.c | 12 ++++++++++++
> > >  1 file changed, 12 insertions(+)
> > >
> > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > index 2978dabd1d54..49d0df80f71a 100644
> > > --- a/drivers/media/i2c/ov5640.c
> > > +++ b/drivers/media/i2c/ov5640.c
> > > @@ -2529,6 +2529,7 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> > >  				   enum ov5640_frame_rate fr,
> > >  				   const struct ov5640_mode_info **new_mode)
> > >  {
> > > +	unsigned int bpp = ov5640_code_to_bpp(fmt->code);
> > >  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > >  	const struct ov5640_mode_info *mode;
> > >  	int i;
> > > @@ -2536,6 +2537,17 @@ static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
> > >  	mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
> > >  	if (!mode)
> > >  		return -EINVAL;
> > > +
> > > +	/*
> > > +	 * Adjust mode according to bpp:
> > > +	 * - 8bpp modes work for resolution >= 1280x720
> > > +	 * - 24bpp modes work resolution < 1280x720
> > > +	 */
> > > +	if (bpp == 8 && mode->crop.width < 1280)
> > > +		mode = &ov5640_mode_data[OV5640_MODE_720P_1280_720];
> > > +	else if (bpp == 24 && mode->crop.width > 1024)
> > > +		mode = &ov5640_mode_data[OV5640_MODE_XGA_1024_768];
> >
> > This is in line with patch 20/21, so
> >
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> > but I'm still not sure about the limitation for 8bpp.
> 
> Me neither. I had on my todo to test 8bpp with different pixel clock
> rates to see if it makes any difference. If only 1080p and full res
> were working I would have guessed it was because those are the only
> modes obtained by cropping the full pixel array without any
> subsampling being applied, but as 1280x720 works too....
> 
> One thinking I had was that the pixel_rate assigned to each mode where
> too slow for an 8bpp mode and that the resulting CSI-2 frequencies
> were consequentially below the minimum required ones, but I would have
> to re-calculate to see if that's the case.

That's something I considered too.

> Anyway, as said in the cover letter, I have now assigned a pixel_rate
> to each mode, regardless of the image format. As the same 'mode' in
> 24bpp or 8bpp requires different pixel rates, I think this first
> approach is good, but to fully exploit all the modes the pixel_rate
> should be controlled by userspace too, in function of the bpp (and the
> number of data lanes). My understanding is that it should happen not
> by changing PIXEL_RATE but rather LINK_FREQ. But with the latter being
> a menu control (something I still don't get the reason for) there's a
> limited number of combination that could be supported there...

That's correct, PIXEL_RATE is read-only while LINK_FREQ is read-write.

LINK_FREQ is a menu control because link frequencies need to be selected
with the whole system taken into consideration, to avoid EMC issues. The
possible link frequencies are thus expected to be specified in DT. I'm
not a big fan of this. While the concept make sense, it's confusing and
under-documented. We're especially missing documentation for sensor
driver developers.

> We'll see, I would have been happy enough if 16bpp gets actually
> fixed when it comes to frame durations, with 8bpp/24bpp being usable.
> 
> We can improve on top of an hopefully now more solid base.

Sure.

> > > +
> > >  	fmt->width = mode->crop.width;
> > >  	fmt->height = mode->crop.height;
> > >

-- 
Regards,

Laurent Pinchart

Re: [PATCH 11/21] media: ov5640: Add VBLANK control

[Laurent Pinchart]

Hi Jacopo,

On Mon, Feb 07, 2022 at 04:09:12PM +0100, Jacopo Mondi wrote:
> On Wed, Feb 02, 2022 at 11:35:31PM +0200, Laurent Pinchart wrote:
> > On Mon, Jan 31, 2022 at 03:32:35PM +0100, Jacopo Mondi wrote:
> > > Add the VBLANK control which allows to select the duration of the
> > > frame vertical blankings and allows to control the framerate.
> > >
> > > The VBLANK control also modifies the exposure time range, which cannot
> > > exceed the maximum frame length.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > >  drivers/media/i2c/ov5640.c | 50 ++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 50 insertions(+)
> > >
> > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > index b2961e8e07c3..6eeb50724195 100644
> > > --- a/drivers/media/i2c/ov5640.c
> > > +++ b/drivers/media/i2c/ov5640.c
> > > @@ -31,6 +31,10 @@
> > >
> > >  #define OV5640_LINK_RATE_MAX	490000000U
> > >
> > > +/* FIXME: not documented. */
> > > +#define OV5640_MIN_VBLANK	24
> > > +#define OV5640_MAX_VTS		1968
> > > +
> > >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> > >
> > >  #define OV5640_REG_SYS_RESET02		0x3002
> > > @@ -267,6 +271,7 @@ struct ov5640_ctrls {
> > >  	struct v4l2_ctrl *pixel_rate;
> > >  	struct v4l2_ctrl *link_freq;
> > >  	struct v4l2_ctrl *hblank;
> > > +	struct v4l2_ctrl *vblank;
> > >  	struct {
> > >  		struct v4l2_ctrl *auto_exp;
> > >  		struct v4l2_ctrl *exposure;
> > > @@ -2531,6 +2536,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> > >  	struct v4l2_mbus_framefmt *fmt = &sensor->fmt;
> > >  	enum ov5640_pixel_rate_id pixel_rate_id = mode->pixel_rate;
> > >  	u32 num_lanes = sensor->ep.bus.mipi_csi2.num_data_lanes;
> > > +	s64 exposure_val, exposure_max;
> > >  	unsigned int hblank;
> > >  	unsigned int i = 0;
> > >  	u32 pixel_rate;
> > > @@ -2586,6 +2592,20 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
> > >  	__v4l2_ctrl_modify_range(sensor->ctrls.hblank,
> > >  				 hblank, hblank, 1, hblank);
> > >
> > > +	__v4l2_ctrl_modify_range(sensor->ctrls.vblank,
> > > +				 OV5640_MIN_VBLANK,
> > > +				 OV5640_MAX_VTS - mode->crop.height, 1,
> > > +				 mode->vblank_def);
> > > +	__v4l2_ctrl_s_ctrl(sensor->ctrls.vblank, mode->vblank_def);
> > > +
> > > +	exposure_max = mode->crop.height + mode->vblank_def - OV5640_MIN_VBLANK;
> >
> > There's typically a fixed margin of a few lines between the maximum
> > exposure and the total vertical size, but that's usually smaller than
> > the minimum vblank value. Are you sure OV5640_MIN_VBLANK is right here ?
> 
> Section "4.6.2 manual exposure control" of the datasheet reports
> some hints about how to enlarge the total frame size to accomodate the
> desired exposure.
> 
> It is not totally clear to me how though. The maximum exposure time
> seems to be limited not only by VTS, but by
> (VTS + [0x350Cd,0x350d]).
> 
> The 0x350c/d registers are said to:
> 
> In auto exposure mode, the extra exposure values (larger than 1 frame)
> in registers 0x350C/0x350D automatically change. In manual exposure
> mode, these registers will not automatically change. The manually set
> exposure in registers 0x3500~0x3502 must be less than the maximum
> exposure value in {0x380E, 0x380F} + {0x350C,0x350D}
> 
> 
> with:
> - 0x350c,0x350d = PK_VTS
> - 0x380e,0x380f = VTS
> 
> I have no idea how manual exposure works with this sensor, as I see
> 0x350c only being read and never programmed.
> 
> This anyway does not suggest any margin between VTS+PK_VTS but rahter
> that the exposure time can be larger than VTS which doesn't make much
> sense to me (maybe I should just consider 'real VTS' = VTS + PK_VTS).

It's quite confusing, as it doesn't tell what PK_VTS is. Note that in
the auto-focus application note, section "4.11.1 Shutter" says

  The shutter of OV5640 controls exposure time. The unit of shutter is
  line period. Shutter value has limitation for each resolution. The
  maximum shutter value is VTS(0x380e, 0x380f) - 4.

Furthermore, the OV5642 datasheet ([1]) has a slightly different
explanation:

  4.6.2 manual exposure control

  To manually change exposure value, you must first set both 0x3503[0]
  and 0x3503[2], where 0x3503[0] enables a manual exposure control and
  0x3503[2] enables manual frame length - the number of lines in each
  frame or maximum exposure time, which is defined by registers 0x350C
  and 0x350D. In auto exposure mode, the maximum exposure values in
  registers 0x350C/0x350D automatically change. In manual exposure mode,
  these registers will not automatically change. The manually set
  exposure in registers 0x3500~0x03502 must be less than the maximum
  exposure value in 0x350C/0x350D. The exposure value in registers
  0x3500~0x3502 is in units of line*16 - the low 4 bits (0x3502[3:0]) is
  the fraction of line, the maximum value in 0x350C/0x350D is in unit of
  line. If the manually set exposure value is less than one pre-defined
  frame period (e.g., 1/15 second in 15fps), there is no need to change
  0x350C/0x350D. If the exposure value needs to be set beyond the
  pre-defined frame period; in another words, if the frame period needs
  to be extended to extend exposure time, then the maximum frame value
  in 0x350C/0x350D needs to be set first, then the exposure can be set
  in registers 0x3500~0x3502 accordingly.

I know it's a different sensor, so the implementation could be
different, but the register addresses and names seem to match.

For what it's worth, "PK" could stand for "power keep" according to
table 7-5 in that document. I have no idea what that means though :-)

[1] https://www.uctronics.com/download/cam_module/OV5642DS.pdf

> Anyway, being aware of such margins being usually required, I used the
> min VBLANK value. Which is also not documented but I experimentally
> tried to progressively shrink until capture did not break.
> 
> Should I remove it from the above calculation ?

More research, experimentation and reverse engineering is likely
required to figure this out, but I think it's a bit out of scope.
Whatever we do now will likely be wrong somehow anyway, so you can pick
the option that you think is the most logical, and we'll fix that later
if/when needed.

> > > +	exposure_val = clamp((s64)sensor->ctrls.exposure->val,
> > > +			     (s64)sensor->ctrls.exposure->minimum,
> > > +			     (s64)exposure_max);
> > > +	__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > > +				 sensor->ctrls.exposure->minimum,
> > > +				 exposure_max, 1, exposure_val);
> > > +
> > >  	return 0;
> > >  }
> > >
> > > @@ -2958,6 +2978,15 @@ static int ov5640_set_ctrl_vflip(struct ov5640_dev *sensor, int value)
> > >  			      (BIT(2) | BIT(1)) : 0);
> > >  }
> > >
> > > +static int ov5640_set_ctrl_vblank(struct ov5640_dev *sensor, int value)
> > > +{
> > > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > > +
> > > +	/* Update the VTOT timing register value. */
> > > +	return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS,
> > > +				  mode->crop.height + value);
> > > +}
> > > +
> > >  static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
> > >  {
> > >  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> > > @@ -2988,10 +3017,22 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> > >  {
> > >  	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
> > >  	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > >  	int ret;
> > >
> > >  	/* v4l2_ctrl_lock() locks our own mutex */
> > >
> > > +	switch (ctrl->id) {
> > > +	case V4L2_CID_VBLANK:
> > > +		/* Update the exposure range to the newly programmed vblank. */
> > > +		unsigned int max = mode->crop.height + ctrl->val - OV5640_MIN_VBLANK;
> > > +
> > > +		__v4l2_ctrl_modify_range(sensor->ctrls.exposure,
> > > +					 sensor->ctrls.exposure->minimum,
> > > +					 max, sensor->ctrls.exposure->step, max);
> >
> > Should the default be set to mode->vblank_def ?
> >
> 
> It should!
> 
> > > +		break;
> > > +	}
> > > +
> > >  	/*
> > >  	 * If the device is not powered up by the host driver do
> > >  	 * not apply any controls to H/W at this time. Instead
> > > @@ -3031,6 +3072,9 @@ static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
> > >  	case V4L2_CID_VFLIP:
> > >  		ret = ov5640_set_ctrl_vflip(sensor, ctrl->val);
> > >  		break;
> > > +	case V4L2_CID_VBLANK:
> > > +		ret = ov5640_set_ctrl_vblank(sensor, ctrl->val);
> > > +		break;
> > >  	default:
> > >  		ret = -EINVAL;
> > >  		break;
> > > @@ -3050,6 +3094,7 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> > >  	const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
> > >  	struct ov5640_ctrls *ctrls = &sensor->ctrls;
> > >  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> > > +	unsigned int max_vblank;
> > >  	unsigned int hblank;
> > >  	int ret;
> > >
> > > @@ -3073,6 +3118,11 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
> > >  	ctrls->hblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HBLANK, hblank,
> > >  					  hblank, 1, hblank);
> > >
> > > +	max_vblank = OV5640_MAX_VTS - mode->crop.height;
> > > +	ctrls->vblank = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_VBLANK,
> > > +					  OV5640_MIN_VBLANK, max_vblank,
> > > +					  1, mode->vblank_def);
> > > +
> > >  	/* Auto/manual white balance */
> > >  	ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
> > >  					   V4L2_CID_AUTO_WHITE_BALANCE,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 14/21] media: ov5640: Implement get_selection

[Laurent Pinchart]

Hi Jacopo,

On Mon, Feb 07, 2022 at 04:47:35PM +0100, Jacopo Mondi wrote:
> On Thu, Feb 03, 2022 at 12:29:47AM +0200, Laurent Pinchart wrote:
> > On Mon, Jan 31, 2022 at 03:44:42PM +0100, Jacopo Mondi wrote:
> > > Implement the get_selection pad operation for the OV5640 sensor driver.
> > >
> > > The supported targets report the sensor's native size, the active pixel
> > > array size and the analog crop rectangle from which the image is
> > > produced.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > >  drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 61 insertions(+)
> > >
> > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > index 762bdca83aec..ae22300b9655 100644
> > > --- a/drivers/media/i2c/ov5640.c
> > > +++ b/drivers/media/i2c/ov5640.c
> > > @@ -35,6 +35,13 @@
> > >  #define OV5640_MIN_VBLANK	24
> > >  #define OV5640_MAX_VTS		1968
> > >
> > > +#define OV5640_NATIVE_WIDTH		2624
> > > +#define OV5640_NATIVE_HEIGHT		1964
> > > +#define OV5640_PIXEL_ARRAY_TOP		8
> > > +#define OV5640_PIXEL_ARRAY_LEFT		16
> > > +#define OV5640_PIXEL_ARRAY_WIDTH	2592
> > > +#define OV5640_PIXEL_ARRAY_HEIGHT	1944
> >
> > According to the datasheet, the sensor has 8 black lines, 6 dummy lines,
> > 1944 active lines and 6 dummy lines. Horizontally, it has 16 dummy
> > columns, 2592 active columns, and 16 dummy columns. If "pixel array" is
> > meant to refer to the active area (I dislike the "active" name here, as
> > the dummy lines and columns are typically "active" too, but that's a
> > digression), then top should be 14.
> 
> Corret, I have only considered the 8 black lines, but dummy too are
> 'active but not valid' ones.
> 
> I'll change to 14
> 
> > > +
> > >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> > >
> > >  #define OV5640_REG_SYS_RESET02		0x3002
> > > @@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> > >  	return ret;
> > >  }
> > >
> > > +static int ov5640_get_selection(struct v4l2_subdev *sd,
> > > +				struct v4l2_subdev_state *sd_state,
> > > +				struct v4l2_subdev_selection *sel)
> > > +{
> > > +	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > > +	const struct v4l2_rect *crop = &mode->crop;
> > > +
> > > +	switch (sel->target) {
> > > +	case V4L2_SEL_TGT_CROP: {
> > > +		mutex_lock(&sensor->lock);
> > > +
> > > +		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> > > +		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> > > +		sel->r.width = analog_crop->width
> > > +			     - analog_crop->left - crop->left;
> >
> > Why do you subtract the left coordinates here ?
> 
> As the analog_crop->width is defined from the full pixel array size
> (black + dummy + active). The TGT_CROP rectangle width and height
> should instead report the dimensions of the portion of the pixel array
> size which is processed to obtain the final image, and thus the
> vertical and horizontal offsets should be subtracted
> 
> 
>     full                            analog_crop.width
>      |                              (0x3804, 0x3805) = 2624
>      |                                   |
>      V                                   V
>   -> +-----------------------------------+
>      |                                   |
>      x-----------------------------------|
>      |  x----  TGT_CROP width  ----------|
>      |  |                                |
>      |  |                                |
>      |  |                                |
>      |  |                                |
>      |  |                                |
>      |  |                                |
>      |  |                                |
>      +--|--------------------------------+
>      |  |
>      |  |-> crop.left = 16
>      |
>      |-> analog_crop.left = 0

With analog_crop.left == 0 that's fine, but if it had a different value,
shouldn't the above use

		sel->r.width = analog_crop->width - crop->left;

?

> > > +		sel->r.height = analog_crop->height
> > > +			      - analog_crop->top - crop->top;
> > > +
> > > +		mutex_unlock(&sensor->lock);
> > > +
> > > +		return 0;
> > > +	}
> > > +
> > > +	case V4L2_SEL_TGT_NATIVE_SIZE:
> > > +		sel->r.top = 0;
> > > +		sel->r.left = 0;
> > > +		sel->r.width = OV5640_NATIVE_WIDTH;
> > > +		sel->r.height = OV5640_NATIVE_HEIGHT;
> > > +
> > > +		return 0;
> > > +
> > > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > +		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
> > > +		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
> > > +		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
> > > +		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;
> >
> > In libcamera we use V4L2_SEL_TGT_CROP_BOUNDS to set PixelArraySize,
> > ignoring the left and top coordinates, and V4L2_SEL_TGT_CROP_DEFAULT to
> > to set the PixelArrayActiveAreas property relative to PixelArraySize.
> > This means that non-zero values for the left and top coordinates of
> > V4L2_SEL_TGT_CROP_BOUNDS will cause issues. Is this an issue in
> > libcamera, or should V4L2_SEL_TGT_CROP_BOUNDS be changed here ?
> >
> > The related question is, can we read the optical black lines and the
> > dummy lines and columns ?
> >
> > > +
> > > +		return 0;
> > > +	}
> > > +
> > > +	return -EINVAL;
> > > +}
> > > +
> > >  static int ov5640_set_framefmt(struct ov5640_dev *sensor,
> > >  			       struct v4l2_mbus_framefmt *format)
> > >  {
> > > @@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
> > >  	.enum_mbus_code = ov5640_enum_mbus_code,
> > >  	.get_fmt = ov5640_get_fmt,
> > >  	.set_fmt = ov5640_set_fmt,
> > > +	.get_selection = ov5640_get_selection,
> > >  	.enum_frame_size = ov5640_enum_frame_size,
> > >  	.enum_frame_interval = ov5640_enum_frame_interval,
> > >  };
> > > @@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> > >  {
> > >  	struct v4l2_mbus_framefmt *try_fmt =
> > >  		v4l2_subdev_get_try_format(sd, fh->state, 0);
> > > +	struct v4l2_rect *try_crop =
> > > +		v4l2_subdev_get_try_crop(sd, fh->state, 0);
> > >
> > >  	*try_fmt = ov5640_default_fmt;
> > >
> > > +	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
> > > +	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
> > > +	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
> > > +	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
> > > +
> > >  	return 0;
> > >  }
> > >

-- 
Regards,

Laurent Pinchart

Re: [PATCH 14/21] media: ov5640: Implement get_selection

[Jacopo Mondi]

Hi Laurent
  sorry, I missed the last question in your comment

On Mon, Feb 07, 2022 at 07:53:45PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Mon, Feb 07, 2022 at 04:47:35PM +0100, Jacopo Mondi wrote:
> > On Thu, Feb 03, 2022 at 12:29:47AM +0200, Laurent Pinchart wrote:
> > > On Mon, Jan 31, 2022 at 03:44:42PM +0100, Jacopo Mondi wrote:
> > > > Implement the get_selection pad operation for the OV5640 sensor driver.
> > > >
> > > > The supported targets report the sensor's native size, the active pixel
> > > > array size and the analog crop rectangle from which the image is
> > > > produced.
> > > >
> > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > > ---
> > > >  drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 61 insertions(+)
> > > >
> > > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > > index 762bdca83aec..ae22300b9655 100644
> > > > --- a/drivers/media/i2c/ov5640.c
> > > > +++ b/drivers/media/i2c/ov5640.c
> > > > @@ -35,6 +35,13 @@
> > > >  #define OV5640_MIN_VBLANK	24
> > > >  #define OV5640_MAX_VTS		1968
> > > >
> > > > +#define OV5640_NATIVE_WIDTH		2624
> > > > +#define OV5640_NATIVE_HEIGHT		1964
> > > > +#define OV5640_PIXEL_ARRAY_TOP		8
> > > > +#define OV5640_PIXEL_ARRAY_LEFT		16
> > > > +#define OV5640_PIXEL_ARRAY_WIDTH	2592
> > > > +#define OV5640_PIXEL_ARRAY_HEIGHT	1944
> > >
> > > According to the datasheet, the sensor has 8 black lines, 6 dummy lines,
> > > 1944 active lines and 6 dummy lines. Horizontally, it has 16 dummy
> > > columns, 2592 active columns, and 16 dummy columns. If "pixel array" is
> > > meant to refer to the active area (I dislike the "active" name here, as
> > > the dummy lines and columns are typically "active" too, but that's a
> > > digression), then top should be 14.
> >
> > Corret, I have only considered the 8 black lines, but dummy too are
> > 'active but not valid' ones.
> >
> > I'll change to 14
> >
> > > > +
> > > >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> > > >
> > > >  #define OV5640_REG_SYS_RESET02		0x3002
> > > > @@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> > > >  	return ret;
> > > >  }
> > > >
> > > > +static int ov5640_get_selection(struct v4l2_subdev *sd,
> > > > +				struct v4l2_subdev_state *sd_state,
> > > > +				struct v4l2_subdev_selection *sel)
> > > > +{
> > > > +	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > > > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > > > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > > > +	const struct v4l2_rect *crop = &mode->crop;
> > > > +
> > > > +	switch (sel->target) {
> > > > +	case V4L2_SEL_TGT_CROP: {
> > > > +		mutex_lock(&sensor->lock);
> > > > +
> > > > +		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> > > > +		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> > > > +		sel->r.width = analog_crop->width
> > > > +			     - analog_crop->left - crop->left;
> > >
> > > Why do you subtract the left coordinates here ?
> >
> > As the analog_crop->width is defined from the full pixel array size
> > (black + dummy + active). The TGT_CROP rectangle width and height
> > should instead report the dimensions of the portion of the pixel array
> > size which is processed to obtain the final image, and thus the
> > vertical and horizontal offsets should be subtracted
> >
> >
> >     full                            analog_crop.width
> >      |                              (0x3804, 0x3805) = 2624
> >      |                                   |
> >      V                                   V
> >   -> +-----------------------------------+
> >      |                                   |
> >      x-----------------------------------|
> >      |  x----  TGT_CROP width  ----------|
> >      |  |                                |
> >      |  |                                |
> >      |  |                                |
> >      |  |                                |
> >      |  |                                |
> >      |  |                                |
> >      |  |                                |
> >      +--|--------------------------------+
> >      |  |
> >      |  |-> crop.left = 16
> >      |
> >      |-> analog_crop.left = 0
>
> With analog_crop.left == 0 that's fine, but if it had a different value,
> shouldn't the above use
>
> 		sel->r.width = analog_crop->width - crop->left;
>
> ?

I don't think so... analog_crop->width counts from 0 (it corresponds
to reg 0x3804 in figure 4.2) and looking at the picture again, it
seems the the actual crop should rather be

                analog_crop->width - analog_crop->left

?


>
> > > > +		sel->r.height = analog_crop->height
> > > > +			      - analog_crop->top - crop->top;
> > > > +
> > > > +		mutex_unlock(&sensor->lock);
> > > > +
> > > > +		return 0;
> > > > +	}
> > > > +
> > > > +	case V4L2_SEL_TGT_NATIVE_SIZE:
> > > > +		sel->r.top = 0;
> > > > +		sel->r.left = 0;
> > > > +		sel->r.width = OV5640_NATIVE_WIDTH;
> > > > +		sel->r.height = OV5640_NATIVE_HEIGHT;
> > > > +
> > > > +		return 0;
> > > > +
> > > > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > > > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > > +		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
> > > > +		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
> > > > +		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
> > > > +		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;
> > >
> > > In libcamera we use V4L2_SEL_TGT_CROP_BOUNDS to set PixelArraySize,
> > > ignoring the left and top coordinates, and V4L2_SEL_TGT_CROP_DEFAULT to
> > > to set the PixelArrayActiveAreas property relative to PixelArraySize.
> > > This means that non-zero values for the left and top coordinates of
> > > V4L2_SEL_TGT_CROP_BOUNDS will cause issues. Is this an issue in
> > > libcamera, or should V4L2_SEL_TGT_CROP_BOUNDS be changed here ?
> > >

For reference:
Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP_BOUNDS`_ to report the readable pixel array area size
Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP_DEFAULT`_ to report the active pixel array area size
Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP`_ to report the analogue selection rectangle

What I see in libcamera::camera_sensor.cpp is

        validateSensorDriver()
        {
                /* Readable pixels */
                pixelArraySize = TGT_BOUNDS ( = [2592, 1944])

                /* Active (or better, valid) pixel array) */
                activeArea = TGT_DEFAULT ( = [14, 16, 2592, 1944])
        }

        sensorInfo()
        {
                /* Analog crop */
                analogCrop = TGT_CROP

                /*
                Gets assembled in the driver as:

		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
		sel->r.width = analog_crop->width
			     - analog_crop->left - crop->left;
		sel->r.height = analog_crop->height
			      - analog_crop->top - crop->top;

                */

                /*
                 * Adjust the V4L2 TGT_CROP to comply with libcamera
                 * analogueCrop
                 */
                analogCrop.x -= activeArea_.x;
                analogCrop.y -= activeArea_.y;
        }

What is not correct that DEFAULT and BOUND have the same size.

If both BOUND and DEFAULT has a size of 2592x1944, then DEFAULT cannot
have (14, 16) as top-left corner, unless we read the last dummy
columns/lines.

So I would

        readable pixel array = (2624, 1964) = TGT_BOUND
        active pixel array (PIXEL_ARRAY_TOP, PIXEL_ARRAY_LEFT, 2592, 1944) = TGT_DEFAULT

        TGT_CROP = {
                    .top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP,
                    .left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
                    .width = analog_crop->width - analog_crop->left - OV5640_PIXEL_ARRAY_LEFT,
                    .height = analog_crop->height - analog_crop->top - OV5640_PIXEL_ARRAY_TOP,
                   }

       which gets adjusted in libcamera as the V4L2 TGT_CROP is defined from
       BOUND while libcamera::analogueCrop is defined from
       libcamera::activeArea ( == TGT_DEFAULT)

        analog_crop = {
                        .top =  TGT_CROP.top - activeArea.top,
                        .left = TGT_CROP.left - activeArea.left,
                        .width = TGT_CROP.width,
                        .height = TGT_CROP.height
                      }

I do however see a problem caused by the current definition of the
driver's mode sizes

      TGT_CROP.width = analog_crop->width - analog_crop->left - OV5640_PIXEL_ARRAY_LEFT
                     = 2624 - 0 - 16 = 2608

which means the chip is instructed to read dummy columns at the end
(as analog_crop.width gets written to register 0x3804 and counts from 0)

We should probably avoid that by changing analog_crop->width and
in the sensor's modes definition.

My goal would then be to see in libcamera:

        Readable = { 2624, 1964 }
        Active = { 14, 16, 2592, 1944 }

        Crop depends on mode, but for VGA in example, asuming I'll fix
        the width and heigh to skip the last dummy cols/lines:

		.analog_crop = {
			.left	= 0,
			.top	= 4,
			.width	= 2608, /* Skip last 16 dummy cols */
			.height	= 1958, /* Skip last 6 dummy lines */
		},

        it should then resul in libcamera in as {0, 4, 2592, 1940 }

Now I wonder if the 4 lines skipped by analog_crop.top are actually
necessary or it's cruft from the existing register tables definitions.

I'll try to remove them and have the full valid pixel array passed in
to the chip's ISP.

> > > The related question is, can we read the optical black lines and the
> > > dummy lines and columns ?
> > >

Not tested, they're addressable (in theory) so I would make BOUND !=
DEFAULT, as explained above.

Sorry for the long email :/

> > > > +
> > > > +		return 0;
> > > > +	}
> > > > +
> > > > +	return -EINVAL;
> > > > +}
> > > > +
> > > >  static int ov5640_set_framefmt(struct ov5640_dev *sensor,
> > > >  			       struct v4l2_mbus_framefmt *format)
> > > >  {
> > > > @@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
> > > >  	.enum_mbus_code = ov5640_enum_mbus_code,
> > > >  	.get_fmt = ov5640_get_fmt,
> > > >  	.set_fmt = ov5640_set_fmt,
> > > > +	.get_selection = ov5640_get_selection,
> > > >  	.enum_frame_size = ov5640_enum_frame_size,
> > > >  	.enum_frame_interval = ov5640_enum_frame_interval,
> > > >  };
> > > > @@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> > > >  {
> > > >  	struct v4l2_mbus_framefmt *try_fmt =
> > > >  		v4l2_subdev_get_try_format(sd, fh->state, 0);
> > > > +	struct v4l2_rect *try_crop =
> > > > +		v4l2_subdev_get_try_crop(sd, fh->state, 0);
> > > >
> > > >  	*try_fmt = ov5640_default_fmt;
> > > >
> > > > +	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
> > > > +	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
> > > > +	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
> > > > +	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
> > > > +
> > > >  	return 0;
> > > >  }
> > > >
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 14/21] media: ov5640: Implement get_selection

[Laurent Pinchart]

Hi Jacopo,

On Tue, Feb 08, 2022 at 03:18:48PM +0100, Jacopo Mondi wrote:
> On Mon, Feb 07, 2022 at 07:53:45PM +0200, Laurent Pinchart wrote:
> > On Mon, Feb 07, 2022 at 04:47:35PM +0100, Jacopo Mondi wrote:
> > > On Thu, Feb 03, 2022 at 12:29:47AM +0200, Laurent Pinchart wrote:
> > > > On Mon, Jan 31, 2022 at 03:44:42PM +0100, Jacopo Mondi wrote:
> > > > > Implement the get_selection pad operation for the OV5640 sensor driver.
> > > > >
> > > > > The supported targets report the sensor's native size, the active pixel
> > > > > array size and the analog crop rectangle from which the image is
> > > > > produced.
> > > > >
> > > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > > > ---
> > > > >  drivers/media/i2c/ov5640.c | 61 ++++++++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 61 insertions(+)
> > > > >
> > > > > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > > > > index 762bdca83aec..ae22300b9655 100644
> > > > > --- a/drivers/media/i2c/ov5640.c
> > > > > +++ b/drivers/media/i2c/ov5640.c
> > > > > @@ -35,6 +35,13 @@
> > > > >  #define OV5640_MIN_VBLANK	24
> > > > >  #define OV5640_MAX_VTS		1968
> > > > >
> > > > > +#define OV5640_NATIVE_WIDTH		2624
> > > > > +#define OV5640_NATIVE_HEIGHT		1964
> > > > > +#define OV5640_PIXEL_ARRAY_TOP		8
> > > > > +#define OV5640_PIXEL_ARRAY_LEFT		16
> > > > > +#define OV5640_PIXEL_ARRAY_WIDTH	2592
> > > > > +#define OV5640_PIXEL_ARRAY_HEIGHT	1944
> > > >
> > > > According to the datasheet, the sensor has 8 black lines, 6 dummy lines,
> > > > 1944 active lines and 6 dummy lines. Horizontally, it has 16 dummy
> > > > columns, 2592 active columns, and 16 dummy columns. If "pixel array" is
> > > > meant to refer to the active area (I dislike the "active" name here, as
> > > > the dummy lines and columns are typically "active" too, but that's a
> > > > digression), then top should be 14.
> > >
> > > Corret, I have only considered the 8 black lines, but dummy too are
> > > 'active but not valid' ones.
> > >
> > > I'll change to 14
> > >
> > > > > +
> > > > >  #define OV5640_DEFAULT_SLAVE_ID 0x3c
> > > > >
> > > > >  #define OV5640_REG_SYS_RESET02		0x3002
> > > > > @@ -2667,6 +2674,52 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> > > > >  	return ret;
> > > > >  }
> > > > >
> > > > > +static int ov5640_get_selection(struct v4l2_subdev *sd,
> > > > > +				struct v4l2_subdev_state *sd_state,
> > > > > +				struct v4l2_subdev_selection *sel)
> > > > > +{
> > > > > +	struct ov5640_dev *sensor = to_ov5640_dev(sd);
> > > > > +	const struct ov5640_mode_info *mode = sensor->current_mode;
> > > > > +	const struct v4l2_rect *analog_crop = &mode->analog_crop;
> > > > > +	const struct v4l2_rect *crop = &mode->crop;
> > > > > +
> > > > > +	switch (sel->target) {
> > > > > +	case V4L2_SEL_TGT_CROP: {
> > > > > +		mutex_lock(&sensor->lock);
> > > > > +
> > > > > +		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> > > > > +		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> > > > > +		sel->r.width = analog_crop->width
> > > > > +			     - analog_crop->left - crop->left;
> > > >
> > > > Why do you subtract the left coordinates here ?
> > >
> > > As the analog_crop->width is defined from the full pixel array size
> > > (black + dummy + active). The TGT_CROP rectangle width and height
> > > should instead report the dimensions of the portion of the pixel array
> > > size which is processed to obtain the final image, and thus the
> > > vertical and horizontal offsets should be subtracted
> > >
> > >
> > >     full                            analog_crop.width
> > >      |                              (0x3804, 0x3805) = 2624
> > >      |                                   |
> > >      V                                   V
> > >   -> +-----------------------------------+
> > >      |                                   |
> > >      x-----------------------------------|
> > >      |  x----  TGT_CROP width  ----------|
> > >      |  |                                |
> > >      |  |                                |
> > >      |  |                                |
> > >      |  |                                |
> > >      |  |                                |
> > >      |  |                                |
> > >      |  |                                |
> > >      +--|--------------------------------+
> > >      |  |
> > >      |  |-> crop.left = 16
> > >      |
> > >      |-> analog_crop.left = 0
> >
> > With analog_crop.left == 0 that's fine, but if it had a different value,
> > shouldn't the above use
> >
> > 		sel->r.width = analog_crop->width - crop->left;
> >
> > ?
> 
> I don't think so... analog_crop->width counts from 0 (it corresponds
> to reg 0x3804 in figure 4.2) and looking at the picture again, it
> seems the the actual crop should rather be
> 
>                 analog_crop->width - analog_crop->left
> 
> ?

I don't think so :-) We seem to have trouble understanding each other
here.

> > > > > +		sel->r.height = analog_crop->height
> > > > > +			      - analog_crop->top - crop->top;
> > > > > +
> > > > > +		mutex_unlock(&sensor->lock);
> > > > > +
> > > > > +		return 0;
> > > > > +	}
> > > > > +
> > > > > +	case V4L2_SEL_TGT_NATIVE_SIZE:
> > > > > +		sel->r.top = 0;
> > > > > +		sel->r.left = 0;
> > > > > +		sel->r.width = OV5640_NATIVE_WIDTH;
> > > > > +		sel->r.height = OV5640_NATIVE_HEIGHT;
> > > > > +
> > > > > +		return 0;
> > > > > +
> > > > > +	case V4L2_SEL_TGT_CROP_DEFAULT:
> > > > > +	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > > > +		sel->r.top = OV5640_PIXEL_ARRAY_TOP;
> > > > > +		sel->r.left = OV5640_PIXEL_ARRAY_LEFT;
> > > > > +		sel->r.width = OV5640_PIXEL_ARRAY_WIDTH;
> > > > > +		sel->r.height = OV5640_PIXEL_ARRAY_HEIGHT;
> > > >
> > > > In libcamera we use V4L2_SEL_TGT_CROP_BOUNDS to set PixelArraySize,
> > > > ignoring the left and top coordinates, and V4L2_SEL_TGT_CROP_DEFAULT to
> > > > to set the PixelArrayActiveAreas property relative to PixelArraySize.
> > > > This means that non-zero values for the left and top coordinates of
> > > > V4L2_SEL_TGT_CROP_BOUNDS will cause issues. Is this an issue in
> > > > libcamera, or should V4L2_SEL_TGT_CROP_BOUNDS be changed here ?
> 
> For reference:
> Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP_BOUNDS`_ to report the readable pixel array area size
> Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP_DEFAULT`_ to report the active pixel array area size
> Documentation/sensor_driver_requirements.rst:* `V4L2_SEL_TGT_CROP`_ to report the analogue selection rectangle
> 
> What I see in libcamera::camera_sensor.cpp is
> 
>         validateSensorDriver()
>         {
>                 /* Readable pixels */
>                 pixelArraySize = TGT_BOUNDS ( = [2592, 1944])
> 
>                 /* Active (or better, valid) pixel array) */
>                 activeArea = TGT_DEFAULT ( = [14, 16, 2592, 1944])
>         }
> 
>         sensorInfo()
>         {
>                 /* Analog crop */
>                 analogCrop = TGT_CROP
> 
>                 /*
>                 Gets assembled in the driver as:
> 
> 		sel->r.top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP;
> 		sel->r.left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
> 		sel->r.width = analog_crop->width
> 			     - analog_crop->left - crop->left;
> 		sel->r.height = analog_crop->height
> 			      - analog_crop->top - crop->top;
> 
>                 */
> 
>                 /*
>                  * Adjust the V4L2 TGT_CROP to comply with libcamera
>                  * analogueCrop
>                  */
>                 analogCrop.x -= activeArea_.x;
>                 analogCrop.y -= activeArea_.y;
>         }
> 
> What is not correct that DEFAULT and BOUND have the same size.
> 
> If both BOUND and DEFAULT has a size of 2592x1944, then DEFAULT cannot
> have (14, 16) as top-left corner, unless we read the last dummy
> columns/lines.
> 
> So I would
> 
>         readable pixel array = (2624, 1964) = TGT_BOUND
>         active pixel array (PIXEL_ARRAY_TOP, PIXEL_ARRAY_LEFT, 2592, 1944) = TGT_DEFAULT
> 
>         TGT_CROP = {
>                     .top = analog_crop->top + OV5640_PIXEL_ARRAY_TOP,
>                     .left = analog_crop->left + OV5640_PIXEL_ARRAY_LEFT;
>                     .width = analog_crop->width - analog_crop->left - OV5640_PIXEL_ARRAY_LEFT,
>                     .height = analog_crop->height - analog_crop->top - OV5640_PIXEL_ARRAY_TOP,
>                    }
> 
>        which gets adjusted in libcamera as the V4L2 TGT_CROP is defined from
>        BOUND while libcamera::analogueCrop is defined from
>        libcamera::activeArea ( == TGT_DEFAULT)
> 
>         analog_crop = {
>                         .top =  TGT_CROP.top - activeArea.top,
>                         .left = TGT_CROP.left - activeArea.left,
>                         .width = TGT_CROP.width,
>                         .height = TGT_CROP.height
>                       }
> 
> I do however see a problem caused by the current definition of the
> driver's mode sizes
> 
>       TGT_CROP.width = analog_crop->width - analog_crop->left - OV5640_PIXEL_ARRAY_LEFT
>                      = 2624 - 0 - 16 = 2608
> 
> which means the chip is instructed to read dummy columns at the end
> (as analog_crop.width gets written to register 0x3804 and counts from 0)
> 
> We should probably avoid that by changing analog_crop->width and
> in the sensor's modes definition.
> 
> My goal would then be to see in libcamera:
> 
>         Readable = { 2624, 1964 }
>         Active = { 14, 16, 2592, 1944 }
> 
>         Crop depends on mode, but for VGA in example, asuming I'll fix
>         the width and heigh to skip the last dummy cols/lines:
> 
> 		.analog_crop = {
> 			.left	= 0,
> 			.top	= 4,
> 			.width	= 2608, /* Skip last 16 dummy cols */
> 			.height	= 1958, /* Skip last 6 dummy lines */
> 		},
> 
>         it should then resul in libcamera in as {0, 4, 2592, 1940 }
> 
> Now I wonder if the 4 lines skipped by analog_crop.top are actually
> necessary or it's cruft from the existing register tables definitions.
> 
> I'll try to remove them and have the full valid pixel array passed in
> to the chip's ISP.

Note that ideally both the analog crop and the digital crop should be
exposed to userspace, and should be controllable independently. This
would require splitting the ov5640 into two subdevs, which I think is
overkill for a device that has reached EOL (not even mentioning the
backward compatibility issues).

> > > > The related question is, can we read the optical black lines and the
> > > > dummy lines and columns ?
> > > >
> 
> Not tested, they're addressable (in theory) so I would make BOUND !=
> DEFAULT, as explained above.
> 
> Sorry for the long email :/
> 
> > > > > +
> > > > > +		return 0;
> > > > > +	}
> > > > > +
> > > > > +	return -EINVAL;
> > > > > +}
> > > > > +
> > > > >  static int ov5640_set_framefmt(struct ov5640_dev *sensor,
> > > > >  			       struct v4l2_mbus_framefmt *format)
> > > > >  {
> > > > > @@ -3369,6 +3422,7 @@ static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
> > > > >  	.enum_mbus_code = ov5640_enum_mbus_code,
> > > > >  	.get_fmt = ov5640_get_fmt,
> > > > >  	.set_fmt = ov5640_set_fmt,
> > > > > +	.get_selection = ov5640_get_selection,
> > > > >  	.enum_frame_size = ov5640_enum_frame_size,
> > > > >  	.enum_frame_interval = ov5640_enum_frame_interval,
> > > > >  };
> > > > > @@ -3383,9 +3437,16 @@ static int ov5640_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> > > > >  {
> > > > >  	struct v4l2_mbus_framefmt *try_fmt =
> > > > >  		v4l2_subdev_get_try_format(sd, fh->state, 0);
> > > > > +	struct v4l2_rect *try_crop =
> > > > > +		v4l2_subdev_get_try_crop(sd, fh->state, 0);
> > > > >
> > > > >  	*try_fmt = ov5640_default_fmt;
> > > > >
> > > > > +	try_crop->left = OV5640_PIXEL_ARRAY_LEFT;
> > > > > +	try_crop->top = OV5640_PIXEL_ARRAY_TOP;
> > > > > +	try_crop->width = OV5640_PIXEL_ARRAY_WIDTH;
> > > > > +	try_crop->height = OV5640_PIXEL_ARRAY_HEIGHT;
> > > > > +
> > > > >  	return 0;
> > > > >  }
> > > > >

-- 
Regards,

Laurent Pinchart