AW: information about driver

AW: information about driver

[Sarah Wicker]

Hi Laurent and everyone,
Thanks a lot for your answer!

As I wrote at the beginning, I have a small issue with the driver concerning some of the control. The driver that is used was describe in this patch and mailing-list : https://www.spinics.net/lists/linux-media/msg159983.html

For the small story, I would like to interface the IMX296 with the board Ultra96 from avnet, it is built with an fpga from Xilinx. I already tested the pipeline in baremetal, I programmed the chip with the i2c port and was able to recover data from the data lane. Following this result I moved to implement the linux kernel and so I used the code forwarded upper, just changed the basic value of exposure and code format (somehow at the initialisation it recognizes the camera as colour camera even if it is monochrome), and also the devicetree is a bit different, here are the changes:
In driver:
static int imx296_ctrls_init(struct imx296 *imx){
[...] 
+ v4l2_ctrl_new_std(&imx->ctrls, &imx296_ctrl_ops,
			  V4L2_CID_EXPOSURE, 1, 1048575, 1, 1048575); //before 1104
[...]
}

static int imx296_enum_mbus_code(struct v4l2_subdev *sd,
				 struct v4l2_subdev_pad_config *cfg,
				 struct v4l2_subdev_mbus_code_enum *code){
[...]
+	code->code = MEDIA_BUS_FMT_Y10_1X10; //imx->mono ? MEDIA_BUS_FMT_Y10_1X10: MEDIA_BUS_FMT_SBGGR10_1X10;
	return 0;
}

static int imx296_set_format(struct v4l2_subdev *sd,
			     struct v4l2_subdev_pad_config *cfg,
			     struct v4l2_subdev_format *fmt)
{
[...]
+	format->code = MEDIA_BUS_FMT_Y10_1X10; //imx->mono ? MEDIA_BUS_FMT_Y10_1X10: MEDIA_BUS_FMT_SBGGR10_1X10;
	fmt->format = *format;

	return 0;
}

In the device-tree:
My device is controlled  via the i2c multiplexer (why it is a node of the i2c@2 interface)
i2c@2 {
					#address-cells = <0x1>;
					#size-cells = <0x0>;
					reg = <0x2>;
					label = "HS-I2C2";

					camera@1a {
						compatible = "sony,imx296";
						reg = <0x1a>;
						clocks = <0x29>;
						pwdn-gpios = <0x10 37 1>, <0x10 39 1>;
						clock-names = "inck"; 
						clock-frequency = <74250000>;
						vddd-supply = <0x33>;
						status = "okay";

						port {

							endpoint {
								clock-lanes = <0x0>;
								data-lanes = <0x1>;
								remote-endpoint = <0x2f>;
								link-frequencies =
							/bits/ 64 <1188000000>;
								linux,phandle = <0x28>;
								phandle = <0x28>;
							};
						};
					};
				};
The gpios (pwdn-gpios) come from the datasheet (from the mipi adapter board)
The clock is defined later in the .dts:
cam_clk@0 {
		#clock-cells = <0x0>;
		compatible = "fixed-clock";
		clock-frequency = <74250000>; 
		linux,phandle = <0x29>;
		phandle = <0x29>;
	};
As well as the regulator:
	fixed_regulator_DVDD@3 {
		compatible = "regulator-fixed";
		regulator-name = "vddd";
		regulator-min-microvolt = <0x16e360>;
		regulator-max-microvolt = <0x16e360>;
		phandle = <0x33>;
	};

The endpoint node of the camera is used to link to the sink pad of the mipi IP (mipi_csi2_rx_subsystem from Xilinx)

There is a Xilinx tool (petalinux) that can be used to create a linux kernel + rootfs + application. But I preferred not to use it and patched the driver to the linux-xlnx repository from Xilinx (just for practical reasons)

The driver is well instantiate, and using the v4l tools I can see my pipeline created (media-ctl -p an in the /dev the v4l-subdevice)
I also check the i2c line at the boot and can observe the configuration of the register been issued.

As explained before, using: media-ctl -v -V '"imx296 4-001a":0 [fmt:Y10_1X10/1440x1080 crop:(8,8)/1440x1080 field:none]' -d /dev/media0

I could modify the size of the frame we are receiving to remove the effective margin issued by the camera. 
Made a test using the yavta tool to capture several frame: the image issued by the camera stays black. Still using the yatva tool I tried modifying the parameters, changed the exposure, actually it was already at 1048575, so just confirmed it, and also changed the gain value. The tool send back that the modifications have been issued but :
1) I am not seeing anything on the i2c port
2) As a consequence of 1) there is no change in the image I am getting from the camera

As a "counter" solution I directly write on the i2c port using i2c tools to modify the register directly. 
Made capture => It works, the exposure is changed, and I observe changes in the images captured 

Change in the driver would be 
///in the define part:
#define EXPOSURE_MAX      16518000
#define EXPOSURE_MIN      29
#define EXPOSURE_DEFAULT  10000
#define TOFFSET           14260
#define T1H               14814
#define SHS_MAX           1117 //VMAX-1
#define SHS_MIN           4


static inline int imx296_set_exposure(struct imx296 *imx296, u32 value)
{
  int Status=0;
  u32 shs;
  //exposure depends of the SHS and VMAX
  //from the datasheet in the All pixel drive mode, Vmax is 1118, the number of INCK in 1H is 1100 => 1H=1100/74250000=14.814us
  //toffset is included in the exposure time and equal to 14.26us
  //if VMAX fix need to measure SHS to fit the value of exposure

  /*
  * for test will take a VMAX constant at 1118
  * Measure of the value of SHS to fit to the exposure value
  * 1) check if value {EXPOSURE_MAX, EXPOSURE_MIN}
  * 2) true : SHS = VMAX - [(value - toffset)/1H]
  * 3) false: SHS = VMAX - [(EXPOSURE_DEFAULT-toffset)/1H]= 444 =>0x1BC
   */
  if (value >= EXPOSURE_MIN && value <= EXPOSURE_MAX){
    shs= (SHS_MAX+1) - [(value - TOFFSET)/T1H];
    Status = imx296_write(imx296, IMX296_SHS1, shs);
  }
  else
    Status = imx296_write(imx296, IMX296_SHS1, 444);

  return Status;
}

//// in imx296_s_ctrl(struct v4l2_ctrl *ctrl) instead of the actual V4L2_CID_EXPOSURE  case:
  struct imx296 *imx = container_of(ctrl->handler, struct imx296, ctrls);
  	unsigned int vmax;
  	int ret = 0;

  	if (!imx->power_count)
  		return 0;

  switch (ctrl->id) {
  	case V4L2_CID_EXPOSURE:
-     /* Clamp the exposure value to VMAX. */
-    	  	vmax = imx->format.height + imx->vblank->cur.val;
-     		ctrl->val = min_t(int, ctrl->val, vmax);
-     		imx296_write(imx, IMX296_SHS1, vmax - ctrl->val, &ret);
+  				imx296_set_exposure(imx, ctrl->val);
  		break;


Hope this is written in a clear way. I haven't tested it yet, but would like to have your input :)


Wish you a great day!
Best regards,
Sarah Wicker

-----Ursprüngliche Nachricht-----
Von: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com] 
Gesendet: Freitag, 17. Juli 2020 03:23
An: Sarah Wicker <S.Wicker@PRIMES.de>
Betreff: Re: information about driver

Hello Sarah,

On Thu, Jul 16, 2020 at 01:59:21PM +0000, Sarah Wicker wrote:
> Good Morning to all,
> 
> I am currently working on a MIPI camera (imx296 monocolor) and Mr 
> Laurent Pichard, made a device driver, almost a year ago. Until now I 
> used it to integrate the camera with an ultra96 board(xilinx 
> ultrascale) into an embedded system.
> 
> I used the driver he developed to load the file from the camera as module.
> 
> However I have an issue, it seems that the controle of the exposure is 
> not working, at least no via v4l tools. I was able to change it by 
> programming the register of the camera directly with i2c.
> 
> I though of adding/modifying the configuration inside the code. But as 
> I am a complet beginner in this field I was wondering if you could 
> point me to some documentation/ressources?

First of all, could you tell me how you have (unsuccessfully) tried to modify the exposure time ?

> Also I was wondering of the compatibility of this code with xilinx 
> achitecture, since it isn’t part of the mainline kernel on git 
> (xilinx/linux-xlnx/drivers/
> media/i2c/) I mean it works for me but except for capture and setting 
> parameters like crop or the resolution I can’t modify more of the 
> controle parameters of the camera using the v4l tools.

I'm not aware of any particular incompatibility. I should resume work on upstreaming this driver, as it's mostly ready. Let's first try to address the exposure time setting issue, and we can then discuss the problems you are facing with the other controls.

Would you mind CC'ing the linux-media@vger.kernel.org mailing list in your reply ? This discussion can be useful for other people, and other developers may also be able to help.

--
Regards,

Laurent Pinchart