* question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 8:17 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-10 8:17 UTC (permalink / raw) To: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, Laurent Pinchart Cc: linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team hi, TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other cameras? I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: Using NXP's CSI bridge driver https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er as well as the CSI driver itself: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er works fine when using the ov5640 camera with this driver: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er (I realize there is a CSI bridge driver in staging, but that need more work to be actually used. Of course after this the goal is to fix and use it; and mainline a CSI phy driver too.) Now I use said NXP's CSI drivers on mainline without problems with the ov5640. For the hi846 (as an example), I want to use a different pixelformat and CSI media bus format. Describing, say, MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 streaming won't start anymore: "not negotiated, -4" and the "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which is how I test). When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is not yet full and "distorted" in a wrong format, which could explain why DMA is never completed in order to create a full frame). Now this is details, but the hi846 camera uses a 10bit CSI format and therefore I need to set PIXEL_BIT in cr1 too, but when I do that for example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course the "base address switching Change Err"). Do you have experience with using this driver with other cameras and can point me in a direction that might help me? If you want to look at the work-in-progress driver: https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c thanks a lot, martin ^ permalink raw reply [flat|nested] 24+ messages in thread
* question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 8:17 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-10 8:17 UTC (permalink / raw) To: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, Laurent Pinchart Cc: Pengutronix Kernel Team, kernel, NXP Linux Team, linux-arm-kernel, linux-media hi, TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other cameras? I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: Using NXP's CSI bridge driver https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er as well as the CSI driver itself: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er works fine when using the ov5640 camera with this driver: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er (I realize there is a CSI bridge driver in staging, but that need more work to be actually used. Of course after this the goal is to fix and use it; and mainline a CSI phy driver too.) Now I use said NXP's CSI drivers on mainline without problems with the ov5640. For the hi846 (as an example), I want to use a different pixelformat and CSI media bus format. Describing, say, MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 streaming won't start anymore: "not negotiated, -4" and the "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which is how I test). When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is not yet full and "distorted" in a wrong format, which could explain why DMA is never completed in order to create a full frame). Now this is details, but the hi846 camera uses a 10bit CSI format and therefore I need to set PIXEL_BIT in cr1 too, but when I do that for example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course the "base address switching Change Err"). Do you have experience with using this driver with other cameras and can point me in a direction that might help me? If you want to look at the work-in-progress driver: https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c thanks a lot, martin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-10 8:17 ` Martin Kepplinger @ 2020-12-10 9:23 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 9:23 UTC (permalink / raw) To: Martin Kepplinger Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Martin, On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > hi, > > TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > cameras? I've recently worked on camera support for i.MX8MM (whose camera IP cores are, if not identical, very similar to the i.MX8MQ's). The least I can say is that it was painful :-( I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and has a CSI-2 interface. My first use case is to capture RAW10, which isn't supported by the mx6s_capture driver. > I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > Using NXP's CSI bridge driver > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > as well as the CSI driver itself: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > works fine when using the ov5640 camera with this driver: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > > (I realize there is a CSI bridge driver in staging, but that need more > work to be actually used. Of course after this the goal is to fix and > use it; and mainline a CSI phy driver too.) I have lots of patches for this driver, which I've developed on i.MX7D for a separate project. I'd like to mainline them, but this is blocked by one last issue that I haven't been able to solve yet. In a nutshell, the CSI writes two consecutive frames in each buffer, overflowing the allocated memory. The registers that control the buffer size seem to be programmed correctly as far as I can tell. I've reported this issue to NXP but haven't received any feedback yet. I've also added support for i.MX8MM to this driver, but haven't been able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best images", but seems to drop 3 out of 4 consecutive pixels. The sensor driver has been tested successfully on i.MX6 so I don't think it's the most likely cause of issues, but I can't rule out bugs on that side either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP cores but doesn't tell much about how the two are connected, so I have lots of unanswered questions about the register fields that control the interface between those IP cores. I'm pretty sure there are issues in that area of the drivers, but I've tested all combinations I could think of, without luck. > Now I use said NXP's CSI drivers on mainline without problems with the > ov5640. For the hi846 (as an example), I want to use a different > pixelformat and CSI media bus format. Describing, say, > MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > streaming won't start anymore: "not negotiated, -4" and the > "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? That will likely be the easy part of the problem. The NXP driver doesn't support RAW10 at all, so you will have to add that, and I don't think it will be easy. I'll push my development branch shortly if you're interested in trying the mainline driver. > I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > is how I test). > > When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > not yet full and "distorted" in a wrong format, which could explain why > DMA is never completed in order to create a full frame). > > Now this is details, but the hi846 camera uses a 10bit CSI format and > therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > the "base address switching Change Err"). > > Do you have experience with using this driver with other cameras and can > point me in a direction that might help me? > > If you want to look at the work-in-progress driver: > https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 9:23 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 9:23 UTC (permalink / raw) To: Martin Kepplinger Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Martin, On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > hi, > > TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > cameras? I've recently worked on camera support for i.MX8MM (whose camera IP cores are, if not identical, very similar to the i.MX8MQ's). The least I can say is that it was painful :-( I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and has a CSI-2 interface. My first use case is to capture RAW10, which isn't supported by the mx6s_capture driver. > I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > Using NXP's CSI bridge driver > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > as well as the CSI driver itself: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > works fine when using the ov5640 camera with this driver: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > > (I realize there is a CSI bridge driver in staging, but that need more > work to be actually used. Of course after this the goal is to fix and > use it; and mainline a CSI phy driver too.) I have lots of patches for this driver, which I've developed on i.MX7D for a separate project. I'd like to mainline them, but this is blocked by one last issue that I haven't been able to solve yet. In a nutshell, the CSI writes two consecutive frames in each buffer, overflowing the allocated memory. The registers that control the buffer size seem to be programmed correctly as far as I can tell. I've reported this issue to NXP but haven't received any feedback yet. I've also added support for i.MX8MM to this driver, but haven't been able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best images", but seems to drop 3 out of 4 consecutive pixels. The sensor driver has been tested successfully on i.MX6 so I don't think it's the most likely cause of issues, but I can't rule out bugs on that side either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP cores but doesn't tell much about how the two are connected, so I have lots of unanswered questions about the register fields that control the interface between those IP cores. I'm pretty sure there are issues in that area of the drivers, but I've tested all combinations I could think of, without luck. > Now I use said NXP's CSI drivers on mainline without problems with the > ov5640. For the hi846 (as an example), I want to use a different > pixelformat and CSI media bus format. Describing, say, > MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > streaming won't start anymore: "not negotiated, -4" and the > "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? That will likely be the easy part of the problem. The NXP driver doesn't support RAW10 at all, so you will have to add that, and I don't think it will be easy. I'll push my development branch shortly if you're interested in trying the mainline driver. > I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > is how I test). > > When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > not yet full and "distorted" in a wrong format, which could explain why > DMA is never completed in order to create a full frame). > > Now this is details, but the hi846 camera uses a 10bit CSI format and > therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > the "base address switching Change Err"). > > Do you have experience with using this driver with other cameras and can > point me in a direction that might help me? > > If you want to look at the work-in-progress driver: > https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-10 9:23 ` Laurent Pinchart @ 2020-12-10 12:17 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 12:17 UTC (permalink / raw) To: Martin Kepplinger Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Martin, On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > > hi, > > > > TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > > cameras? > > I've recently worked on camera support for i.MX8MM (whose camera IP > cores are, if not identical, very similar to the i.MX8MQ's). The least I > can say is that it was painful :-( > > I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > has a CSI-2 interface. My first use case is to capture RAW10, which > isn't supported by the mx6s_capture driver. > > > I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > > Using NXP's CSI bridge driver > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > > as well as the CSI driver itself: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > > works fine when using the ov5640 camera with this driver: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > > > > (I realize there is a CSI bridge driver in staging, but that need more > > work to be actually used. Of course after this the goal is to fix and > > use it; and mainline a CSI phy driver too.) > > I have lots of patches for this driver, which I've developed on i.MX7D > for a separate project. I'd like to mainline them, but this is blocked > by one last issue that I haven't been able to solve yet. In a nutshell, > the CSI writes two consecutive frames in each buffer, overflowing the > allocated memory. The registers that control the buffer size seem to be > programmed correctly as far as I can tell. I've reported this issue to > NXP but haven't received any feedback yet. > > I've also added support for i.MX8MM to this driver, but haven't been > able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best > images", but seems to drop 3 out of 4 consecutive pixels. The sensor > driver has been tested successfully on i.MX6 so I don't think it's the > most likely cause of issues, but I can't rule out bugs on that side > either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP > cores but doesn't tell much about how the two are connected, so I have > lots of unanswered questions about the register fields that control the > interface between those IP cores. I'm pretty sure there are issues in > that area of the drivers, but I've tested all combinations I could think > of, without luck. > > > Now I use said NXP's CSI drivers on mainline without problems with the > > ov5640. For the hi846 (as an example), I want to use a different > > pixelformat and CSI media bus format. Describing, say, > > MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > > as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > > streaming won't start anymore: "not negotiated, -4" and the > > "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? > > That will likely be the easy part of the problem. The NXP driver doesn't > support RAW10 at all, so you will have to add that, and I don't think it > will be easy. > > I'll push my development branch shortly if you're interested in trying > the mainline driver. git://linuxtv.org/pinchartl/media.git imx/csi/imx8 The topmost patches are random debugging tests. > > I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > > video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > > is how I test). > > > > When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > > only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > > not yet full and "distorted" in a wrong format, which could explain why > > DMA is never completed in order to create a full frame). > > > > Now this is details, but the hi846 camera uses a 10bit CSI format and > > therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > > example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > > the "base address switching Change Err"). > > > > Do you have experience with using this driver with other cameras and can > > point me in a direction that might help me? > > > > If you want to look at the work-in-progress driver: > > https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 12:17 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 12:17 UTC (permalink / raw) To: Martin Kepplinger Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Martin, On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > > hi, > > > > TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > > cameras? > > I've recently worked on camera support for i.MX8MM (whose camera IP > cores are, if not identical, very similar to the i.MX8MQ's). The least I > can say is that it was painful :-( > > I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > has a CSI-2 interface. My first use case is to capture RAW10, which > isn't supported by the mx6s_capture driver. > > > I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > > Using NXP's CSI bridge driver > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > > as well as the CSI driver itself: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > > works fine when using the ov5640 camera with this driver: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > > > > (I realize there is a CSI bridge driver in staging, but that need more > > work to be actually used. Of course after this the goal is to fix and > > use it; and mainline a CSI phy driver too.) > > I have lots of patches for this driver, which I've developed on i.MX7D > for a separate project. I'd like to mainline them, but this is blocked > by one last issue that I haven't been able to solve yet. In a nutshell, > the CSI writes two consecutive frames in each buffer, overflowing the > allocated memory. The registers that control the buffer size seem to be > programmed correctly as far as I can tell. I've reported this issue to > NXP but haven't received any feedback yet. > > I've also added support for i.MX8MM to this driver, but haven't been > able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best > images", but seems to drop 3 out of 4 consecutive pixels. The sensor > driver has been tested successfully on i.MX6 so I don't think it's the > most likely cause of issues, but I can't rule out bugs on that side > either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP > cores but doesn't tell much about how the two are connected, so I have > lots of unanswered questions about the register fields that control the > interface between those IP cores. I'm pretty sure there are issues in > that area of the drivers, but I've tested all combinations I could think > of, without luck. > > > Now I use said NXP's CSI drivers on mainline without problems with the > > ov5640. For the hi846 (as an example), I want to use a different > > pixelformat and CSI media bus format. Describing, say, > > MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > > as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > > streaming won't start anymore: "not negotiated, -4" and the > > "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? > > That will likely be the easy part of the problem. The NXP driver doesn't > support RAW10 at all, so you will have to add that, and I don't think it > will be easy. > > I'll push my development branch shortly if you're interested in trying > the mainline driver. git://linuxtv.org/pinchartl/media.git imx/csi/imx8 The topmost patches are random debugging tests. > > I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > > video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > > is how I test). > > > > When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > > only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > > not yet full and "distorted" in a wrong format, which could explain why > > DMA is never completed in order to create a full frame). > > > > Now this is details, but the hi846 camera uses a 10bit CSI format and > > therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > > example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > > the "base address switching Change Err"). > > > > Do you have experience with using this driver with other cameras and can > > point me in a direction that might help me? > > > > If you want to look at the work-in-progress driver: > > https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-10 12:17 ` Laurent Pinchart @ 2020-12-10 13:12 ` Martin Kepplinger -1 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-10 13:12 UTC (permalink / raw) To: Laurent Pinchart Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team On 10.12.20 13:17, Laurent Pinchart wrote: > Hi Martin, > > On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>> hi, >>> >>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>> cameras? >> >> I've recently worked on camera support for i.MX8MM (whose camera IP >> cores are, if not identical, very similar to the i.MX8MQ's). The least I >> can say is that it was painful :-( >> >> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >> has a CSI-2 interface. My first use case is to capture RAW10, which >> isn't supported by the mx6s_capture driver. so did you successfully use the NXP mx6s_capture driver with that sensor too? >> >>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>> Using NXP's CSI bridge driver >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>> as well as the CSI driver itself: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>> works fine when using the ov5640 camera with this driver: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>> >>> (I realize there is a CSI bridge driver in staging, but that need more >>> work to be actually used. Of course after this the goal is to fix and >>> use it; and mainline a CSI phy driver too.) >> >> I have lots of patches for this driver, which I've developed on i.MX7D >> for a separate project. I'd like to mainline them, but this is blocked >> by one last issue that I haven't been able to solve yet. In a nutshell, >> the CSI writes two consecutive frames in each buffer, overflowing the >> allocated memory. The registers that control the buffer size seem to be >> programmed correctly as far as I can tell. I've reported this issue to >> NXP but haven't received any feedback yet. that's different from where I am. So you don't get any interrupt (EOF or other) when only *one* frame is written into one buffer? >> >> I've also added support for i.MX8MM to this driver, but haven't been >> able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best >> images", but seems to drop 3 out of 4 consecutive pixels. The sensor >> driver has been tested successfully on i.MX6 so I don't think it's the >> most likely cause of issues, but I can't rule out bugs on that side >> either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP >> cores but doesn't tell much about how the two are connected, so I have >> lots of unanswered questions about the register fields that control the >> interface between those IP cores. I'm pretty sure there are issues in >> that area of the drivers, but I've tested all combinations I could think >> of, without luck. >> >>> Now I use said NXP's CSI drivers on mainline without problems with the >>> ov5640. For the hi846 (as an example), I want to use a different >>> pixelformat and CSI media bus format. Describing, say, >>> MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but >>> as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 >>> streaming won't start anymore: "not negotiated, -4" and the >>> "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? >> >> That will likely be the easy part of the problem. The NXP driver doesn't >> support RAW10 at all, so you will have to add that, and I don't think it >> will be easy. the staging driver (imx7-media-csi) sets PIXEL_BIT (cr1) and BIT_MIPI_DATA_FORMAT_RAW10 (cr18) which I hoped to be enough for supporting RAW10. What else do you think is missing? >> >> I'll push my development branch shortly if you're interested in trying >> the mainline driver. > > git://linuxtv.org/pinchartl/media.git imx/csi/imx8 > > The topmost patches are random debugging tests. Thanks a lot. I'll definitely have a look and try running imx7-media-csi once more. > >>> I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! >>> video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which >>> is how I test). >>> >>> When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but >>> only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is >>> not yet full and "distorted" in a wrong format, which could explain why >>> DMA is never completed in order to create a full frame). >>> >>> Now this is details, but the hi846 camera uses a 10bit CSI format and >>> therefore I need to set PIXEL_BIT in cr1 too, but when I do that for >>> example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course >>> the "base address switching Change Err"). >>> >>> Do you have experience with using this driver with other cameras and can >>> point me in a direction that might help me? >>> >>> If you want to look at the work-in-progress driver: >>> https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 13:12 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-10 13:12 UTC (permalink / raw) To: Laurent Pinchart Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media On 10.12.20 13:17, Laurent Pinchart wrote: > Hi Martin, > > On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>> hi, >>> >>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>> cameras? >> >> I've recently worked on camera support for i.MX8MM (whose camera IP >> cores are, if not identical, very similar to the i.MX8MQ's). The least I >> can say is that it was painful :-( >> >> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >> has a CSI-2 interface. My first use case is to capture RAW10, which >> isn't supported by the mx6s_capture driver. so did you successfully use the NXP mx6s_capture driver with that sensor too? >> >>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>> Using NXP's CSI bridge driver >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>> as well as the CSI driver itself: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>> works fine when using the ov5640 camera with this driver: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>> >>> (I realize there is a CSI bridge driver in staging, but that need more >>> work to be actually used. Of course after this the goal is to fix and >>> use it; and mainline a CSI phy driver too.) >> >> I have lots of patches for this driver, which I've developed on i.MX7D >> for a separate project. I'd like to mainline them, but this is blocked >> by one last issue that I haven't been able to solve yet. In a nutshell, >> the CSI writes two consecutive frames in each buffer, overflowing the >> allocated memory. The registers that control the buffer size seem to be >> programmed correctly as far as I can tell. I've reported this issue to >> NXP but haven't received any feedback yet. that's different from where I am. So you don't get any interrupt (EOF or other) when only *one* frame is written into one buffer? >> >> I've also added support for i.MX8MM to this driver, but haven't been >> able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best >> images", but seems to drop 3 out of 4 consecutive pixels. The sensor >> driver has been tested successfully on i.MX6 so I don't think it's the >> most likely cause of issues, but I can't rule out bugs on that side >> either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP >> cores but doesn't tell much about how the two are connected, so I have >> lots of unanswered questions about the register fields that control the >> interface between those IP cores. I'm pretty sure there are issues in >> that area of the drivers, but I've tested all combinations I could think >> of, without luck. >> >>> Now I use said NXP's CSI drivers on mainline without problems with the >>> ov5640. For the hi846 (as an example), I want to use a different >>> pixelformat and CSI media bus format. Describing, say, >>> MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but >>> as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 >>> streaming won't start anymore: "not negotiated, -4" and the >>> "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? >> >> That will likely be the easy part of the problem. The NXP driver doesn't >> support RAW10 at all, so you will have to add that, and I don't think it >> will be easy. the staging driver (imx7-media-csi) sets PIXEL_BIT (cr1) and BIT_MIPI_DATA_FORMAT_RAW10 (cr18) which I hoped to be enough for supporting RAW10. What else do you think is missing? >> >> I'll push my development branch shortly if you're interested in trying >> the mainline driver. > > git://linuxtv.org/pinchartl/media.git imx/csi/imx8 > > The topmost patches are random debugging tests. Thanks a lot. I'll definitely have a look and try running imx7-media-csi once more. > >>> I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! >>> video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which >>> is how I test). >>> >>> When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but >>> only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is >>> not yet full and "distorted" in a wrong format, which could explain why >>> DMA is never completed in order to create a full frame). >>> >>> Now this is details, but the hi846 camera uses a 10bit CSI format and >>> therefore I need to set PIXEL_BIT in cr1 too, but when I do that for >>> example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course >>> the "base address switching Change Err"). >>> >>> Do you have experience with using this driver with other cameras and can >>> point me in a direction that might help me? >>> >>> If you want to look at the work-in-progress driver: >>> https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-10 13:12 ` Martin Kepplinger @ 2020-12-10 16:04 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 16:04 UTC (permalink / raw) To: Martin Kepplinger Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Martin, On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > On 10.12.20 13:17, Laurent Pinchart wrote: > > On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>> hi, > >>> > >>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > >>> cameras? > >> > >> I've recently worked on camera support for i.MX8MM (whose camera IP > >> cores are, if not identical, very similar to the i.MX8MQ's). The least I > >> can say is that it was painful :-( > >> > >> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > >> has a CSI-2 interface. My first use case is to capture RAW10, which > >> isn't supported by the mx6s_capture driver. > > so did you successfully use the NXP mx6s_capture driver with that sensor > too? I haven't tried. The mx6s_capture driver and the mainline driver share a similar architecture, as they came from the same code base. They have diverged, with the mainline driver receiving bug fixes and new features, and my large no-yet-upstreamed patch series adds lots of fixes that are required for the mt9m114 driver to work with the driver. For that reason, I've considered that porting the mt9m114 driver to the NXP mx6s_capture, and fixing the same issues in the mx6s_capture driver than were present in the mainline driver, would require lots of time for likely very little gain. That being said, I've compared the two drivers, and I haven't seen anything in mx6s_capture that would address the specific issues I'm facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera module, and I will test those with the mainline driver. If they don't work, and assuming the NXP kernel driver works on that platform, I'll have two code bases to compare. > >>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>> Using NXP's CSI bridge driver > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>> as well as the CSI driver itself: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>> works fine when using the ov5640 camera with this driver: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>> > >>> (I realize there is a CSI bridge driver in staging, but that need more > >>> work to be actually used. Of course after this the goal is to fix and > >>> use it; and mainline a CSI phy driver too.) > >> > >> I have lots of patches for this driver, which I've developed on i.MX7D > >> for a separate project. I'd like to mainline them, but this is blocked > >> by one last issue that I haven't been able to solve yet. In a nutshell, > >> the CSI writes two consecutive frames in each buffer, overflowing the > >> allocated memory. The registers that control the buffer size seem to be > >> programmed correctly as far as I can tell. I've reported this issue to > >> NXP but haven't received any feedback yet. > > that's different from where I am. So you don't get any interrupt (EOF or > other) when only *one* frame is written into one buffer? I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I haven't checked other interrupts. I thus get half of the frame rate. If I configure the CSI_CSIIMAG_PARA register with half of the actual image height, buffers then contain a single image and the frame rate goes to the expected value. That gives me a workaround, but without understanding the root cause, I'm worried that just halving the height would introduce other breakages. > >> I've also added support for i.MX8MM to this driver, but haven't been > >> able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best > >> images", but seems to drop 3 out of 4 consecutive pixels. The sensor > >> driver has been tested successfully on i.MX6 so I don't think it's the > >> most likely cause of issues, but I can't rule out bugs on that side > >> either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP > >> cores but doesn't tell much about how the two are connected, so I have > >> lots of unanswered questions about the register fields that control the > >> interface between those IP cores. I'm pretty sure there are issues in > >> that area of the drivers, but I've tested all combinations I could think > >> of, without luck. > >> > >>> Now I use said NXP's CSI drivers on mainline without problems with the > >>> ov5640. For the hi846 (as an example), I want to use a different > >>> pixelformat and CSI media bus format. Describing, say, > >>> MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > >>> as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > >>> streaming won't start anymore: "not negotiated, -4" and the > >>> "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? > >> > >> That will likely be the easy part of the problem. The NXP driver doesn't > >> support RAW10 at all, so you will have to add that, and I don't think it > >> will be easy. > > the staging driver (imx7-media-csi) sets PIXEL_BIT (cr1) and > BIT_MIPI_DATA_FORMAT_RAW10 (cr18) which I hoped to be enough for > supporting RAW10. What else do you think is missing? That I don't know, given that I haven't been able to get RAW10 to work correctly with the staging driver yet :-) > >> I'll push my development branch shortly if you're interested in trying > >> the mainline driver. > > > > git://linuxtv.org/pinchartl/media.git imx/csi/imx8 > > > > The topmost patches are random debugging tests. > > Thanks a lot. I'll definitely have a look and try running imx7-media-csi > once more. > > >>> I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > >>> video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > >>> is how I test). > >>> > >>> When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > >>> only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > >>> not yet full and "distorted" in a wrong format, which could explain why > >>> DMA is never completed in order to create a full frame). > >>> > >>> Now this is details, but the hi846 camera uses a 10bit CSI format and > >>> therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > >>> example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > >>> the "base address switching Change Err"). > >>> > >>> Do you have experience with using this driver with other cameras and can > >>> point me in a direction that might help me? > >>> > >>> If you want to look at the work-in-progress driver: > >>> https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-10 16:04 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-10 16:04 UTC (permalink / raw) To: Martin Kepplinger Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Martin, On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > On 10.12.20 13:17, Laurent Pinchart wrote: > > On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>> hi, > >>> > >>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > >>> cameras? > >> > >> I've recently worked on camera support for i.MX8MM (whose camera IP > >> cores are, if not identical, very similar to the i.MX8MQ's). The least I > >> can say is that it was painful :-( > >> > >> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > >> has a CSI-2 interface. My first use case is to capture RAW10, which > >> isn't supported by the mx6s_capture driver. > > so did you successfully use the NXP mx6s_capture driver with that sensor > too? I haven't tried. The mx6s_capture driver and the mainline driver share a similar architecture, as they came from the same code base. They have diverged, with the mainline driver receiving bug fixes and new features, and my large no-yet-upstreamed patch series adds lots of fixes that are required for the mt9m114 driver to work with the driver. For that reason, I've considered that porting the mt9m114 driver to the NXP mx6s_capture, and fixing the same issues in the mx6s_capture driver than were present in the mainline driver, would require lots of time for likely very little gain. That being said, I've compared the two drivers, and I haven't seen anything in mx6s_capture that would address the specific issues I'm facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera module, and I will test those with the mainline driver. If they don't work, and assuming the NXP kernel driver works on that platform, I'll have two code bases to compare. > >>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>> Using NXP's CSI bridge driver > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>> as well as the CSI driver itself: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>> works fine when using the ov5640 camera with this driver: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>> > >>> (I realize there is a CSI bridge driver in staging, but that need more > >>> work to be actually used. Of course after this the goal is to fix and > >>> use it; and mainline a CSI phy driver too.) > >> > >> I have lots of patches for this driver, which I've developed on i.MX7D > >> for a separate project. I'd like to mainline them, but this is blocked > >> by one last issue that I haven't been able to solve yet. In a nutshell, > >> the CSI writes two consecutive frames in each buffer, overflowing the > >> allocated memory. The registers that control the buffer size seem to be > >> programmed correctly as far as I can tell. I've reported this issue to > >> NXP but haven't received any feedback yet. > > that's different from where I am. So you don't get any interrupt (EOF or > other) when only *one* frame is written into one buffer? I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I haven't checked other interrupts. I thus get half of the frame rate. If I configure the CSI_CSIIMAG_PARA register with half of the actual image height, buffers then contain a single image and the frame rate goes to the expected value. That gives me a workaround, but without understanding the root cause, I'm worried that just halving the height would introduce other breakages. > >> I've also added support for i.MX8MM to this driver, but haven't been > >> able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best > >> images", but seems to drop 3 out of 4 consecutive pixels. The sensor > >> driver has been tested successfully on i.MX6 so I don't think it's the > >> most likely cause of issues, but I can't rule out bugs on that side > >> either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP > >> cores but doesn't tell much about how the two are connected, so I have > >> lots of unanswered questions about the register fields that control the > >> interface between those IP cores. I'm pretty sure there are issues in > >> that area of the drivers, but I've tested all combinations I could think > >> of, without luck. > >> > >>> Now I use said NXP's CSI drivers on mainline without problems with the > >>> ov5640. For the hi846 (as an example), I want to use a different > >>> pixelformat and CSI media bus format. Describing, say, > >>> MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > >>> as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > >>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > >>> streaming won't start anymore: "not negotiated, -4" and the > >>> "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? > >> > >> That will likely be the easy part of the problem. The NXP driver doesn't > >> support RAW10 at all, so you will have to add that, and I don't think it > >> will be easy. > > the staging driver (imx7-media-csi) sets PIXEL_BIT (cr1) and > BIT_MIPI_DATA_FORMAT_RAW10 (cr18) which I hoped to be enough for > supporting RAW10. What else do you think is missing? That I don't know, given that I haven't been able to get RAW10 to work correctly with the staging driver yet :-) > >> I'll push my development branch shortly if you're interested in trying > >> the mainline driver. > > > > git://linuxtv.org/pinchartl/media.git imx/csi/imx8 > > > > The topmost patches are random debugging tests. > > Thanks a lot. I'll definitely have a look and try running imx7-media-csi > once more. > > >>> I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > >>> video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > >>> is how I test). > >>> > >>> When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > >>> only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > >>> not yet full and "distorted" in a wrong format, which could explain why > >>> DMA is never completed in order to create a full frame). > >>> > >>> Now this is details, but the hi846 camera uses a 10bit CSI format and > >>> therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > >>> example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > >>> the "base address switching Change Err"). > >>> > >>> Do you have experience with using this driver with other cameras and can > >>> point me in a direction that might help me? > >>> > >>> If you want to look at the work-in-progress driver: > >>> https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-10 16:04 ` Laurent Pinchart @ 2020-12-22 14:06 ` Martin Kepplinger -1 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-22 14:06 UTC (permalink / raw) To: Laurent Pinchart Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team On 10.12.20 17:04, Laurent Pinchart wrote: > Hi Martin, > > On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >> On 10.12.20 13:17, Laurent Pinchart wrote: >>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>> hi, >>>>> >>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>>>> cameras? >>>> >>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I >>>> can say is that it was painful :-( >>>> >>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>> isn't supported by the mx6s_capture driver. >> >> so did you successfully use the NXP mx6s_capture driver with that sensor >> too? > > I haven't tried. The mx6s_capture driver and the mainline driver share a > similar architecture, as they came from the same code base. They have > diverged, with the mainline driver receiving bug fixes and new > features, and my large no-yet-upstreamed patch series adds lots of fixes > that are required for the mt9m114 driver to work with the driver. For > that reason, I've considered that porting the mt9m114 driver to the NXP > mx6s_capture, and fixing the same issues in the mx6s_capture driver than > were present in the mainline driver, would require lots of time for > likely very little gain. > > That being said, I've compared the two drivers, and I haven't seen > anything in mx6s_capture that would address the specific issues I'm > facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > module, and I will test those with the mainline driver. If they don't > work, and assuming the NXP kernel driver works on that platform, I'll > have two code bases to compare. > >>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>>>> Using NXP's CSI bridge driver >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>> as well as the CSI driver itself: >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>> works fine when using the ov5640 camera with this driver: >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>> >>>>> (I realize there is a CSI bridge driver in staging, but that need more >>>>> work to be actually used. Of course after this the goal is to fix and >>>>> use it; and mainline a CSI phy driver too.) >>>> >>>> I have lots of patches for this driver, which I've developed on i.MX7D >>>> for a separate project. I'd like to mainline them, but this is blocked >>>> by one last issue that I haven't been able to solve yet. In a nutshell, >>>> the CSI writes two consecutive frames in each buffer, overflowing the >>>> allocated memory. The registers that control the buffer size seem to be >>>> programmed correctly as far as I can tell. I've reported this issue to >>>> NXP but haven't received any feedback yet. >> >> that's different from where I am. So you don't get any interrupt (EOF or >> other) when only *one* frame is written into one buffer? > > I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > haven't checked other interrupts. I thus get half of the frame rate. If > I configure the CSI_CSIIMAG_PARA register with half of the actual image > height, buffers then contain a single image and the frame rate goes to > the expected value. That gives me a workaround, but without > understanding the root cause, I'm worried that just halving the height > would introduce other breakages. > Hi Laurent, Thanks a lot, exchanging things this way alone is much appreciated. I'll give you detailed feedback on the staging drivers later - I tried running them, they probe but still a (gstreamer) stream would not start. But most all of your addition make a lot of sense. Just so you know, to emphasize that once more: I'm still on the nxp drivers (yes, they are very similar) just because the ov56 cam works with them over here. The main problem I have is that, as soon as I try to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically nothing works anymore. userspace stops to "be interested" pretty soon. If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) completed, the usual way. And when I break the ov56 by using a wrong pixelformat in mx6s_capture (nxp bridge driver), I get equally get only one frame (dma) completed there. How can I make this thing use a different pixelformat (by setting the media bus format in the sensor driver)? thanks again for sharing your code early. have a nice rest of the year, martin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-22 14:06 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-22 14:06 UTC (permalink / raw) To: Laurent Pinchart Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media On 10.12.20 17:04, Laurent Pinchart wrote: > Hi Martin, > > On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >> On 10.12.20 13:17, Laurent Pinchart wrote: >>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>> hi, >>>>> >>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>>>> cameras? >>>> >>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I >>>> can say is that it was painful :-( >>>> >>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>> isn't supported by the mx6s_capture driver. >> >> so did you successfully use the NXP mx6s_capture driver with that sensor >> too? > > I haven't tried. The mx6s_capture driver and the mainline driver share a > similar architecture, as they came from the same code base. They have > diverged, with the mainline driver receiving bug fixes and new > features, and my large no-yet-upstreamed patch series adds lots of fixes > that are required for the mt9m114 driver to work with the driver. For > that reason, I've considered that porting the mt9m114 driver to the NXP > mx6s_capture, and fixing the same issues in the mx6s_capture driver than > were present in the mainline driver, would require lots of time for > likely very little gain. > > That being said, I've compared the two drivers, and I haven't seen > anything in mx6s_capture that would address the specific issues I'm > facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > module, and I will test those with the mainline driver. If they don't > work, and assuming the NXP kernel driver works on that platform, I'll > have two code bases to compare. > >>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>>>> Using NXP's CSI bridge driver >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>> as well as the CSI driver itself: >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>> works fine when using the ov5640 camera with this driver: >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>> >>>>> (I realize there is a CSI bridge driver in staging, but that need more >>>>> work to be actually used. Of course after this the goal is to fix and >>>>> use it; and mainline a CSI phy driver too.) >>>> >>>> I have lots of patches for this driver, which I've developed on i.MX7D >>>> for a separate project. I'd like to mainline them, but this is blocked >>>> by one last issue that I haven't been able to solve yet. In a nutshell, >>>> the CSI writes two consecutive frames in each buffer, overflowing the >>>> allocated memory. The registers that control the buffer size seem to be >>>> programmed correctly as far as I can tell. I've reported this issue to >>>> NXP but haven't received any feedback yet. >> >> that's different from where I am. So you don't get any interrupt (EOF or >> other) when only *one* frame is written into one buffer? > > I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > haven't checked other interrupts. I thus get half of the frame rate. If > I configure the CSI_CSIIMAG_PARA register with half of the actual image > height, buffers then contain a single image and the frame rate goes to > the expected value. That gives me a workaround, but without > understanding the root cause, I'm worried that just halving the height > would introduce other breakages. > Hi Laurent, Thanks a lot, exchanging things this way alone is much appreciated. I'll give you detailed feedback on the staging drivers later - I tried running them, they probe but still a (gstreamer) stream would not start. But most all of your addition make a lot of sense. Just so you know, to emphasize that once more: I'm still on the nxp drivers (yes, they are very similar) just because the ov56 cam works with them over here. The main problem I have is that, as soon as I try to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically nothing works anymore. userspace stops to "be interested" pretty soon. If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) completed, the usual way. And when I break the ov56 by using a wrong pixelformat in mx6s_capture (nxp bridge driver), I get equally get only one frame (dma) completed there. How can I make this thing use a different pixelformat (by setting the media bus format in the sensor driver)? thanks again for sharing your code early. have a nice rest of the year, martin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-22 14:06 ` Martin Kepplinger @ 2020-12-22 15:35 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-22 15:35 UTC (permalink / raw) To: Martin Kepplinger Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Martin, On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: > On 10.12.20 17:04, Laurent Pinchart wrote: > > On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > >> On 10.12.20 13:17, Laurent Pinchart wrote: > >>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>>>> hi, > >>>>> > >>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > >>>>> cameras? > >>>> > >>>> I've recently worked on camera support for i.MX8MM (whose camera IP > >>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I > >>>> can say is that it was painful :-( > >>>> > >>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > >>>> has a CSI-2 interface. My first use case is to capture RAW10, which > >>>> isn't supported by the mx6s_capture driver. > >> > >> so did you successfully use the NXP mx6s_capture driver with that sensor > >> too? > > > > I haven't tried. The mx6s_capture driver and the mainline driver share a > > similar architecture, as they came from the same code base. They have > > diverged, with the mainline driver receiving bug fixes and new > > features, and my large no-yet-upstreamed patch series adds lots of fixes > > that are required for the mt9m114 driver to work with the driver. For > > that reason, I've considered that porting the mt9m114 driver to the NXP > > mx6s_capture, and fixing the same issues in the mx6s_capture driver than > > were present in the mainline driver, would require lots of time for > > likely very little gain. > > > > That being said, I've compared the two drivers, and I haven't seen > > anything in mx6s_capture that would address the specific issues I'm > > facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > > module, and I will test those with the mainline driver. If they don't > > work, and assuming the NXP kernel driver works on that platform, I'll > > have two code bases to compare. > > > >>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>>>> Using NXP's CSI bridge driver > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>>>> as well as the CSI driver itself: > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>>>> works fine when using the ov5640 camera with this driver: > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>>>> > >>>>> (I realize there is a CSI bridge driver in staging, but that need more > >>>>> work to be actually used. Of course after this the goal is to fix and > >>>>> use it; and mainline a CSI phy driver too.) > >>>> > >>>> I have lots of patches for this driver, which I've developed on i.MX7D > >>>> for a separate project. I'd like to mainline them, but this is blocked > >>>> by one last issue that I haven't been able to solve yet. In a nutshell, > >>>> the CSI writes two consecutive frames in each buffer, overflowing the > >>>> allocated memory. The registers that control the buffer size seem to be > >>>> programmed correctly as far as I can tell. I've reported this issue to > >>>> NXP but haven't received any feedback yet. > >> > >> that's different from where I am. So you don't get any interrupt (EOF or > >> other) when only *one* frame is written into one buffer? > > > > I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > > haven't checked other interrupts. I thus get half of the frame rate. If > > I configure the CSI_CSIIMAG_PARA register with half of the actual image > > height, buffers then contain a single image and the frame rate goes to > > the expected value. That gives me a workaround, but without > > understanding the root cause, I'm worried that just halving the height > > would introduce other breakages. > > Hi Laurent, > > Thanks a lot, exchanging things this way alone is much appreciated. I'll > give you detailed feedback on the staging drivers later - I tried > running them, they probe but still a (gstreamer) stream would not start. > But most all of your addition make a lot of sense. > > Just so you know, to emphasize that once more: I'm still on the nxp > drivers (yes, they are very similar) just because the ov56 cam works > with them over here. I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and when I try to capture images, with the NXP kernel, I just don't get any. *sigh* I'll have to debug that. > The main problem I have is that, as soon as I try > to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically > nothing works anymore. userspace stops to "be interested" pretty soon. > > If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) > completed, the usual way. > > And when I break the ov56 by using a wrong pixelformat in mx6s_capture > (nxp bridge driver), I get equally get only one frame (dma) completed there. > > How can I make this thing use a different pixelformat (by setting the > media bus format in the sensor driver)? All the components have to support the format you want to capture. This includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The pixel format only matters in the CSI driver, for the other two it's the media bus format that matters. To capture raw bayer data, you'll first need to extend the OV5640 driver to support the corresponding media bus format (and of course configure the sensor to output that format), then the MIPI_CSI driver to support that media bus format as well, and finally the CSI driver. > thanks again for sharing your code early. have a nice rest of the year, -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-22 15:35 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-22 15:35 UTC (permalink / raw) To: Martin Kepplinger Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Martin, On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: > On 10.12.20 17:04, Laurent Pinchart wrote: > > On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > >> On 10.12.20 13:17, Laurent Pinchart wrote: > >>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>>>> hi, > >>>>> > >>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > >>>>> cameras? > >>>> > >>>> I've recently worked on camera support for i.MX8MM (whose camera IP > >>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I > >>>> can say is that it was painful :-( > >>>> > >>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and > >>>> has a CSI-2 interface. My first use case is to capture RAW10, which > >>>> isn't supported by the mx6s_capture driver. > >> > >> so did you successfully use the NXP mx6s_capture driver with that sensor > >> too? > > > > I haven't tried. The mx6s_capture driver and the mainline driver share a > > similar architecture, as they came from the same code base. They have > > diverged, with the mainline driver receiving bug fixes and new > > features, and my large no-yet-upstreamed patch series adds lots of fixes > > that are required for the mt9m114 driver to work with the driver. For > > that reason, I've considered that porting the mt9m114 driver to the NXP > > mx6s_capture, and fixing the same issues in the mx6s_capture driver than > > were present in the mainline driver, would require lots of time for > > likely very little gain. > > > > That being said, I've compared the two drivers, and I haven't seen > > anything in mx6s_capture that would address the specific issues I'm > > facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > > module, and I will test those with the mainline driver. If they don't > > work, and assuming the NXP kernel driver works on that platform, I'll > > have two code bases to compare. > > > >>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>>>> Using NXP's CSI bridge driver > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>>>> as well as the CSI driver itself: > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>>>> works fine when using the ov5640 camera with this driver: > >>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>>>> > >>>>> (I realize there is a CSI bridge driver in staging, but that need more > >>>>> work to be actually used. Of course after this the goal is to fix and > >>>>> use it; and mainline a CSI phy driver too.) > >>>> > >>>> I have lots of patches for this driver, which I've developed on i.MX7D > >>>> for a separate project. I'd like to mainline them, but this is blocked > >>>> by one last issue that I haven't been able to solve yet. In a nutshell, > >>>> the CSI writes two consecutive frames in each buffer, overflowing the > >>>> allocated memory. The registers that control the buffer size seem to be > >>>> programmed correctly as far as I can tell. I've reported this issue to > >>>> NXP but haven't received any feedback yet. > >> > >> that's different from where I am. So you don't get any interrupt (EOF or > >> other) when only *one* frame is written into one buffer? > > > > I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > > haven't checked other interrupts. I thus get half of the frame rate. If > > I configure the CSI_CSIIMAG_PARA register with half of the actual image > > height, buffers then contain a single image and the frame rate goes to > > the expected value. That gives me a workaround, but without > > understanding the root cause, I'm worried that just halving the height > > would introduce other breakages. > > Hi Laurent, > > Thanks a lot, exchanging things this way alone is much appreciated. I'll > give you detailed feedback on the staging drivers later - I tried > running them, they probe but still a (gstreamer) stream would not start. > But most all of your addition make a lot of sense. > > Just so you know, to emphasize that once more: I'm still on the nxp > drivers (yes, they are very similar) just because the ov56 cam works > with them over here. I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and when I try to capture images, with the NXP kernel, I just don't get any. *sigh* I'll have to debug that. > The main problem I have is that, as soon as I try > to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically > nothing works anymore. userspace stops to "be interested" pretty soon. > > If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) > completed, the usual way. > > And when I break the ov56 by using a wrong pixelformat in mx6s_capture > (nxp bridge driver), I get equally get only one frame (dma) completed there. > > How can I make this thing use a different pixelformat (by setting the > media bus format in the sensor driver)? All the components have to support the format you want to capture. This includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The pixel format only matters in the CSI driver, for the other two it's the media bus format that matters. To capture raw bayer data, you'll first need to extend the OV5640 driver to support the corresponding media bus format (and of course configure the sensor to output that format), then the MIPI_CSI driver to support that media bus format as well, and finally the CSI driver. > thanks again for sharing your code early. have a nice rest of the year, -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-22 15:35 ` Laurent Pinchart @ 2020-12-22 15:55 ` Martin Kepplinger -1 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-22 15:55 UTC (permalink / raw) To: Laurent Pinchart Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team On 22.12.20 16:35, Laurent Pinchart wrote: > Hi Martin, > > On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: >> On 10.12.20 17:04, Laurent Pinchart wrote: >>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >>>> On 10.12.20 13:17, Laurent Pinchart wrote: >>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>>>> hi, >>>>>>> >>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>>>>>> cameras? >>>>>> >>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I >>>>>> can say is that it was painful :-( >>>>>> >>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>>>> isn't supported by the mx6s_capture driver. >>>> >>>> so did you successfully use the NXP mx6s_capture driver with that sensor >>>> too? >>> >>> I haven't tried. The mx6s_capture driver and the mainline driver share a >>> similar architecture, as they came from the same code base. They have >>> diverged, with the mainline driver receiving bug fixes and new >>> features, and my large no-yet-upstreamed patch series adds lots of fixes >>> that are required for the mt9m114 driver to work with the driver. For >>> that reason, I've considered that porting the mt9m114 driver to the NXP >>> mx6s_capture, and fixing the same issues in the mx6s_capture driver than >>> were present in the mainline driver, would require lots of time for >>> likely very little gain. >>> >>> That being said, I've compared the two drivers, and I haven't seen >>> anything in mx6s_capture that would address the specific issues I'm >>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera >>> module, and I will test those with the mainline driver. If they don't >>> work, and assuming the NXP kernel driver works on that platform, I'll >>> have two code bases to compare. >>> >>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>>>>>> Using NXP's CSI bridge driver >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> as well as the CSI driver itself: >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> works fine when using the ov5640 camera with this driver: >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> >>>>>>> (I realize there is a CSI bridge driver in staging, but that need more >>>>>>> work to be actually used. Of course after this the goal is to fix and >>>>>>> use it; and mainline a CSI phy driver too.) >>>>>> >>>>>> I have lots of patches for this driver, which I've developed on i.MX7D >>>>>> for a separate project. I'd like to mainline them, but this is blocked >>>>>> by one last issue that I haven't been able to solve yet. In a nutshell, >>>>>> the CSI writes two consecutive frames in each buffer, overflowing the >>>>>> allocated memory. The registers that control the buffer size seem to be >>>>>> programmed correctly as far as I can tell. I've reported this issue to >>>>>> NXP but haven't received any feedback yet. >>>> >>>> that's different from where I am. So you don't get any interrupt (EOF or >>>> other) when only *one* frame is written into one buffer? >>> >>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I >>> haven't checked other interrupts. I thus get half of the frame rate. If >>> I configure the CSI_CSIIMAG_PARA register with half of the actual image >>> height, buffers then contain a single image and the frame rate goes to >>> the expected value. That gives me a workaround, but without >>> understanding the root cause, I'm worried that just halving the height >>> would introduce other breakages. >> >> Hi Laurent, >> >> Thanks a lot, exchanging things this way alone is much appreciated. I'll >> give you detailed feedback on the staging drivers later - I tried >> running them, they probe but still a (gstreamer) stream would not start. >> But most all of your addition make a lot of sense. >> >> Just so you know, to emphasize that once more: I'm still on the nxp >> drivers (yes, they are very similar) just because the ov56 cam works >> with them over here. > > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > when I try to capture images, with the NXP kernel, I just don't get any. > *sigh* I'll have to debug that. oh you have it already. very good. I guess the nxp kernel directly *should* work, but I use a kernel based on v5.9.X and include the drivers like so: https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > >> The main problem I have is that, as soon as I try >> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically >> nothing works anymore. userspace stops to "be interested" pretty soon. >> >> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) >> completed, the usual way. >> >> And when I break the ov56 by using a wrong pixelformat in mx6s_capture >> (nxp bridge driver), I get equally get only one frame (dma) completed there. >> >> How can I make this thing use a different pixelformat (by setting the >> media bus format in the sensor driver)? > > All the components have to support the format you want to capture. This > includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The > pixel format only matters in the CSI driver, for the other two it's the > media bus format that matters. To capture raw bayer data, you'll first > need to extend the OV5640 driver to support the corresponding media bus > format (and of course configure the sensor to output that format), then > the MIPI_CSI driver to support that media bus format as well, and > finally the CSI driver. ok, the sensor I work with outputs raw bayer only, and that sounds like I should pay more attention to the mipi csi driver. that links to my experiments I occasionally update: https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > >> thanks again for sharing your code early. have a nice rest of the year, > thanks, martin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-22 15:55 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2020-12-22 15:55 UTC (permalink / raw) To: Laurent Pinchart Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media On 22.12.20 16:35, Laurent Pinchart wrote: > Hi Martin, > > On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: >> On 10.12.20 17:04, Laurent Pinchart wrote: >>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >>>> On 10.12.20 13:17, Laurent Pinchart wrote: >>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>>>> hi, >>>>>>> >>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other >>>>>>> cameras? >>>>>> >>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The least I >>>>>> can say is that it was painful :-( >>>>>> >>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and >>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>>>> isn't supported by the mx6s_capture driver. >>>> >>>> so did you successfully use the NXP mx6s_capture driver with that sensor >>>> too? >>> >>> I haven't tried. The mx6s_capture driver and the mainline driver share a >>> similar architecture, as they came from the same code base. They have >>> diverged, with the mainline driver receiving bug fixes and new >>> features, and my large no-yet-upstreamed patch series adds lots of fixes >>> that are required for the mt9m114 driver to work with the driver. For >>> that reason, I've considered that porting the mt9m114 driver to the NXP >>> mx6s_capture, and fixing the same issues in the mx6s_capture driver than >>> were present in the mainline driver, would require lots of time for >>> likely very little gain. >>> >>> That being said, I've compared the two drivers, and I haven't seen >>> anything in mx6s_capture that would address the specific issues I'm >>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera >>> module, and I will test those with the mainline driver. If they don't >>> work, and assuming the NXP kernel driver works on that platform, I'll >>> have two code bases to compare. >>> >>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: >>>>>>> Using NXP's CSI bridge driver >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> as well as the CSI driver itself: >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> works fine when using the ov5640 camera with this driver: >>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>>>> >>>>>>> (I realize there is a CSI bridge driver in staging, but that need more >>>>>>> work to be actually used. Of course after this the goal is to fix and >>>>>>> use it; and mainline a CSI phy driver too.) >>>>>> >>>>>> I have lots of patches for this driver, which I've developed on i.MX7D >>>>>> for a separate project. I'd like to mainline them, but this is blocked >>>>>> by one last issue that I haven't been able to solve yet. In a nutshell, >>>>>> the CSI writes two consecutive frames in each buffer, overflowing the >>>>>> allocated memory. The registers that control the buffer size seem to be >>>>>> programmed correctly as far as I can tell. I've reported this issue to >>>>>> NXP but haven't received any feedback yet. >>>> >>>> that's different from where I am. So you don't get any interrupt (EOF or >>>> other) when only *one* frame is written into one buffer? >>> >>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I >>> haven't checked other interrupts. I thus get half of the frame rate. If >>> I configure the CSI_CSIIMAG_PARA register with half of the actual image >>> height, buffers then contain a single image and the frame rate goes to >>> the expected value. That gives me a workaround, but without >>> understanding the root cause, I'm worried that just halving the height >>> would introduce other breakages. >> >> Hi Laurent, >> >> Thanks a lot, exchanging things this way alone is much appreciated. I'll >> give you detailed feedback on the staging drivers later - I tried >> running them, they probe but still a (gstreamer) stream would not start. >> But most all of your addition make a lot of sense. >> >> Just so you know, to emphasize that once more: I'm still on the nxp >> drivers (yes, they are very similar) just because the ov56 cam works >> with them over here. > > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > when I try to capture images, with the NXP kernel, I just don't get any. > *sigh* I'll have to debug that. oh you have it already. very good. I guess the nxp kernel directly *should* work, but I use a kernel based on v5.9.X and include the drivers like so: https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > >> The main problem I have is that, as soon as I try >> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically >> nothing works anymore. userspace stops to "be interested" pretty soon. >> >> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) >> completed, the usual way. >> >> And when I break the ov56 by using a wrong pixelformat in mx6s_capture >> (nxp bridge driver), I get equally get only one frame (dma) completed there. >> >> How can I make this thing use a different pixelformat (by setting the >> media bus format in the sensor driver)? > > All the components have to support the format you want to capture. This > includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The > pixel format only matters in the CSI driver, for the other two it's the > media bus format that matters. To capture raw bayer data, you'll first > need to extend the OV5640 driver to support the corresponding media bus > format (and of course configure the sensor to output that format), then > the MIPI_CSI driver to support that media bus format as well, and > finally the CSI driver. ok, the sensor I work with outputs raw bayer only, and that sounds like I should pay more attention to the mipi csi driver. that links to my experiments I occasionally update: https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > >> thanks again for sharing your code early. have a nice rest of the year, > thanks, martin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-22 15:55 ` Martin Kepplinger @ 2021-02-23 9:04 ` Martin Kepplinger -1 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2021-02-23 9:04 UTC (permalink / raw) To: Laurent Pinchart Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team On 22.12.20 16:55, Martin Kepplinger wrote: > On 22.12.20 16:35, Laurent Pinchart wrote: >> Hi Martin, >> >> On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: >>> On 10.12.20 17:04, Laurent Pinchart wrote: >>>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >>>>> On 10.12.20 13:17, Laurent Pinchart wrote: >>>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>>>>> hi, >>>>>>>> >>>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with >>>>>>>> other >>>>>>>> cameras? >>>>>>> >>>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The >>>>>>> least I >>>>>>> can say is that it was painful :-( >>>>>>> >>>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and >>>>>>> YUV and >>>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>>>>> isn't supported by the mx6s_capture driver. >>>>> >>>>> so did you successfully use the NXP mx6s_capture driver with that >>>>> sensor >>>>> too? >>>> >>>> I haven't tried. The mx6s_capture driver and the mainline driver >>>> share a >>>> similar architecture, as they came from the same code base. They have >>>> diverged, with the mainline driver receiving bug fixes and new >>>> features, and my large no-yet-upstreamed patch series adds lots of >>>> fixes >>>> that are required for the mt9m114 driver to work with the driver. For >>>> that reason, I've considered that porting the mt9m114 driver to the NXP >>>> mx6s_capture, and fixing the same issues in the mx6s_capture driver >>>> than >>>> were present in the mainline driver, would require lots of time for >>>> likely very little gain. >>>> >>>> That being said, I've compared the two drivers, and I haven't seen >>>> anything in mx6s_capture that would address the specific issues I'm >>>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera >>>> module, and I will test those with the mainline driver. If they don't >>>> work, and assuming the NXP kernel driver works on that platform, I'll >>>> have two code bases to compare. >>>> >>>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on >>>>>>>> imx8mq: >>>>>>>> Using NXP's CSI bridge driver >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> as well as the CSI driver itself: >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> works fine when using the ov5640 camera with this driver: >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> >>>>>>>> (I realize there is a CSI bridge driver in staging, but that >>>>>>>> need more >>>>>>>> work to be actually used. Of course after this the goal is to >>>>>>>> fix and >>>>>>>> use it; and mainline a CSI phy driver too.) >>>>>>> >>>>>>> I have lots of patches for this driver, which I've developed on >>>>>>> i.MX7D >>>>>>> for a separate project. I'd like to mainline them, but this is >>>>>>> blocked >>>>>>> by one last issue that I haven't been able to solve yet. In a >>>>>>> nutshell, >>>>>>> the CSI writes two consecutive frames in each buffer, overflowing >>>>>>> the >>>>>>> allocated memory. The registers that control the buffer size seem >>>>>>> to be >>>>>>> programmed correctly as far as I can tell. I've reported this >>>>>>> issue to >>>>>>> NXP but haven't received any feedback yet. >>>>> >>>>> that's different from where I am. So you don't get any interrupt >>>>> (EOF or >>>>> other) when only *one* frame is written into one buffer? >>>> >>>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I >>>> haven't checked other interrupts. I thus get half of the frame rate. If >>>> I configure the CSI_CSIIMAG_PARA register with half of the actual image >>>> height, buffers then contain a single image and the frame rate goes to >>>> the expected value. That gives me a workaround, but without >>>> understanding the root cause, I'm worried that just halving the height >>>> would introduce other breakages. >>> >>> Hi Laurent, >>> >>> Thanks a lot, exchanging things this way alone is much appreciated. I'll >>> give you detailed feedback on the staging drivers later - I tried >>> running them, they probe but still a (gstreamer) stream would not start. >>> But most all of your addition make a lot of sense. >>> >>> Just so you know, to emphasize that once more: I'm still on the nxp >>> drivers (yes, they are very similar) just because the ov56 cam works >>> with them over here. >> >> I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and >> when I try to capture images, with the NXP kernel, I just don't get any. >> *sigh* I'll have to debug that. > > oh you have it already. very good. > > I guess the nxp kernel directly *should* work, but I use a kernel based > on v5.9.X and include the drivers like so: > https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 > > https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > > >> >>> The main problem I have is that, as soon as I try >>> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically >>> nothing works anymore. userspace stops to "be interested" pretty soon. >>> >>> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) >>> completed, the usual way. >>> >>> And when I break the ov56 by using a wrong pixelformat in mx6s_capture >>> (nxp bridge driver), I get equally get only one frame (dma) completed >>> there. >>> >>> How can I make this thing use a different pixelformat (by setting the >>> media bus format in the sensor driver)? >> >> All the components have to support the format you want to capture. This >> includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The >> pixel format only matters in the CSI driver, for the other two it's the >> media bus format that matters. To capture raw bayer data, you'll first >> need to extend the OV5640 driver to support the corresponding media bus >> format (and of course configure the sensor to output that format), then >> the MIPI_CSI driver to support that media bus format as well, and >> finally the CSI driver. > > ok, the sensor I work with outputs raw bayer only, and that sounds like > I should pay more attention to the mipi csi driver. > > that links to my experiments I occasionally update: > https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > >> >>> thanks again for sharing your code early. have a nice rest of the year, >> > > thanks, > martin hi Laurent, how are you? Am I right in that you tried to run the ov5640 mainline driver with the CSI bridge staging driver on imx8mq? Which mipi driver would you then use? Did you have progress in that area and in resolving things in order to push some of your changes out? I'm still struggling with the different sensor I want to use on imx8mq (hi846) and I still use the nxp bsp drivers for mipi and csi bridge: I was able to add and fix a few things and get some kind of image at least, but there's still definitely some sizes or timings wrong (in every resolution mode I think). 1280x720 shows it clearly as the image looks "almost" normal only when I resize it to 916 pixel width. Also, it "shifts" with later frames. For a few images of this, see https://source.puri.sm/Librem5/linux-next/-/issues/43#note_143875 I just wanted to describe the above just in case you've seen something like that during your experiments with sensors and can point me to places to look at. I do think that the SoC side should be ok now (but still now 100% sure), but many sensors are similar and maybe you have ideas there too. anyways, thanks for looking at this and as soon as I have a satisfying image with my current setup, I'll switch to the staging csi driver. But there's no driver for the mipi ("phy") in staging for imx8mq, right? thanks! martin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2021-02-23 9:04 ` Martin Kepplinger 0 siblings, 0 replies; 24+ messages in thread From: Martin Kepplinger @ 2021-02-23 9:04 UTC (permalink / raw) To: Laurent Pinchart Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media On 22.12.20 16:55, Martin Kepplinger wrote: > On 22.12.20 16:35, Laurent Pinchart wrote: >> Hi Martin, >> >> On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: >>> On 10.12.20 17:04, Laurent Pinchart wrote: >>>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: >>>>> On 10.12.20 13:17, Laurent Pinchart wrote: >>>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: >>>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: >>>>>>>> hi, >>>>>>>> >>>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with >>>>>>>> other >>>>>>>> cameras? >>>>>>> >>>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP >>>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The >>>>>>> least I >>>>>>> can say is that it was painful :-( >>>>>>> >>>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and >>>>>>> YUV and >>>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which >>>>>>> isn't supported by the mx6s_capture driver. >>>>> >>>>> so did you successfully use the NXP mx6s_capture driver with that >>>>> sensor >>>>> too? >>>> >>>> I haven't tried. The mx6s_capture driver and the mainline driver >>>> share a >>>> similar architecture, as they came from the same code base. They have >>>> diverged, with the mainline driver receiving bug fixes and new >>>> features, and my large no-yet-upstreamed patch series adds lots of >>>> fixes >>>> that are required for the mt9m114 driver to work with the driver. For >>>> that reason, I've considered that porting the mt9m114 driver to the NXP >>>> mx6s_capture, and fixing the same issues in the mx6s_capture driver >>>> than >>>> were present in the mainline driver, would require lots of time for >>>> likely very little gain. >>>> >>>> That being said, I've compared the two drivers, and I haven't seen >>>> anything in mx6s_capture that would address the specific issues I'm >>>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera >>>> module, and I will test those with the mainline driver. If they don't >>>> work, and assuming the NXP kernel driver works on that platform, I'll >>>> have two code bases to compare. >>>> >>>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on >>>>>>>> imx8mq: >>>>>>>> Using NXP's CSI bridge driver >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> as well as the CSI driver itself: >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> works fine when using the ov5640 camera with this driver: >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er >>>>>>>> >>>>>>>> >>>>>>>> (I realize there is a CSI bridge driver in staging, but that >>>>>>>> need more >>>>>>>> work to be actually used. Of course after this the goal is to >>>>>>>> fix and >>>>>>>> use it; and mainline a CSI phy driver too.) >>>>>>> >>>>>>> I have lots of patches for this driver, which I've developed on >>>>>>> i.MX7D >>>>>>> for a separate project. I'd like to mainline them, but this is >>>>>>> blocked >>>>>>> by one last issue that I haven't been able to solve yet. In a >>>>>>> nutshell, >>>>>>> the CSI writes two consecutive frames in each buffer, overflowing >>>>>>> the >>>>>>> allocated memory. The registers that control the buffer size seem >>>>>>> to be >>>>>>> programmed correctly as far as I can tell. I've reported this >>>>>>> issue to >>>>>>> NXP but haven't received any feedback yet. >>>>> >>>>> that's different from where I am. So you don't get any interrupt >>>>> (EOF or >>>>> other) when only *one* frame is written into one buffer? >>>> >>>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I >>>> haven't checked other interrupts. I thus get half of the frame rate. If >>>> I configure the CSI_CSIIMAG_PARA register with half of the actual image >>>> height, buffers then contain a single image and the frame rate goes to >>>> the expected value. That gives me a workaround, but without >>>> understanding the root cause, I'm worried that just halving the height >>>> would introduce other breakages. >>> >>> Hi Laurent, >>> >>> Thanks a lot, exchanging things this way alone is much appreciated. I'll >>> give you detailed feedback on the staging drivers later - I tried >>> running them, they probe but still a (gstreamer) stream would not start. >>> But most all of your addition make a lot of sense. >>> >>> Just so you know, to emphasize that once more: I'm still on the nxp >>> drivers (yes, they are very similar) just because the ov56 cam works >>> with them over here. >> >> I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and >> when I try to capture images, with the NXP kernel, I just don't get any. >> *sigh* I'll have to debug that. > > oh you have it already. very good. > > I guess the nxp kernel directly *should* work, but I use a kernel based > on v5.9.X and include the drivers like so: > https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 > > https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > > >> >>> The main problem I have is that, as soon as I try >>> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically >>> nothing works anymore. userspace stops to "be interested" pretty soon. >>> >>> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) >>> completed, the usual way. >>> >>> And when I break the ov56 by using a wrong pixelformat in mx6s_capture >>> (nxp bridge driver), I get equally get only one frame (dma) completed >>> there. >>> >>> How can I make this thing use a different pixelformat (by setting the >>> media bus format in the sensor driver)? >> >> All the components have to support the format you want to capture. This >> includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The >> pixel format only matters in the CSI driver, for the other two it's the >> media bus format that matters. To capture raw bayer data, you'll first >> need to extend the OV5640 driver to support the corresponding media bus >> format (and of course configure the sensor to output that format), then >> the MIPI_CSI driver to support that media bus format as well, and >> finally the CSI driver. > > ok, the sensor I work with outputs raw bayer only, and that sounds like > I should pay more attention to the mipi csi driver. > > that links to my experiments I occasionally update: > https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > >> >>> thanks again for sharing your code early. have a nice rest of the year, >> > > thanks, > martin hi Laurent, how are you? Am I right in that you tried to run the ov5640 mainline driver with the CSI bridge staging driver on imx8mq? Which mipi driver would you then use? Did you have progress in that area and in resolving things in order to push some of your changes out? I'm still struggling with the different sensor I want to use on imx8mq (hi846) and I still use the nxp bsp drivers for mipi and csi bridge: I was able to add and fix a few things and get some kind of image at least, but there's still definitely some sizes or timings wrong (in every resolution mode I think). 1280x720 shows it clearly as the image looks "almost" normal only when I resize it to 916 pixel width. Also, it "shifts" with later frames. For a few images of this, see https://source.puri.sm/Librem5/linux-next/-/issues/43#note_143875 I just wanted to describe the above just in case you've seen something like that during your experiments with sensors and can point me to places to look at. I do think that the SoC side should be ok now (but still now 100% sure), but many sensors are similar and maybe you have ideas there too. anyways, thanks for looking at this and as soon as I have a satisfying image with my current setup, I'll switch to the staging csi driver. But there's no driver for the mipi ("phy") in staging for imx8mq, right? thanks! martin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2021-02-23 9:04 ` Martin Kepplinger @ 2021-02-23 11:32 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2021-02-23 11:32 UTC (permalink / raw) To: Martin Kepplinger Cc: rogerio.silva, Philipp Zabel, slongerbeam, Fabio Estevam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Martin, On Tue, Feb 23, 2021 at 10:04:44AM +0100, Martin Kepplinger wrote: > On 22.12.20 16:55, Martin Kepplinger wrote: > > On 22.12.20 16:35, Laurent Pinchart wrote: > >> On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: > >>> On 10.12.20 17:04, Laurent Pinchart wrote: > >>>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > >>>>> On 10.12.20 13:17, Laurent Pinchart wrote: > >>>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >>>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>>>>>>> hi, > >>>>>>>> > >>>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with > >>>>>>>> other > >>>>>>>> cameras? > >>>>>>> > >>>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP > >>>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The > >>>>>>> least I > >>>>>>> can say is that it was painful :-( > >>>>>>> > >>>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and > >>>>>>> YUV and > >>>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which > >>>>>>> isn't supported by the mx6s_capture driver. > >>>>> > >>>>> so did you successfully use the NXP mx6s_capture driver with that > >>>>> sensor too? > >>>> > >>>> I haven't tried. The mx6s_capture driver and the mainline driver share a > >>>> similar architecture, as they came from the same code base. They have > >>>> diverged, with the mainline driver receiving bug fixes and new > >>>> features, and my large no-yet-upstreamed patch series adds lots of fixes > >>>> that are required for the mt9m114 driver to work with the driver. For > >>>> that reason, I've considered that porting the mt9m114 driver to the NXP > >>>> mx6s_capture, and fixing the same issues in the mx6s_capture driver than > >>>> were present in the mainline driver, would require lots of time for > >>>> likely very little gain. > >>>> > >>>> That being said, I've compared the two drivers, and I haven't seen > >>>> anything in mx6s_capture that would address the specific issues I'm > >>>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > >>>> module, and I will test those with the mainline driver. If they don't > >>>> work, and assuming the NXP kernel driver works on that platform, I'll > >>>> have two code bases to compare. > >>>> > >>>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>>>>>>> Using NXP's CSI bridge driver > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> as well as the CSI driver itself: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> works fine when using the ov5640 camera with this driver: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> > >>>>>>>> (I realize there is a CSI bridge driver in staging, but that need more > >>>>>>>> work to be actually used. Of course after this the goal is to fix and > >>>>>>>> use it; and mainline a CSI phy driver too.) > >>>>>>> > >>>>>>> I have lots of patches for this driver, which I've developed on i.MX7D > >>>>>>> for a separate project. I'd like to mainline them, but this is blocked > >>>>>>> by one last issue that I haven't been able to solve yet. In a nutshell, > >>>>>>> the CSI writes two consecutive frames in each buffer, overflowing the > >>>>>>> allocated memory. The registers that control the buffer size seem to be > >>>>>>> programmed correctly as far as I can tell. I've reported this issue to > >>>>>>> NXP but haven't received any feedback yet. > >>>>> > >>>>> that's different from where I am. So you don't get any interrupt (EOF or > >>>>> other) when only *one* frame is written into one buffer? > >>>> > >>>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > >>>> haven't checked other interrupts. I thus get half of the frame rate. If > >>>> I configure the CSI_CSIIMAG_PARA register with half of the actual image > >>>> height, buffers then contain a single image and the frame rate goes to > >>>> the expected value. That gives me a workaround, but without > >>>> understanding the root cause, I'm worried that just halving the height > >>>> would introduce other breakages. > >>> > >>> Hi Laurent, > >>> > >>> Thanks a lot, exchanging things this way alone is much appreciated. I'll > >>> give you detailed feedback on the staging drivers later - I tried > >>> running them, they probe but still a (gstreamer) stream would not start. > >>> But most all of your addition make a lot of sense. > >>> > >>> Just so you know, to emphasize that once more: I'm still on the nxp > >>> drivers (yes, they are very similar) just because the ov56 cam works > >>> with them over here. > >> > >> I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > >> when I try to capture images, with the NXP kernel, I just don't get any. > >> *sigh* I'll have to debug that. > > > > oh you have it already. very good. > > > > I guess the nxp kernel directly *should* work, but I use a kernel based > > on v5.9.X and include the drivers like so: > > https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 > > > > https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > > > >>> The main problem I have is that, as soon as I try > >>> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically > >>> nothing works anymore. userspace stops to "be interested" pretty soon. > >>> > >>> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) > >>> completed, the usual way. > >>> > >>> And when I break the ov56 by using a wrong pixelformat in mx6s_capture > >>> (nxp bridge driver), I get equally get only one frame (dma) completed > >>> there. > >>> > >>> How can I make this thing use a different pixelformat (by setting the > >>> media bus format in the sensor driver)? > >> > >> All the components have to support the format you want to capture. This > >> includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The > >> pixel format only matters in the CSI driver, for the other two it's the > >> media bus format that matters. To capture raw bayer data, you'll first > >> need to extend the OV5640 driver to support the corresponding media bus > >> format (and of course configure the sensor to output that format), then > >> the MIPI_CSI driver to support that media bus format as well, and > >> finally the CSI driver. > > > > ok, the sensor I work with outputs raw bayer only, and that sounds like > > I should pay more attention to the mipi csi driver. > > > > that links to my experiments I occasionally update: > > https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > > > >>> thanks again for sharing your code early. have a nice rest of the year, > > hi Laurent, how are you? Am I right in that you tried to run the ov5640 > mainline driver with the CSI bridge staging driver on imx8mq? Which mipi > driver would you then use? I've run it with the staging driver on i.MX8MP, not i.MX8MQ. The MQ seems to have the same CSI bridge, but a different CSI-2 receiver (and PHY). It would be great if you could contribute a driver for that :-) > Did you have progress in that area and in resolving things in order to > push some of your changes out? I've sent a large patch series recently, if you could review it it would speed up integration ;-) > I'm still struggling with the different sensor I want to use on imx8mq > (hi846) and I still use the nxp bsp drivers for mipi and csi bridge: > > I was able to add and fix a few things and get some kind of image at > least, but there's still definitely some sizes or timings wrong (in > every resolution mode I think). 1280x720 shows it clearly as the image > looks "almost" normal only when I resize it to 916 pixel width. Also, it > "shifts" with later frames. For a few images of this, see > > https://source.puri.sm/Librem5/linux-next/-/issues/43#note_143875 > > I just wanted to describe the above just in case you've seen something > like that during your experiments with sensors and can point me to > places to look at. I do think that the SoC side should be ok now (but > still now 100% sure), but many sensors are similar and maybe you have > ideas there too. I've seen similar (but not identical) issues that were due to incorrect Ths_settle values in the D-PHY. You could start by trying different values there. This seems to be controlled on i.MX8MQ by CSI2_1_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR34 or CSI2_2_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR41, but the value of the fields are not properly documented :-S The i.MX8MM suffered from the same issue, but a table with register values for different clock frequencies was posted by NXP on their support forum. Maybe the same information could be available there for i.M8XMQ ? > anyways, thanks for looking at this and as soon as I have a satisfying > image with my current setup, I'll switch to the staging csi driver. But > there's no driver for the mipi ("phy") in staging for imx8mq, right? -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2021-02-23 11:32 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2021-02-23 11:32 UTC (permalink / raw) To: Martin Kepplinger Cc: kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Fabio Estevam, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Martin, On Tue, Feb 23, 2021 at 10:04:44AM +0100, Martin Kepplinger wrote: > On 22.12.20 16:55, Martin Kepplinger wrote: > > On 22.12.20 16:35, Laurent Pinchart wrote: > >> On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: > >>> On 10.12.20 17:04, Laurent Pinchart wrote: > >>>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > >>>>> On 10.12.20 13:17, Laurent Pinchart wrote: > >>>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >>>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>>>>>>> hi, > >>>>>>>> > >>>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with > >>>>>>>> other > >>>>>>>> cameras? > >>>>>>> > >>>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP > >>>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The > >>>>>>> least I > >>>>>>> can say is that it was painful :-( > >>>>>>> > >>>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and > >>>>>>> YUV and > >>>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which > >>>>>>> isn't supported by the mx6s_capture driver. > >>>>> > >>>>> so did you successfully use the NXP mx6s_capture driver with that > >>>>> sensor too? > >>>> > >>>> I haven't tried. The mx6s_capture driver and the mainline driver share a > >>>> similar architecture, as they came from the same code base. They have > >>>> diverged, with the mainline driver receiving bug fixes and new > >>>> features, and my large no-yet-upstreamed patch series adds lots of fixes > >>>> that are required for the mt9m114 driver to work with the driver. For > >>>> that reason, I've considered that porting the mt9m114 driver to the NXP > >>>> mx6s_capture, and fixing the same issues in the mx6s_capture driver than > >>>> were present in the mainline driver, would require lots of time for > >>>> likely very little gain. > >>>> > >>>> That being said, I've compared the two drivers, and I haven't seen > >>>> anything in mx6s_capture that would address the specific issues I'm > >>>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > >>>> module, and I will test those with the mainline driver. If they don't > >>>> work, and assuming the NXP kernel driver works on that platform, I'll > >>>> have two code bases to compare. > >>>> > >>>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>>>>>>> Using NXP's CSI bridge driver > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> as well as the CSI driver itself: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> works fine when using the ov5640 camera with this driver: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> > >>>>>>>> (I realize there is a CSI bridge driver in staging, but that need more > >>>>>>>> work to be actually used. Of course after this the goal is to fix and > >>>>>>>> use it; and mainline a CSI phy driver too.) > >>>>>>> > >>>>>>> I have lots of patches for this driver, which I've developed on i.MX7D > >>>>>>> for a separate project. I'd like to mainline them, but this is blocked > >>>>>>> by one last issue that I haven't been able to solve yet. In a nutshell, > >>>>>>> the CSI writes two consecutive frames in each buffer, overflowing the > >>>>>>> allocated memory. The registers that control the buffer size seem to be > >>>>>>> programmed correctly as far as I can tell. I've reported this issue to > >>>>>>> NXP but haven't received any feedback yet. > >>>>> > >>>>> that's different from where I am. So you don't get any interrupt (EOF or > >>>>> other) when only *one* frame is written into one buffer? > >>>> > >>>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > >>>> haven't checked other interrupts. I thus get half of the frame rate. If > >>>> I configure the CSI_CSIIMAG_PARA register with half of the actual image > >>>> height, buffers then contain a single image and the frame rate goes to > >>>> the expected value. That gives me a workaround, but without > >>>> understanding the root cause, I'm worried that just halving the height > >>>> would introduce other breakages. > >>> > >>> Hi Laurent, > >>> > >>> Thanks a lot, exchanging things this way alone is much appreciated. I'll > >>> give you detailed feedback on the staging drivers later - I tried > >>> running them, they probe but still a (gstreamer) stream would not start. > >>> But most all of your addition make a lot of sense. > >>> > >>> Just so you know, to emphasize that once more: I'm still on the nxp > >>> drivers (yes, they are very similar) just because the ov56 cam works > >>> with them over here. > >> > >> I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > >> when I try to capture images, with the NXP kernel, I just don't get any. > >> *sigh* I'll have to debug that. > > > > oh you have it already. very good. > > > > I guess the nxp kernel directly *should* work, but I use a kernel based > > on v5.9.X and include the drivers like so: > > https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 > > > > https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > > > >>> The main problem I have is that, as soon as I try > >>> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically > >>> nothing works anymore. userspace stops to "be interested" pretty soon. > >>> > >>> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) > >>> completed, the usual way. > >>> > >>> And when I break the ov56 by using a wrong pixelformat in mx6s_capture > >>> (nxp bridge driver), I get equally get only one frame (dma) completed > >>> there. > >>> > >>> How can I make this thing use a different pixelformat (by setting the > >>> media bus format in the sensor driver)? > >> > >> All the components have to support the format you want to capture. This > >> includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The > >> pixel format only matters in the CSI driver, for the other two it's the > >> media bus format that matters. To capture raw bayer data, you'll first > >> need to extend the OV5640 driver to support the corresponding media bus > >> format (and of course configure the sensor to output that format), then > >> the MIPI_CSI driver to support that media bus format as well, and > >> finally the CSI driver. > > > > ok, the sensor I work with outputs raw bayer only, and that sounds like > > I should pay more attention to the mipi csi driver. > > > > that links to my experiments I occasionally update: > > https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > > > >>> thanks again for sharing your code early. have a nice rest of the year, > > hi Laurent, how are you? Am I right in that you tried to run the ov5640 > mainline driver with the CSI bridge staging driver on imx8mq? Which mipi > driver would you then use? I've run it with the staging driver on i.MX8MP, not i.MX8MQ. The MQ seems to have the same CSI bridge, but a different CSI-2 receiver (and PHY). It would be great if you could contribute a driver for that :-) > Did you have progress in that area and in resolving things in order to > push some of your changes out? I've sent a large patch series recently, if you could review it it would speed up integration ;-) > I'm still struggling with the different sensor I want to use on imx8mq > (hi846) and I still use the nxp bsp drivers for mipi and csi bridge: > > I was able to add and fix a few things and get some kind of image at > least, but there's still definitely some sizes or timings wrong (in > every resolution mode I think). 1280x720 shows it clearly as the image > looks "almost" normal only when I resize it to 916 pixel width. Also, it > "shifts" with later frames. For a few images of this, see > > https://source.puri.sm/Librem5/linux-next/-/issues/43#note_143875 > > I just wanted to describe the above just in case you've seen something > like that during your experiments with sensors and can point me to > places to look at. I do think that the SoC side should be ok now (but > still now 100% sure), but many sensors are similar and maybe you have > ideas there too. I've seen similar (but not identical) issues that were due to incorrect Ths_settle values in the D-PHY. You could start by trying different values there. This seems to be controlled on i.MX8MQ by CSI2_1_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR34 or CSI2_2_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR41, but the value of the fields are not properly documented :-S The i.MX8MM suffered from the same issue, but a table with register values for different clock frequencies was posted by NXP on their support forum. Maybe the same information could be available there for i.M8XMQ ? > anyways, thanks for looking at this and as soon as I have a satisfying > image with my current setup, I'll switch to the staging csi driver. But > there's no driver for the mipi ("phy") in staging for imx8mq, right? -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-22 15:35 ` Laurent Pinchart @ 2020-12-22 17:15 ` Fabio Estevam -1 siblings, 0 replies; 24+ messages in thread From: Fabio Estevam @ 2020-12-22 17:15 UTC (permalink / raw) To: Laurent Pinchart Cc: Martin Kepplinger, rogerio.silva, Philipp Zabel, slongerbeam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Laurent, On Tue, Dec 22, 2020 at 12:35 PM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > when I try to capture images, with the NXP kernel, I just don't get any. > *sigh* I'll have to debug that. OV5647? The default camera for the imx8mm-evk board is the OV5640. Regards, Fabio Estevam ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-22 17:15 ` Fabio Estevam 0 siblings, 0 replies; 24+ messages in thread From: Fabio Estevam @ 2020-12-22 17:15 UTC (permalink / raw) To: Laurent Pinchart Cc: Martin Kepplinger, kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Laurent, On Tue, Dec 22, 2020 at 12:35 PM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > when I try to capture images, with the NXP kernel, I just don't get any. > *sigh* I'll have to debug that. OV5647? The default camera for the imx8mm-evk board is the OV5640. Regards, Fabio Estevam _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? 2020-12-22 17:15 ` Fabio Estevam @ 2020-12-22 18:16 ` Laurent Pinchart -1 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-22 18:16 UTC (permalink / raw) To: Fabio Estevam Cc: Martin Kepplinger, rogerio.silva, Philipp Zabel, slongerbeam, linux-arm-kernel, linux-media, NXP Linux Team, kernel, Pengutronix Kernel Team Hi Fabio, On Tue, Dec 22, 2020 at 02:15:19PM -0300, Fabio Estevam wrote: > On Tue, Dec 22, 2020 at 12:35 PM Laurent Pinchart wrote: > > > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > > when I try to capture images, with the NXP kernel, I just don't get any. > > *sigh* I'll have to debug that. > > OV5647? The default camera for the imx8mm-evk board is the OV5640. Sorry, I meant OV5640, it was a typo. -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? @ 2020-12-22 18:16 ` Laurent Pinchart 0 siblings, 0 replies; 24+ messages in thread From: Laurent Pinchart @ 2020-12-22 18:16 UTC (permalink / raw) To: Fabio Estevam Cc: Martin Kepplinger, kernel, NXP Linux Team, Philipp Zabel, slongerbeam, rogerio.silva, Pengutronix Kernel Team, linux-arm-kernel, linux-media Hi Fabio, On Tue, Dec 22, 2020 at 02:15:19PM -0300, Fabio Estevam wrote: > On Tue, Dec 22, 2020 at 12:35 PM Laurent Pinchart wrote: > > > I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > > when I try to capture images, with the NXP kernel, I just don't get any. > > *sigh* I'll have to debug that. > > OV5647? The default camera for the imx8mm-evk board is the OV5640. Sorry, I meant OV5640, it was a typo. -- Regards, Laurent Pinchart _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2021-02-23 11:34 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-10 8:17 question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? Martin Kepplinger 2020-12-10 8:17 ` Martin Kepplinger 2020-12-10 9:23 ` Laurent Pinchart 2020-12-10 9:23 ` Laurent Pinchart 2020-12-10 12:17 ` Laurent Pinchart 2020-12-10 12:17 ` Laurent Pinchart 2020-12-10 13:12 ` Martin Kepplinger 2020-12-10 13:12 ` Martin Kepplinger 2020-12-10 16:04 ` Laurent Pinchart 2020-12-10 16:04 ` Laurent Pinchart 2020-12-22 14:06 ` Martin Kepplinger 2020-12-22 14:06 ` Martin Kepplinger 2020-12-22 15:35 ` Laurent Pinchart 2020-12-22 15:35 ` Laurent Pinchart 2020-12-22 15:55 ` Martin Kepplinger 2020-12-22 15:55 ` Martin Kepplinger 2021-02-23 9:04 ` Martin Kepplinger 2021-02-23 9:04 ` Martin Kepplinger 2021-02-23 11:32 ` Laurent Pinchart 2021-02-23 11:32 ` Laurent Pinchart 2020-12-22 17:15 ` Fabio Estevam 2020-12-22 17:15 ` Fabio Estevam 2020-12-22 18:16 ` Laurent Pinchart 2020-12-22 18:16 ` Laurent Pinchart
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.