soc_camera dynamically cropping and scaling

soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi all,

I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
Image sensor. I am able to use the cropping and scaling capabilities
V4L2 driver.
The question is :

Is it possible dynamically change the cropping and scaling values
without close and re-open  the camera every time ?

Now I am using the streaming I/O memory mapping and to dynamically
change the cropping and scaling values I do :

1) stop capturing using VIDIOC_STREAMOFF;
2) unmap all the buffers;
3) close the device;
4) open the device;
5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
the target image width and height, (scaling).
6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
mmap each buffer using VIDIOC_QUERYBUF and mmap():

this procedure works but take 400 ms.

If I omit steps 3) and 4)  (close and re-open the device) I get this errors:

camera 0-0: S_CROP denied: queue initialised and sizes differ
camera 0-0: S_FMT denied: queue initialised
VIDIOC_S_FMT error 16, Device or resource busy
pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0

Do you have some Idea regarding why I have to close and reopen the
device and regarding a way to speed up these change?

Thanks in advance

Paolo Santinelli

-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

Re: soc_camera dynamically cropping and scaling

[Guennadi Liakhovetski]

On Tue, 29 Mar 2011, Paolo Santinelli wrote:

> Hi all,
> 
> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
> Image sensor. I am able to use the cropping and scaling capabilities
> V4L2 driver.
> The question is :
> 
> Is it possible dynamically change the cropping and scaling values
> without close and re-open  the camera every time ?
> 
> Now I am using the streaming I/O memory mapping and to dynamically
> change the cropping and scaling values I do :
> 
> 1) stop capturing using VIDIOC_STREAMOFF;
> 2) unmap all the buffers;
> 3) close the device;
> 4) open the device;
> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
> the target image width and height, (scaling).
> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
> mmap each buffer using VIDIOC_QUERYBUF and mmap():
> 
> this procedure works but take 400 ms.
> 
> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
> 
> camera 0-0: S_CROP denied: queue initialised and sizes differ
> camera 0-0: S_FMT denied: queue initialised
> VIDIOC_S_FMT error 16, Device or resource busy
> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
> 
> Do you have some Idea regarding why I have to close and reopen the
> device and regarding a way to speed up these change?

Yes, by chance I do;-) First of all you have to make it more precise - 
what exactly do you mean - dynamic (I call it "live") scaling or cropping? 
If you want to change output format, that will not be easy ATM, that will 
require the snapshot mode API, which is not yet even in an RFC state. If 
you only want to change the cropping and keep the output format (zoom), 
then I've just implemented that for sh_mobile_ceu_camera. This requires a 
couple of extensions to the soc-camera core, which I can post tomorrow. 
But in fact that is also a hack, because the proper way to implement this 
is to port soc-camera to the Media Controller framework and use the 
pad-level API. So, I am not sure, whether we want this in the mainline, 
but if already two of us need it now - before the transition to pad-level 
operations, maybe it would make sense to mainline this. If, however, you 
do have to change your output window, maybe you could tell us your 
use-case, so that we could consider, what's the best way to support that.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi Guennadi,

thank you for the quick answer.

Here is what I mean with dynamic: I take "live" one frame at high
resolution, for example a picture at VGA or  QVGA resolution, then a
sequence of frames that depict a cropped area (200x200 or 100x100)
from the original full-resolution frame, and then a new full
resolution image (VGA or QVGA) and again the sequence of frames  that
depict a cropped area from the original full resolution, and so on.
That means takes one frame in 640x480 and  than takes some frames at
100x100 (or 200x200) and so on.

The best would be have two different fixed-output image formats, the
WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
pictures obtained cropping the a region of the whole image. The
cropping area could be even wider  than 100x100 and then scaled down
to the 100x100 ROI format.

Probably it is more simple have a cropping area of the same dimension
of the ROI format, 100x100.

In this way there is a reduction of the computation load of the CPU
(smaller images).

Thank you very much!

Paolo

2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>
>> Hi all,
>>
>> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>> Image sensor. I am able to use the cropping and scaling capabilities
>> V4L2 driver.
>> The question is :
>>
>> Is it possible dynamically change the cropping and scaling values
>> without close and re-open  the camera every time ?
>>
>> Now I am using the streaming I/O memory mapping and to dynamically
>> change the cropping and scaling values I do :
>>
>> 1) stop capturing using VIDIOC_STREAMOFF;
>> 2) unmap all the buffers;
>> 3) close the device;
>> 4) open the device;
>> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>> the target image width and height, (scaling).
>> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>>
>> this procedure works but take 400 ms.
>>
>> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>>
>> camera 0-0: S_CROP denied: queue initialised and sizes differ
>> camera 0-0: S_FMT denied: queue initialised
>> VIDIOC_S_FMT error 16, Device or resource busy
>> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>>
>> Do you have some Idea regarding why I have to close and reopen the
>> device and regarding a way to speed up these change?
>
> Yes, by chance I do;-) First of all you have to make it more precise -
> what exactly do you mean - dynamic (I call it "live") scaling or cropping?
> If you want to change output format, that will not be easy ATM, that will
> require the snapshot mode API, which is not yet even in an RFC state. If
> you only want to change the cropping and keep the output format (zoom),
> then I've just implemented that for sh_mobile_ceu_camera. This requires a
> couple of extensions to the soc-camera core, which I can post tomorrow.
> But in fact that is also a hack, because the proper way to implement this
> is to port soc-camera to the Media Controller framework and use the
> pad-level API. So, I am not sure, whether we want this in the mainline,
> but if already two of us need it now - before the transition to pad-level
> operations, maybe it would make sense to mainline this. If, however, you
> do have to change your output window, maybe you could tell us your
> use-case, so that we could consider, what's the best way to support that.
>
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

[PATCH/DRAFT 0/2] Add livecrop to soc-camera and to sh CEU

[Guennadi Liakhovetski]

This is just a dump of an idea, I've been toying with to support live zoom 
on soc-camera. I do not know, if we'll end up mainlining this, because the 
proper way to do this is to use MC / pad-level operations, and even if we 
do this, at least the second patch will have to be further split into 2 or 
three patches. So, this is mainly just for Paolo to have a look.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

[PATCH 1/2] V4L: soc-camera: add a livecrop host operation

[Guennadi Liakhovetski]

Add an soc-camera host livecrop operation to implement live zoom. If
a host driver implements it, it should take care to preserve output
frame format, then live crop doesn't break streaming.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/soc_camera.c |   20 ++++++++++++++------
 include/media/soc_camera.h       |    5 +++++
 2 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index 2f0fd2f..11f0f1e 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -41,6 +41,11 @@
 #define DEFAULT_WIDTH	640
 #define DEFAULT_HEIGHT	480
 
+#define is_streaming(ici, icd)				\
+	(((ici)->ops->init_videobuf) ?			\
+	 (icd)->vb_vidq.streaming :			\
+	 vb2_is_streaming(&(icd)->vb2_vidq))
+
 static LIST_HEAD(hosts);
 static LIST_HEAD(devices);
 static DEFINE_MUTEX(list_lock);		/* Protects the list of hosts */
@@ -626,7 +631,7 @@ static int soc_camera_s_fmt_vid_cap(struct file *file, void *priv,
 	if (icd->streamer && icd->streamer != file)
 		return -EBUSY;
 
-	if (icd->vb_vidq.bufs[0]) {
+	if (is_streaming(to_soc_camera_host(icd->dev.parent), icd)) {
 		dev_err(&icd->dev, "S_FMT denied: queue initialised\n");
 		return -EBUSY;
 	}
@@ -867,14 +872,17 @@ static int soc_camera_s_crop(struct file *file, void *fh,
 	if (ret < 0) {
 		dev_err(&icd->dev,
 			"S_CROP denied: getting current crop failed\n");
-	} else if (icd->vb_vidq.bufs[0] &&
-		   (a->c.width != current_crop.c.width ||
-		    a->c.height != current_crop.c.height)) {
+	} else if ((a->c.width == current_crop.c.width &&
+		    a->c.height == current_crop.c.height) ||
+		   !is_streaming(ici, icd)) {
+		/* same size or not streaming - use .set_crop() */
+		ret = ici->ops->set_crop(icd, a);
+	} else if (ici->ops->set_livecrop) {
+		ret = ici->ops->set_livecrop(icd, a);
+	} else {
 		dev_err(&icd->dev,
 			"S_CROP denied: queue initialised and sizes differ\n");
 		ret = -EBUSY;
-	} else {
-		ret = ici->ops->set_crop(icd, a);
 	}
 
 	return ret;
diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
index f80b537..844cd09 100644
--- a/include/media/soc_camera.h
+++ b/include/media/soc_camera.h
@@ -80,6 +80,11 @@ struct soc_camera_host_ops {
 	int (*cropcap)(struct soc_camera_device *, struct v4l2_cropcap *);
 	int (*get_crop)(struct soc_camera_device *, struct v4l2_crop *);
 	int (*set_crop)(struct soc_camera_device *, struct v4l2_crop *);
+	/*
+	 * The difference to .set_crop() is, that .set_livecrop is not allowed
+	 * to change the output sizes
+	 */
+	int (*set_livecrop)(struct soc_camera_device *, struct v4l2_crop *);
 	int (*set_fmt)(struct soc_camera_device *, struct v4l2_format *);
 	int (*try_fmt)(struct soc_camera_device *, struct v4l2_format *);
 	void (*init_videobuf)(struct videobuf_queue *,
-- 
1.7.2.5

[PATCH/DRAFT 2/2] V4L: sh_mobile_ceu_camera: implement live cropping

[Guennadi Liakhovetski]

PRELIMINARY: break out spinlock changes; consider multiple completing
feames, causing multiple complete() calles.

Add live crop support to the sh_mobile_ceu driver.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---

As I mentioned in the introductory mail, this is just a draft, that at the 
very least has to be split into 2 or three individual patches. The parts 
in this patch, relevant to live-crop are the addition of the 
.set_livecrop() method itself, of course, the addition of a completion, 
the "frozen" flag and changed to sh_mobile_ceu_capture().

 drivers/media/video/sh_mobile_ceu_camera.c |  143 ++++++++++++++++++++++++----
 1 files changed, 125 insertions(+), 18 deletions(-)

diff --git a/drivers/media/video/sh_mobile_ceu_camera.c b/drivers/media/video/sh_mobile_ceu_camera.c
index 8ff3683..d1446ad 100644
--- a/drivers/media/video/sh_mobile_ceu_camera.c
+++ b/drivers/media/video/sh_mobile_ceu_camera.c
@@ -17,6 +17,7 @@
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/io.h>
+#include <linux/completion.h>
 #include <linux/delay.h>
 #include <linux/dma-mapping.h>
 #include <linux/errno.h>
@@ -106,6 +107,7 @@ struct sh_mobile_ceu_dev {
 	struct vb2_alloc_ctx *alloc_ctx;
 
 	struct sh_mobile_ceu_info *pdata;
+	struct completion complete;
 
 	u32 cflcr;
 
@@ -114,6 +116,7 @@ struct sh_mobile_ceu_dev {
 
 	unsigned int image_mode:1;
 	unsigned int is_16bit:1;
+	unsigned int frozen:1;
 };
 
 struct sh_mobile_ceu_cam {
@@ -273,7 +276,8 @@ static int sh_mobile_ceu_capture(struct sh_mobile_ceu_dev *pcdev)
 	ceu_write(pcdev, CEIER, ceu_read(pcdev, CEIER) & ~CEU_CEIER_MASK);
 	status = ceu_read(pcdev, CETCR);
 	ceu_write(pcdev, CETCR, ~status & CEU_CETCR_MAGIC);
-	ceu_write(pcdev, CEIER, ceu_read(pcdev, CEIER) | CEU_CEIER_MASK);
+	if (!pcdev->frozen)
+		ceu_write(pcdev, CEIER, ceu_read(pcdev, CEIER) | CEU_CEIER_MASK);
 	ceu_write(pcdev, CAPCR, ceu_read(pcdev, CAPCR) & ~CEU_CAPCR_CTNCP);
 	ceu_write(pcdev, CETCR, CEU_CETCR_MAGIC ^ CEU_CETCR_IGRW);
 
@@ -287,6 +291,11 @@ static int sh_mobile_ceu_capture(struct sh_mobile_ceu_dev *pcdev)
 		ret = -EIO;
 	}
 
+	if (pcdev->frozen) {
+		complete(&pcdev->complete);
+		return ret;
+	}
+
 	if (!pcdev->active)
 		return ret;
 
@@ -378,12 +387,11 @@ static void sh_mobile_ceu_videobuf_queue(struct vb2_buffer *vb)
 	struct soc_camera_host *ici = to_soc_camera_host(icd->dev.parent);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
 	struct sh_mobile_ceu_buffer *buf = to_ceu_vb(vb);
-	unsigned long flags;
 
 	dev_dbg(icd->dev.parent, "%s (vb=0x%p) 0x%p %lu\n", __func__,
 		vb, vb2_plane_vaddr(vb, 0), vb2_get_plane_payload(vb, 0));
 
-	spin_lock_irqsave(&pcdev->lock, flags);
+	spin_lock_irq(&pcdev->lock);
 	list_add_tail(&buf->queue, &pcdev->capture);
 
 	if (!pcdev->active) {
@@ -395,7 +403,7 @@ static void sh_mobile_ceu_videobuf_queue(struct vb2_buffer *vb)
 		pcdev->active = vb;
 		sh_mobile_ceu_capture(pcdev);
 	}
-	spin_unlock_irqrestore(&pcdev->lock, flags);
+	spin_unlock_irq(&pcdev->lock);
 }
 
 static void sh_mobile_ceu_videobuf_release(struct vb2_buffer *vb)
@@ -404,9 +412,8 @@ static void sh_mobile_ceu_videobuf_release(struct vb2_buffer *vb)
 	struct soc_camera_host *ici = to_soc_camera_host(icd->dev.parent);
 	struct sh_mobile_ceu_buffer *buf = to_ceu_vb(vb);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
-	unsigned long flags;
 
-	spin_lock_irqsave(&pcdev->lock, flags);
+	spin_lock_irq(&pcdev->lock);
 
 	if (pcdev->active == vb) {
 		/* disable capture (release DMA buffer), reset */
@@ -417,7 +424,7 @@ static void sh_mobile_ceu_videobuf_release(struct vb2_buffer *vb)
 	/* Doesn't hurt also if the list is empty */
 	list_del_init(&buf->queue);
 
-	spin_unlock_irqrestore(&pcdev->lock, flags);
+	spin_unlock_irq(&pcdev->lock);
 }
 
 static int sh_mobile_ceu_videobuf_init(struct vb2_buffer *vb)
@@ -433,16 +440,15 @@ static int sh_mobile_ceu_stop_streaming(struct vb2_queue *q)
 	struct soc_camera_host *ici = to_soc_camera_host(icd->dev.parent);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
 	struct list_head *buf_head, *tmp;
-	unsigned long flags;
 
-	spin_lock_irqsave(&pcdev->lock, flags);
+	spin_lock_irq(&pcdev->lock);
 
 	pcdev->active = NULL;
 
 	list_for_each_safe(buf_head, tmp, &pcdev->capture)
 		list_del_init(buf_head);
 
-	spin_unlock_irqrestore(&pcdev->lock, flags);
+	spin_unlock_irq(&pcdev->lock);
 
 	return sh_mobile_ceu_soft_reset(pcdev);
 }
@@ -521,7 +527,6 @@ static void sh_mobile_ceu_remove_device(struct soc_camera_device *icd)
 {
 	struct soc_camera_host *ici = to_soc_camera_host(icd->dev.parent);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
-	unsigned long flags;
 
 	BUG_ON(icd != pcdev->icd);
 
@@ -530,13 +535,13 @@ static void sh_mobile_ceu_remove_device(struct soc_camera_device *icd)
 	sh_mobile_ceu_soft_reset(pcdev);
 
 	/* make sure active buffer is canceled */
-	spin_lock_irqsave(&pcdev->lock, flags);
+	spin_lock_irq(&pcdev->lock);
 	if (pcdev->active) {
 		list_del_init(&to_ceu_vb(pcdev->active)->queue);
 		vb2_buffer_done(pcdev->active, VB2_BUF_STATE_ERROR);
 		pcdev->active = NULL;
 	}
-	spin_unlock_irqrestore(&pcdev->lock, flags);
+	spin_unlock_irq(&pcdev->lock);
 
 	pm_runtime_put_sync(ici->v4l2_dev.dev);
 
@@ -1351,7 +1356,7 @@ static int client_scale(struct soc_camera_device *icd,
 /*
  * CEU can scale and crop, but we don't want to waste bandwidth and kill the
  * framerate by always requesting the maximum image from the client. See
- * Documentation/video4linux/sh_mobile_camera_ceu.txt for a description of
+ * Documentation/video4linux/sh_mobile_ceu_camera.txt for a description of
  * scaling and cropping algorithms and for the meaning of referenced here steps.
  */
 static int sh_mobile_ceu_set_crop(struct soc_camera_device *icd,
@@ -1398,10 +1403,6 @@ static int sh_mobile_ceu_set_crop(struct soc_camera_device *icd,
 	if (mf.width > 2560 || mf.height > 1920)
 		return -EINVAL;
 
-	/* Cache camera output window */
-	cam->width	= mf.width;
-	cam->height	= mf.height;
-
 	/* 4. Calculate camera scales */
 	scale_cam_h	= calc_generic_scale(cam_rect->width, mf.width);
 	scale_cam_v	= calc_generic_scale(cam_rect->height, mf.height);
@@ -1410,6 +1411,39 @@ static int sh_mobile_ceu_set_crop(struct soc_camera_device *icd,
 	interm_width	= scale_down(rect->width, scale_cam_h);
 	interm_height	= scale_down(rect->height, scale_cam_v);
 
+	if (interm_width < icd->user_width) {
+		u32 new_scale_h;
+
+		new_scale_h = calc_generic_scale(rect->width, icd->user_width);
+
+		mf.width = scale_down(cam_rect->width, new_scale_h);
+	}
+
+	if (interm_height < icd->user_height) {
+		u32 new_scale_v;
+
+		new_scale_v = calc_generic_scale(rect->height, icd->user_height);
+
+		mf.height = scale_down(cam_rect->height, new_scale_v);
+	}
+
+	if (interm_width < icd->user_width || interm_height < icd->user_height) {
+		ret = v4l2_device_call_until_err(sd->v4l2_dev, (int)icd, video,
+						 s_mbus_fmt, &mf);
+		if (ret < 0)
+			return ret;
+
+		dev_geo(dev, "New camera output %ux%u\n", mf.width, mf.height);
+		scale_cam_h	= calc_generic_scale(cam_rect->width, mf.width);
+		scale_cam_v	= calc_generic_scale(cam_rect->height, mf.height);
+		interm_width	= scale_down(rect->width, scale_cam_h);
+		interm_height	= scale_down(rect->height, scale_cam_v);
+	}
+
+	/* Cache camera output window */
+	cam->width	= mf.width;
+	cam->height	= mf.height;
+
 	if (pcdev->image_mode) {
 		out_width	= min(interm_width, icd->user_width);
 		out_height	= min(interm_height, icd->user_height);
@@ -1725,6 +1759,78 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 	return ret;
 }
 
+static int sh_mobile_ceu_set_livecrop(struct soc_camera_device *icd,
+				      struct v4l2_crop *a)
+{
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	struct soc_camera_host *ici = to_soc_camera_host(icd->dev.parent);
+	struct sh_mobile_ceu_dev *pcdev = ici->priv;
+	u32 out_width = icd->user_width, out_height = icd->user_height;
+	int ret;
+
+	init_completion(&pcdev->complete);
+
+	/* Freeze queue */
+	pcdev->frozen = 1;
+
+	/* Wait for frame */
+	if (pcdev->active) {
+		/*
+		 * No race: if an interrupt hits here, ->frozen is already set
+		 * and the completion will be signalled
+		 */
+		ret = wait_for_completion_interruptible(&pcdev->complete);
+		if (ret < 0) {
+			/* This includes -EINTR */
+			pcdev->frozen = 0;
+			return ret;
+		}
+	}
+
+	/* Stop the client */
+	ret = v4l2_subdev_call(sd, video, s_stream, 0);
+	if (ret < 0)
+		dev_warn(icd->dev.parent,
+			 "Client failed to stop the stream: %d\n", ret);
+	else
+		/* Do the crop, if it fails, there's nothing more we can do */
+		sh_mobile_ceu_set_crop(icd, a);
+
+	dev_geo(icd->dev.parent, "Output after crop: %ux%u\n", icd->user_width, icd->user_height);
+
+	if (icd->user_width != out_width || icd->user_height != out_height) {
+		struct v4l2_format f = {
+			.type	= V4L2_BUF_TYPE_VIDEO_CAPTURE,
+			.fmt.pix	= {
+				.width		= out_width,
+				.height		= out_height,
+				.pixelformat	= icd->current_fmt->host_fmt->fourcc,
+				.field		= pcdev->field,
+				.colorspace	= icd->colorspace,
+			},
+		};
+		ret = sh_mobile_ceu_set_fmt(icd, &f);
+		if (!ret && (out_width != f.fmt.pix.width ||
+			     out_height != f.fmt.pix.height))
+			ret = -EINVAL;
+		if (!ret) {
+			icd->user_width		= out_width;
+			icd->user_height	= out_height;
+			ret = sh_mobile_ceu_set_bus_param(icd,
+					icd->current_fmt->host_fmt->fourcc);
+		}
+	}
+
+	/* Thaw the queue */
+	pcdev->frozen = 0;
+	spin_lock_irq(&pcdev->lock);
+	sh_mobile_ceu_capture(pcdev);
+	spin_unlock_irq(&pcdev->lock);
+	/* Start the client */
+	ret = v4l2_subdev_call(sd, video, s_stream, 1);
+	return ret;
+}
+
 static unsigned int sh_mobile_ceu_poll(struct file *file, poll_table *pt)
 {
 	struct soc_camera_device *icd = file->private_data;
@@ -1811,6 +1917,7 @@ static struct soc_camera_host_ops sh_mobile_ceu_host_ops = {
 	.put_formats	= sh_mobile_ceu_put_formats,
 	.get_crop	= sh_mobile_ceu_get_crop,
 	.set_crop	= sh_mobile_ceu_set_crop,
+	.set_livecrop	= sh_mobile_ceu_set_livecrop,
 	.set_fmt	= sh_mobile_ceu_set_fmt,
 	.try_fmt	= sh_mobile_ceu_try_fmt,
 	.set_ctrl	= sh_mobile_ceu_set_ctrl,
-- 
1.7.2.5

Re: soc_camera dynamically cropping and scaling

[Guennadi Liakhovetski]

On Tue, 29 Mar 2011, Paolo Santinelli wrote:

> Hi Guennadi,
> 
> thank you for the quick answer.
> 
> Here is what I mean with dynamic: I take "live" one frame at high
> resolution, for example a picture at VGA or  QVGA resolution, then a
> sequence of frames that depict a cropped area (200x200 or 100x100)
> from the original full-resolution frame, and then a new full
> resolution image (VGA or QVGA) and again the sequence of frames  that
> depict a cropped area from the original full resolution, and so on.
> That means takes one frame in 640x480 and  than takes some frames at
> 100x100 (or 200x200) and so on.

Ic, so, if you can live with a fixed output format and only change the 
input cropping rectangle, the patch set, that I've just sent could give 
you a hint, how this can be done. This would work if you're ok with first 
obtaining VGA images scaled down to, say, 160x120, and then take 160x120 
cropped frames unscaled. But I'm not sure, this is something, that would 
work for you. Otherwise, unless your sensor can upscale cropped images to 
VGA output size, you'll also want fast switching between different output 
sizes, which you'd have to wait for (or implement yourself;-))

Thanks
Guennadi

> 
> The best would be have two different fixed-output image formats, the
> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
> pictures obtained cropping the a region of the whole image. The
> cropping area could be even wider  than 100x100 and then scaled down
> to the 100x100 ROI format.
> 
> Probably it is more simple have a cropping area of the same dimension
> of the ROI format, 100x100.
> 
> In this way there is a reduction of the computation load of the CPU
> (smaller images).
> 
> Thank you very much!
> 
> Paolo
> 
> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
> >
> >> Hi all,
> >>
> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
> >> Image sensor. I am able to use the cropping and scaling capabilities
> >> V4L2 driver.
> >> The question is :
> >>
> >> Is it possible dynamically change the cropping and scaling values
> >> without close and re-open  the camera every time ?
> >>
> >> Now I am using the streaming I/O memory mapping and to dynamically
> >> change the cropping and scaling values I do :
> >>
> >> 1) stop capturing using VIDIOC_STREAMOFF;
> >> 2) unmap all the buffers;
> >> 3) close the device;
> >> 4) open the device;
> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
> >> the target image width and height, (scaling).
> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
> >>
> >> this procedure works but take 400 ms.
> >>
> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
> >>
> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
> >> camera 0-0: S_FMT denied: queue initialised
> >> VIDIOC_S_FMT error 16, Device or resource busy
> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
> >>
> >> Do you have some Idea regarding why I have to close and reopen the
> >> device and regarding a way to speed up these change?
> >
> > Yes, by chance I do;-) First of all you have to make it more precise -
> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
> > If you want to change output format, that will not be easy ATM, that will
> > require the snapshot mode API, which is not yet even in an RFC state. If
> > you only want to change the cropping and keep the output format (zoom),
> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
> > couple of extensions to the soc-camera core, which I can post tomorrow.
> > But in fact that is also a hack, because the proper way to implement this
> > is to port soc-camera to the Media Controller framework and use the
> > pad-level API. So, I am not sure, whether we want this in the mainline,
> > but if already two of us need it now - before the transition to pad-level
> > operations, maybe it would make sense to mainline this. If, however, you
> > do have to change your output window, maybe you could tell us your
> > use-case, so that we could consider, what's the best way to support that.
> >
> > Thanks
> > Guennadi
> > ---
> > Guennadi Liakhovetski, Ph.D.
> > Freelance Open-Source Software Developer
> > http://www.open-technology.de/
> >
> 
> 
> 
> -- 
> --------------------------------------------------
> Paolo Santinelli
> ImageLab Computer Vision and Pattern Recognition Lab
> Dipartimento di Ingegneria dell'Informazione
> Universita' di Modena e Reggio Emilia
> via Vignolese 905/B, 41125, Modena, Italy
> 
> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> --------------------------------------------------
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi Guennadi,

thank you very much for the patch. I am going to apply it in order to
start toying with the new capability. I think it is a  useful
capability.

I'll let you know.

Again thank you.

Paolo

2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>
>> Hi Guennadi,
>>
>> thank you for the quick answer.
>>
>> Here is what I mean with dynamic: I take "live" one frame at high
>> resolution, for example a picture at VGA or  QVGA resolution, then a
>> sequence of frames that depict a cropped area (200x200 or 100x100)
>> from the original full-resolution frame, and then a new full
>> resolution image (VGA or QVGA) and again the sequence of frames  that
>> depict a cropped area from the original full resolution, and so on.
>> That means takes one frame in 640x480 and  than takes some frames at
>> 100x100 (or 200x200) and so on.
>
> Ic, so, if you can live with a fixed output format and only change the
> input cropping rectangle, the patch set, that I've just sent could give
> you a hint, how this can be done. This would work if you're ok with first
> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
> cropped frames unscaled. But I'm not sure, this is something, that would
> work for you. Otherwise, unless your sensor can upscale cropped images to
> VGA output size, you'll also want fast switching between different output
> sizes, which you'd have to wait for (or implement yourself;-))
>
> Thanks
> Guennadi
>
>>
>> The best would be have two different fixed-output image formats, the
>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
>> pictures obtained cropping the a region of the whole image. The
>> cropping area could be even wider  than 100x100 and then scaled down
>> to the 100x100 ROI format.
>>
>> Probably it is more simple have a cropping area of the same dimension
>> of the ROI format, 100x100.
>>
>> In this way there is a reduction of the computation load of the CPU
>> (smaller images).
>>
>> Thank you very much!
>>
>> Paolo
>>
>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>> >
>> >> Hi all,
>> >>
>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>> >> Image sensor. I am able to use the cropping and scaling capabilities
>> >> V4L2 driver.
>> >> The question is :
>> >>
>> >> Is it possible dynamically change the cropping and scaling values
>> >> without close and re-open  the camera every time ?
>> >>
>> >> Now I am using the streaming I/O memory mapping and to dynamically
>> >> change the cropping and scaling values I do :
>> >>
>> >> 1) stop capturing using VIDIOC_STREAMOFF;
>> >> 2) unmap all the buffers;
>> >> 3) close the device;
>> >> 4) open the device;
>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>> >> the target image width and height, (scaling).
>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>> >>
>> >> this procedure works but take 400 ms.
>> >>
>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>> >>
>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
>> >> camera 0-0: S_FMT denied: queue initialised
>> >> VIDIOC_S_FMT error 16, Device or resource busy
>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>> >>
>> >> Do you have some Idea regarding why I have to close and reopen the
>> >> device and regarding a way to speed up these change?
>> >
>> > Yes, by chance I do;-) First of all you have to make it more precise -
>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
>> > If you want to change output format, that will not be easy ATM, that will
>> > require the snapshot mode API, which is not yet even in an RFC state. If
>> > you only want to change the cropping and keep the output format (zoom),
>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
>> > couple of extensions to the soc-camera core, which I can post tomorrow.
>> > But in fact that is also a hack, because the proper way to implement this
>> > is to port soc-camera to the Media Controller framework and use the
>> > pad-level API. So, I am not sure, whether we want this in the mainline,
>> > but if already two of us need it now - before the transition to pad-level
>> > operations, maybe it would make sense to mainline this. If, however, you
>> > do have to change your output window, maybe you could tell us your
>> > use-case, so that we could consider, what's the best way to support that.
>> >
>> > Thanks
>> > Guennadi
>> > ---
>> > Guennadi Liakhovetski, Ph.D.
>> > Freelance Open-Source Software Developer
>> > http://www.open-technology.de/
>> >
>>
>>
>>
>> --
>> --------------------------------------------------
>> Paolo Santinelli
>> ImageLab Computer Vision and Pattern Recognition Lab
>> Dipartimento di Ingegneria dell'Informazione
>> Universita' di Modena e Reggio Emilia
>> via Vignolese 905/B, 41125, Modena, Italy
>>
>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> --------------------------------------------------
>>
>
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi Guennadi,

Am I wrong or do  I have to add some functions ?

I have hand applied the changes at the soc_camera.c and soc_camera.h
files. At a fist glance to these files seems that I have to add the
function:

 .set_livecrop()

and probably even something more:

  CC      drivers/media/video/soc_camera.o
drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
drivers/media/video/soc_camera.c:545: error: implicit declaration of
function 'vb2_is_streaming'
drivers/media/video/soc_camera.c:545: error: 'struct
soc_camera_device' has no member named 'vb2_vidq'
drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
drivers/media/video/soc_camera.c:799: error: 'struct
soc_camera_device' has no member named 'vb2_vidq'
make[3]: *** [drivers/media/video/soc_camera.o] Error 1

What about vb2_is_streaming and vb2_vidq ?

Any tips regarding these functions ?

Thanks

Paolo
2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
> Hi Guennadi,
>
> thank you very much for the patch. I am going to apply it in order to
> start toying with the new capability. I think it is a  useful
> capability.
>
> I'll let you know.
>
> Again thank you.
>
> Paolo
>
> 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>>
>>> Hi Guennadi,
>>>
>>> thank you for the quick answer.
>>>
>>> Here is what I mean with dynamic: I take "live" one frame at high
>>> resolution, for example a picture at VGA or  QVGA resolution, then a
>>> sequence of frames that depict a cropped area (200x200 or 100x100)
>>> from the original full-resolution frame, and then a new full
>>> resolution image (VGA or QVGA) and again the sequence of frames  that
>>> depict a cropped area from the original full resolution, and so on.
>>> That means takes one frame in 640x480 and  than takes some frames at
>>> 100x100 (or 200x200) and so on.
>>
>> Ic, so, if you can live with a fixed output format and only change the
>> input cropping rectangle, the patch set, that I've just sent could give
>> you a hint, how this can be done. This would work if you're ok with first
>> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
>> cropped frames unscaled. But I'm not sure, this is something, that would
>> work for you. Otherwise, unless your sensor can upscale cropped images to
>> VGA output size, you'll also want fast switching between different output
>> sizes, which you'd have to wait for (or implement yourself;-))
>>
>> Thanks
>> Guennadi
>>
>>>
>>> The best would be have two different fixed-output image formats, the
>>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
>>> pictures obtained cropping the a region of the whole image. The
>>> cropping area could be even wider  than 100x100 and then scaled down
>>> to the 100x100 ROI format.
>>>
>>> Probably it is more simple have a cropping area of the same dimension
>>> of the ROI format, 100x100.
>>>
>>> In this way there is a reduction of the computation load of the CPU
>>> (smaller images).
>>>
>>> Thank you very much!
>>>
>>> Paolo
>>>
>>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>>> >> Image sensor. I am able to use the cropping and scaling capabilities
>>> >> V4L2 driver.
>>> >> The question is :
>>> >>
>>> >> Is it possible dynamically change the cropping and scaling values
>>> >> without close and re-open  the camera every time ?
>>> >>
>>> >> Now I am using the streaming I/O memory mapping and to dynamically
>>> >> change the cropping and scaling values I do :
>>> >>
>>> >> 1) stop capturing using VIDIOC_STREAMOFF;
>>> >> 2) unmap all the buffers;
>>> >> 3) close the device;
>>> >> 4) open the device;
>>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>>> >> the target image width and height, (scaling).
>>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>>> >>
>>> >> this procedure works but take 400 ms.
>>> >>
>>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>>> >>
>>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
>>> >> camera 0-0: S_FMT denied: queue initialised
>>> >> VIDIOC_S_FMT error 16, Device or resource busy
>>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>>> >>
>>> >> Do you have some Idea regarding why I have to close and reopen the
>>> >> device and regarding a way to speed up these change?
>>> >
>>> > Yes, by chance I do;-) First of all you have to make it more precise -
>>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
>>> > If you want to change output format, that will not be easy ATM, that will
>>> > require the snapshot mode API, which is not yet even in an RFC state. If
>>> > you only want to change the cropping and keep the output format (zoom),
>>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
>>> > couple of extensions to the soc-camera core, which I can post tomorrow.
>>> > But in fact that is also a hack, because the proper way to implement this
>>> > is to port soc-camera to the Media Controller framework and use the
>>> > pad-level API. So, I am not sure, whether we want this in the mainline,
>>> > but if already two of us need it now - before the transition to pad-level
>>> > operations, maybe it would make sense to mainline this. If, however, you
>>> > do have to change your output window, maybe you could tell us your
>>> > use-case, so that we could consider, what's the best way to support that.
>>> >
>>> > Thanks
>>> > Guennadi
>>> > ---
>>> > Guennadi Liakhovetski, Ph.D.
>>> > Freelance Open-Source Software Developer
>>> > http://www.open-technology.de/
>>> >
>>>
>>>
>>>
>>> --
>>> --------------------------------------------------
>>> Paolo Santinelli
>>> ImageLab Computer Vision and Pattern Recognition Lab
>>> Dipartimento di Ingegneria dell'Informazione
>>> Universita' di Modena e Reggio Emilia
>>> via Vignolese 905/B, 41125, Modena, Italy
>>>
>>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>>> --------------------------------------------------
>>>
>>
>> ---
>> Guennadi Liakhovetski, Ph.D.
>> Freelance Open-Source Software Developer
>> http://www.open-technology.de/
>>
>
>
>
> --
> --------------------------------------------------
> Paolo Santinelli
> ImageLab Computer Vision and Pattern Recognition Lab
> Dipartimento di Ingegneria dell'Informazione
> Universita' di Modena e Reggio Emilia
> via Vignolese 905/B, 41125, Modena, Italy
>
> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> --------------------------------------------------
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

Re: soc_camera dynamically cropping and scaling

[Guennadi Liakhovetski]

On Wed, 30 Mar 2011, Paolo Santinelli wrote:

> Hi Guennadi,
> 
> Am I wrong or do  I have to add some functions ?
> 
> I have hand applied the changes at the soc_camera.c and soc_camera.h
> files. At a fist glance to these files seems that I have to add the
> function:
> 
>  .set_livecrop()

Yes, I think, I mentioned this in my last mail, that's what my sh-ceu 
example should have illustrated.

> 
> and probably even something more:
> 
>   CC      drivers/media/video/soc_camera.o
> drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
> drivers/media/video/soc_camera.c:545: error: implicit declaration of
> function 'vb2_is_streaming'
> drivers/media/video/soc_camera.c:545: error: 'struct
> soc_camera_device' has no member named 'vb2_vidq'
> drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
> drivers/media/video/soc_camera.c:799: error: 'struct
> soc_camera_device' has no member named 'vb2_vidq'
> make[3]: *** [drivers/media/video/soc_camera.o] Error 1

You have to use current sources. 2.6.39-rc1 should be ok.

Thanks
Guennadi

> 
> What about vb2_is_streaming and vb2_vidq ?
> 
> Any tips regarding these functions ?
> 
> Thanks
> 
> Paolo
> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
> > Hi Guennadi,
> >
> > thank you very much for the patch. I am going to apply it in order to
> > start toying with the new capability. I think it is a  useful
> > capability.
> >
> > I'll let you know.
> >
> > Again thank you.
> >
> > Paolo
> >
> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
> >>
> >>> Hi Guennadi,
> >>>
> >>> thank you for the quick answer.
> >>>
> >>> Here is what I mean with dynamic: I take "live" one frame at high
> >>> resolution, for example a picture at VGA or  QVGA resolution, then a
> >>> sequence of frames that depict a cropped area (200x200 or 100x100)
> >>> from the original full-resolution frame, and then a new full
> >>> resolution image (VGA or QVGA) and again the sequence of frames  that
> >>> depict a cropped area from the original full resolution, and so on.
> >>> That means takes one frame in 640x480 and  than takes some frames at
> >>> 100x100 (or 200x200) and so on.
> >>
> >> Ic, so, if you can live with a fixed output format and only change the
> >> input cropping rectangle, the patch set, that I've just sent could give
> >> you a hint, how this can be done. This would work if you're ok with first
> >> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
> >> cropped frames unscaled. But I'm not sure, this is something, that would
> >> work for you. Otherwise, unless your sensor can upscale cropped images to
> >> VGA output size, you'll also want fast switching between different output
> >> sizes, which you'd have to wait for (or implement yourself;-))
> >>
> >> Thanks
> >> Guennadi
> >>
> >>>
> >>> The best would be have two different fixed-output image formats, the
> >>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
> >>> pictures obtained cropping the a region of the whole image. The
> >>> cropping area could be even wider  than 100x100 and then scaled down
> >>> to the 100x100 ROI format.
> >>>
> >>> Probably it is more simple have a cropping area of the same dimension
> >>> of the ROI format, 100x100.
> >>>
> >>> In this way there is a reduction of the computation load of the CPU
> >>> (smaller images).
> >>>
> >>> Thank you very much!
> >>>
> >>> Paolo
> >>>
> >>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
> >>> >
> >>> >> Hi all,
> >>> >>
> >>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
> >>> >> Image sensor. I am able to use the cropping and scaling capabilities
> >>> >> V4L2 driver.
> >>> >> The question is :
> >>> >>
> >>> >> Is it possible dynamically change the cropping and scaling values
> >>> >> without close and re-open  the camera every time ?
> >>> >>
> >>> >> Now I am using the streaming I/O memory mapping and to dynamically
> >>> >> change the cropping and scaling values I do :
> >>> >>
> >>> >> 1) stop capturing using VIDIOC_STREAMOFF;
> >>> >> 2) unmap all the buffers;
> >>> >> 3) close the device;
> >>> >> 4) open the device;
> >>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
> >>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
> >>> >> the target image width and height, (scaling).
> >>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
> >>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
> >>> >>
> >>> >> this procedure works but take 400 ms.
> >>> >>
> >>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
> >>> >>
> >>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
> >>> >> camera 0-0: S_FMT denied: queue initialised
> >>> >> VIDIOC_S_FMT error 16, Device or resource busy
> >>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
> >>> >>
> >>> >> Do you have some Idea regarding why I have to close and reopen the
> >>> >> device and regarding a way to speed up these change?
> >>> >
> >>> > Yes, by chance I do;-) First of all you have to make it more precise -
> >>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
> >>> > If you want to change output format, that will not be easy ATM, that will
> >>> > require the snapshot mode API, which is not yet even in an RFC state. If
> >>> > you only want to change the cropping and keep the output format (zoom),
> >>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
> >>> > couple of extensions to the soc-camera core, which I can post tomorrow.
> >>> > But in fact that is also a hack, because the proper way to implement this
> >>> > is to port soc-camera to the Media Controller framework and use the
> >>> > pad-level API. So, I am not sure, whether we want this in the mainline,
> >>> > but if already two of us need it now - before the transition to pad-level
> >>> > operations, maybe it would make sense to mainline this. If, however, you
> >>> > do have to change your output window, maybe you could tell us your
> >>> > use-case, so that we could consider, what's the best way to support that.
> >>> >
> >>> > Thanks
> >>> > Guennadi
> >>> > ---
> >>> > Guennadi Liakhovetski, Ph.D.
> >>> > Freelance Open-Source Software Developer
> >>> > http://www.open-technology.de/
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> --------------------------------------------------
> >>> Paolo Santinelli
> >>> ImageLab Computer Vision and Pattern Recognition Lab
> >>> Dipartimento di Ingegneria dell'Informazione
> >>> Universita' di Modena e Reggio Emilia
> >>> via Vignolese 905/B, 41125, Modena, Italy
> >>>
> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> >>> --------------------------------------------------
> >>>
> >>
> >> ---
> >> Guennadi Liakhovetski, Ph.D.
> >> Freelance Open-Source Software Developer
> >> http://www.open-technology.de/
> >>
> >
> >
> >
> > --
> > --------------------------------------------------
> > Paolo Santinelli
> > ImageLab Computer Vision and Pattern Recognition Lab
> > Dipartimento di Ingegneria dell'Informazione
> > Universita' di Modena e Reggio Emilia
> > via Vignolese 905/B, 41125, Modena, Italy
> >
> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> > --------------------------------------------------
> >
> 
> 
> 
> -- 
> --------------------------------------------------
> Paolo Santinelli
> ImageLab Computer Vision and Pattern Recognition Lab
> Dipartimento di Ingegneria dell'Informazione
> Universita' di Modena e Reggio Emilia
> via Vignolese 905/B, 41125, Modena, Italy
> 
> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> --------------------------------------------------
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

OK!

Thanks

Paolo

2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Wed, 30 Mar 2011, Paolo Santinelli wrote:
>
>> Hi Guennadi,
>>
>> Am I wrong or do  I have to add some functions ?
>>
>> I have hand applied the changes at the soc_camera.c and soc_camera.h
>> files. At a fist glance to these files seems that I have to add the
>> function:
>>
>>  .set_livecrop()
>
> Yes, I think, I mentioned this in my last mail, that's what my sh-ceu
> example should have illustrated.
>
>>
>> and probably even something more:
>>
>>   CC      drivers/media/video/soc_camera.o
>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
>> drivers/media/video/soc_camera.c:545: error: implicit declaration of
>> function 'vb2_is_streaming'
>> drivers/media/video/soc_camera.c:545: error: 'struct
>> soc_camera_device' has no member named 'vb2_vidq'
>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
>> drivers/media/video/soc_camera.c:799: error: 'struct
>> soc_camera_device' has no member named 'vb2_vidq'
>> make[3]: *** [drivers/media/video/soc_camera.o] Error 1
>
> You have to use current sources. 2.6.39-rc1 should be ok.
>
> Thanks
> Guennadi
>
>>
>> What about vb2_is_streaming and vb2_vidq ?
>>
>> Any tips regarding these functions ?
>>
>> Thanks
>>
>> Paolo
>> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
>> > Hi Guennadi,
>> >
>> > thank you very much for the patch. I am going to apply it in order to
>> > start toying with the new capability. I think it is a  useful
>> > capability.
>> >
>> > I'll let you know.
>> >
>> > Again thank you.
>> >
>> > Paolo
>> >
>> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>> >>
>> >>> Hi Guennadi,
>> >>>
>> >>> thank you for the quick answer.
>> >>>
>> >>> Here is what I mean with dynamic: I take "live" one frame at high
>> >>> resolution, for example a picture at VGA or  QVGA resolution, then a
>> >>> sequence of frames that depict a cropped area (200x200 or 100x100)
>> >>> from the original full-resolution frame, and then a new full
>> >>> resolution image (VGA or QVGA) and again the sequence of frames  that
>> >>> depict a cropped area from the original full resolution, and so on.
>> >>> That means takes one frame in 640x480 and  than takes some frames at
>> >>> 100x100 (or 200x200) and so on.
>> >>
>> >> Ic, so, if you can live with a fixed output format and only change the
>> >> input cropping rectangle, the patch set, that I've just sent could give
>> >> you a hint, how this can be done. This would work if you're ok with first
>> >> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
>> >> cropped frames unscaled. But I'm not sure, this is something, that would
>> >> work for you. Otherwise, unless your sensor can upscale cropped images to
>> >> VGA output size, you'll also want fast switching between different output
>> >> sizes, which you'd have to wait for (or implement yourself;-))
>> >>
>> >> Thanks
>> >> Guennadi
>> >>
>> >>>
>> >>> The best would be have two different fixed-output image formats, the
>> >>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
>> >>> pictures obtained cropping the a region of the whole image. The
>> >>> cropping area could be even wider  than 100x100 and then scaled down
>> >>> to the 100x100 ROI format.
>> >>>
>> >>> Probably it is more simple have a cropping area of the same dimension
>> >>> of the ROI format, 100x100.
>> >>>
>> >>> In this way there is a reduction of the computation load of the CPU
>> >>> (smaller images).
>> >>>
>> >>> Thank you very much!
>> >>>
>> >>> Paolo
>> >>>
>> >>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>> >>> >
>> >>> >> Hi all,
>> >>> >>
>> >>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>> >>> >> Image sensor. I am able to use the cropping and scaling capabilities
>> >>> >> V4L2 driver.
>> >>> >> The question is :
>> >>> >>
>> >>> >> Is it possible dynamically change the cropping and scaling values
>> >>> >> without close and re-open  the camera every time ?
>> >>> >>
>> >>> >> Now I am using the streaming I/O memory mapping and to dynamically
>> >>> >> change the cropping and scaling values I do :
>> >>> >>
>> >>> >> 1) stop capturing using VIDIOC_STREAMOFF;
>> >>> >> 2) unmap all the buffers;
>> >>> >> 3) close the device;
>> >>> >> 4) open the device;
>> >>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>> >>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>> >>> >> the target image width and height, (scaling).
>> >>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>> >>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>> >>> >>
>> >>> >> this procedure works but take 400 ms.
>> >>> >>
>> >>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>> >>> >>
>> >>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
>> >>> >> camera 0-0: S_FMT denied: queue initialised
>> >>> >> VIDIOC_S_FMT error 16, Device or resource busy
>> >>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>> >>> >>
>> >>> >> Do you have some Idea regarding why I have to close and reopen the
>> >>> >> device and regarding a way to speed up these change?
>> >>> >
>> >>> > Yes, by chance I do;-) First of all you have to make it more precise -
>> >>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
>> >>> > If you want to change output format, that will not be easy ATM, that will
>> >>> > require the snapshot mode API, which is not yet even in an RFC state. If
>> >>> > you only want to change the cropping and keep the output format (zoom),
>> >>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
>> >>> > couple of extensions to the soc-camera core, which I can post tomorrow.
>> >>> > But in fact that is also a hack, because the proper way to implement this
>> >>> > is to port soc-camera to the Media Controller framework and use the
>> >>> > pad-level API. So, I am not sure, whether we want this in the mainline,
>> >>> > but if already two of us need it now - before the transition to pad-level
>> >>> > operations, maybe it would make sense to mainline this. If, however, you
>> >>> > do have to change your output window, maybe you could tell us your
>> >>> > use-case, so that we could consider, what's the best way to support that.
>> >>> >
>> >>> > Thanks
>> >>> > Guennadi
>> >>> > ---
>> >>> > Guennadi Liakhovetski, Ph.D.
>> >>> > Freelance Open-Source Software Developer
>> >>> > http://www.open-technology.de/
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> --------------------------------------------------
>> >>> Paolo Santinelli
>> >>> ImageLab Computer Vision and Pattern Recognition Lab
>> >>> Dipartimento di Ingegneria dell'Informazione
>> >>> Universita' di Modena e Reggio Emilia
>> >>> via Vignolese 905/B, 41125, Modena, Italy
>> >>>
>> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> >>> --------------------------------------------------
>> >>>
>> >>
>> >> ---
>> >> Guennadi Liakhovetski, Ph.D.
>> >> Freelance Open-Source Software Developer
>> >> http://www.open-technology.de/
>> >>
>> >
>> >
>> >
>> > --
>> > --------------------------------------------------
>> > Paolo Santinelli
>> > ImageLab Computer Vision and Pattern Recognition Lab
>> > Dipartimento di Ingegneria dell'Informazione
>> > Universita' di Modena e Reggio Emilia
>> > via Vignolese 905/B, 41125, Modena, Italy
>> >
>> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> > --------------------------------------------------
>> >
>>
>>
>>
>> --
>> --------------------------------------------------
>> Paolo Santinelli
>> ImageLab Computer Vision and Pattern Recognition Lab
>> Dipartimento di Ingegneria dell'Informazione
>> Universita' di Modena e Reggio Emilia
>> via Vignolese 905/B, 41125, Modena, Italy
>>
>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> --------------------------------------------------
>>
>
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi Guennadi,

I have tried to implement the changes you suggested me in order to get
the new live cropping capabilities. I am able to successfully compile
the patched driver even though I still have problems  when an
application program tries to do live cropping (calling run time the
same ."ioctl VIDIOC_S_CROP" but with different cropping parameters).

This is the run time error I get when  I call the "ioctl
VIDIOC_S_CROP" in order to change live the  cropping area:

ov9655 0-0030: Scaled for 320x240 (320x240) : 0x0
ov9655 0-0030: Adjusted for 320x240 : hstart 240, hstop 560 = 320,
vstart 11, vstop 252 = 241
pxa27x-camera pxa27x-camera.0: Output after crop: 320x240

pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
select timeout



Here is what I have done:

1) I have used the kernel  linux-2.6.39-rc1;
2) I have carefully  added the patch at the  soc_camera.c and
soc_camera.h files as you indicated in your mail "[PATCH 1/2] V4L:
soc-camera: add a livecrop host operation";
4) I have changed pxa_camera.c file trying to put the patch as you
have indicated for the sh_mobile_ceu_camera.c, that is:

-I have changed the "pxa_camera_dev" struct  instead of
"sh_mobile_ceu_dev" adding the elements "struct completion complete"
and "unsigned int frozen:1;"

-I have added the .set_livecrop() method. Inside this method I  have
replaced the  prefix "sh_mobile_ceu_"  with "pxa_camera_". This method
calls the function "sh_mobile_ceu_capture()" but doesn't exist an
equivalent function named "pxa_camera_capture()", instead there are
the two functions "pxa_camera_start_capture()" and
"pxa_camera_stop_capture()". So I call "pxa_camera_start_capture()"
instead of "sh_mobile_ceu_capture()", but I am not sure this is the
right solution;

-I have changed the  "pxa_camera_start_capture()" function instead of
"sh_mobile_ceu_capture()" adding at the end "if (pcdev->frozen)
complete(&pcdev->complete);" even if I'm afraid that is not good.

-Of course I have  added the line ".set_livecrop	=
sh_mobile_ceu_set_livecrop," at the structure  "static struct
soc_camera_host_ops pxa_soc_camera_host_ops" instead of  "static
struct soc_camera_host_ops sh_mobile_ceu_host_ops".

-I didn't change anything else.

How to do the live cropping from the application program ? Do I have
to invoke the usual "ioctl VIDIOC_S_CROP" or I have to invoke the
".livecrop()" method adding a new "ioctl VIDIOC_S_LIVECROP" ?


Thank you very much.

Paolo



2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
> OK!
>
> Thanks
>
> Paolo
>
> 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> On Wed, 30 Mar 2011, Paolo Santinelli wrote:
>>
>>> Hi Guennadi,
>>>
>>> Am I wrong or do  I have to add some functions ?
>>>
>>> I have hand applied the changes at the soc_camera.c and soc_camera.h
>>> files. At a fist glance to these files seems that I have to add the
>>> function:
>>>
>>>  .set_livecrop()
>>
>> Yes, I think, I mentioned this in my last mail, that's what my sh-ceu
>> example should have illustrated.
>>
>>>
>>> and probably even something more:
>>>
>>>   CC      drivers/media/video/soc_camera.o
>>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
>>> drivers/media/video/soc_camera.c:545: error: implicit declaration of
>>> function 'vb2_is_streaming'
>>> drivers/media/video/soc_camera.c:545: error: 'struct
>>> soc_camera_device' has no member named 'vb2_vidq'
>>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
>>> drivers/media/video/soc_camera.c:799: error: 'struct
>>> soc_camera_device' has no member named 'vb2_vidq'
>>> make[3]: *** [drivers/media/video/soc_camera.o] Error 1
>>
>> You have to use current sources. 2.6.39-rc1 should be ok.
>>
>> Thanks
>> Guennadi
>>
>>>
>>> What about vb2_is_streaming and vb2_vidq ?
>>>
>>> Any tips regarding these functions ?
>>>
>>> Thanks
>>>
>>> Paolo
>>> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
>>> > Hi Guennadi,
>>> >
>>> > thank you very much for the patch. I am going to apply it in order to
>>> > start toying with the new capability. I think it is a  useful
>>> > capability.
>>> >
>>> > I'll let you know.
>>> >
>>> > Again thank you.
>>> >
>>> > Paolo
>>> >
>>> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>>> >> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>>> >>
>>> >>> Hi Guennadi,
>>> >>>
>>> >>> thank you for the quick answer.
>>> >>>
>>> >>> Here is what I mean with dynamic: I take "live" one frame at high
>>> >>> resolution, for example a picture at VGA or  QVGA resolution, then a
>>> >>> sequence of frames that depict a cropped area (200x200 or 100x100)
>>> >>> from the original full-resolution frame, and then a new full
>>> >>> resolution image (VGA or QVGA) and again the sequence of frames  that
>>> >>> depict a cropped area from the original full resolution, and so on.
>>> >>> That means takes one frame in 640x480 and  than takes some frames at
>>> >>> 100x100 (or 200x200) and so on.
>>> >>
>>> >> Ic, so, if you can live with a fixed output format and only change the
>>> >> input cropping rectangle, the patch set, that I've just sent could give
>>> >> you a hint, how this can be done. This would work if you're ok with first
>>> >> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
>>> >> cropped frames unscaled. But I'm not sure, this is something, that would
>>> >> work for you. Otherwise, unless your sensor can upscale cropped images to
>>> >> VGA output size, you'll also want fast switching between different output
>>> >> sizes, which you'd have to wait for (or implement yourself;-))
>>> >>
>>> >> Thanks
>>> >> Guennadi
>>> >>
>>> >>>
>>> >>> The best would be have two different fixed-output image formats, the
>>> >>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
>>> >>> pictures obtained cropping the a region of the whole image. The
>>> >>> cropping area could be even wider  than 100x100 and then scaled down
>>> >>> to the 100x100 ROI format.
>>> >>>
>>> >>> Probably it is more simple have a cropping area of the same dimension
>>> >>> of the ROI format, 100x100.
>>> >>>
>>> >>> In this way there is a reduction of the computation load of the CPU
>>> >>> (smaller images).
>>> >>>
>>> >>> Thank you very much!
>>> >>>
>>> >>> Paolo
>>> >>>
>>> >>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>>> >>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>>> >>> >
>>> >>> >> Hi all,
>>> >>> >>
>>> >>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>>> >>> >> Image sensor. I am able to use the cropping and scaling capabilities
>>> >>> >> V4L2 driver.
>>> >>> >> The question is :
>>> >>> >>
>>> >>> >> Is it possible dynamically change the cropping and scaling values
>>> >>> >> without close and re-open  the camera every time ?
>>> >>> >>
>>> >>> >> Now I am using the streaming I/O memory mapping and to dynamically
>>> >>> >> change the cropping and scaling values I do :
>>> >>> >>
>>> >>> >> 1) stop capturing using VIDIOC_STREAMOFF;
>>> >>> >> 2) unmap all the buffers;
>>> >>> >> 3) close the device;
>>> >>> >> 4) open the device;
>>> >>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>>> >>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>>> >>> >> the target image width and height, (scaling).
>>> >>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>>> >>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>>> >>> >>
>>> >>> >> this procedure works but take 400 ms.
>>> >>> >>
>>> >>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>>> >>> >>
>>> >>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
>>> >>> >> camera 0-0: S_FMT denied: queue initialised
>>> >>> >> VIDIOC_S_FMT error 16, Device or resource busy
>>> >>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>>> >>> >>
>>> >>> >> Do you have some Idea regarding why I have to close and reopen the
>>> >>> >> device and regarding a way to speed up these change?
>>> >>> >
>>> >>> > Yes, by chance I do;-) First of all you have to make it more precise -
>>> >>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
>>> >>> > If you want to change output format, that will not be easy ATM, that will
>>> >>> > require the snapshot mode API, which is not yet even in an RFC state. If
>>> >>> > you only want to change the cropping and keep the output format (zoom),
>>> >>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
>>> >>> > couple of extensions to the soc-camera core, which I can post tomorrow.
>>> >>> > But in fact that is also a hack, because the proper way to implement this
>>> >>> > is to port soc-camera to the Media Controller framework and use the
>>> >>> > pad-level API. So, I am not sure, whether we want this in the mainline,
>>> >>> > but if already two of us need it now - before the transition to pad-level
>>> >>> > operations, maybe it would make sense to mainline this. If, however, you
>>> >>> > do have to change your output window, maybe you could tell us your
>>> >>> > use-case, so that we could consider, what's the best way to support that.
>>> >>> >
>>> >>> > Thanks
>>> >>> > Guennadi
>>> >>> > ---
>>> >>> > Guennadi Liakhovetski, Ph.D.
>>> >>> > Freelance Open-Source Software Developer
>>> >>> > http://www.open-technology.de/
>>> >>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> --------------------------------------------------
>>> >>> Paolo Santinelli
>>> >>> ImageLab Computer Vision and Pattern Recognition Lab
>>> >>> Dipartimento di Ingegneria dell'Informazione
>>> >>> Universita' di Modena e Reggio Emilia
>>> >>> via Vignolese 905/B, 41125, Modena, Italy
>>> >>>
>>> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>>> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>>> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>>> >>> --------------------------------------------------
>>> >>>
>>> >>
>>> >> ---
>>> >> Guennadi Liakhovetski, Ph.D.
>>> >> Freelance Open-Source Software Developer
>>> >> http://www.open-technology.de/
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > --------------------------------------------------
>>> > Paolo Santinelli
>>> > ImageLab Computer Vision and Pattern Recognition Lab
>>> > Dipartimento di Ingegneria dell'Informazione
>>> > Universita' di Modena e Reggio Emilia
>>> > via Vignolese 905/B, 41125, Modena, Italy
>>> >
>>> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>>> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>>> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>>> > --------------------------------------------------
>>> >
>>>
>>>
>>>
>>> --
>>> --------------------------------------------------
>>> Paolo Santinelli
>>> ImageLab Computer Vision and Pattern Recognition Lab
>>> Dipartimento di Ingegneria dell'Informazione
>>> Universita' di Modena e Reggio Emilia
>>> via Vignolese 905/B, 41125, Modena, Italy
>>>
>>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>>> --------------------------------------------------
>>>
>>
>> ---
>> Guennadi Liakhovetski, Ph.D.
>> Freelance Open-Source Software Developer
>> http://www.open-technology.de/
>>
>
>
>
> --
> --------------------------------------------------
> Paolo Santinelli
> ImageLab Computer Vision and Pattern Recognition Lab
> Dipartimento di Ingegneria dell'Informazione
> Universita' di Modena e Reggio Emilia
> via Vignolese 905/B, 41125, Modena, Italy
>
> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> --------------------------------------------------
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------

Re: soc_camera dynamically cropping and scaling

[Guennadi Liakhovetski]

On Mon, 4 Apr 2011, Paolo Santinelli wrote:

> Hi Guennadi,
> 
> I have tried to implement the changes you suggested me in order to get
> the new live cropping capabilities. I am able to successfully compile
> the patched driver even though I still have problems  when an
> application program tries to do live cropping (calling run time the
> same ."ioctl VIDIOC_S_CROP" but with different cropping parameters).
> 
> This is the run time error I get when  I call the "ioctl
> VIDIOC_S_CROP" in order to change live the  cropping area:

Yes, this is correct, my patches don't add any new ioctl()s.

> ov9655 0-0030: Scaled for 320x240 (320x240) : 0x0
> ov9655 0-0030: Adjusted for 320x240 : hstart 240, hstop 560 = 320,
> vstart 11, vstop 252 = 241
> pxa27x-camera pxa27x-camera.0: Output after crop: 320x240
> 
> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
> select timeout

Well, you certainly realise, that sh-mobile and PXA270 are two absolutely 
different architectures, using different IPs for there video capture 
interfaces and DMA engines. What I have provided to you is a generic patch 
for soc-camera, which you have to take as is, which you've also done. But 
the sh-mobile part was just an example, which you could only take as a 
basis for your PXA work. But for PXA you would actually have to _develop_ 
(test and debug) your own implementation, which may well end up pretty 
different from the sh-mobile version. PXA270 doesn't implement any video 
scaling, and the driver currently doesn't implement any host-side 
cropping. So, you have to make sure, that your sensor produces exactly the 
same size output in both your cropped modes. And then just go wild and 
experiment with the driver until you get the desired result:-)

Good luck
Guennadi

> Here is what I have done:
> 
> 1) I have used the kernel  linux-2.6.39-rc1;
> 2) I have carefully  added the patch at the  soc_camera.c and
> soc_camera.h files as you indicated in your mail "[PATCH 1/2] V4L:
> soc-camera: add a livecrop host operation";
> 4) I have changed pxa_camera.c file trying to put the patch as you
> have indicated for the sh_mobile_ceu_camera.c, that is:
> 
> -I have changed the "pxa_camera_dev" struct  instead of
> "sh_mobile_ceu_dev" adding the elements "struct completion complete"
> and "unsigned int frozen:1;"
> 
> -I have added the .set_livecrop() method. Inside this method I  have
> replaced the  prefix "sh_mobile_ceu_"  with "pxa_camera_". This method
> calls the function "sh_mobile_ceu_capture()" but doesn't exist an
> equivalent function named "pxa_camera_capture()", instead there are
> the two functions "pxa_camera_start_capture()" and
> "pxa_camera_stop_capture()". So I call "pxa_camera_start_capture()"
> instead of "sh_mobile_ceu_capture()", but I am not sure this is the
> right solution;
> 
> -I have changed the  "pxa_camera_start_capture()" function instead of
> "sh_mobile_ceu_capture()" adding at the end "if (pcdev->frozen)
> complete(&pcdev->complete);" even if I'm afraid that is not good.
> 
> -Of course I have  added the line ".set_livecrop	=
> sh_mobile_ceu_set_livecrop," at the structure  "static struct
> soc_camera_host_ops pxa_soc_camera_host_ops" instead of  "static
> struct soc_camera_host_ops sh_mobile_ceu_host_ops".
> 
> -I didn't change anything else.
> 
> How to do the live cropping from the application program ? Do I have
> to invoke the usual "ioctl VIDIOC_S_CROP" or I have to invoke the
> ".livecrop()" method adding a new "ioctl VIDIOC_S_LIVECROP" ?
> 
> 
> Thank you very much.
> 
> Paolo
> 
> 
> 
> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
> > OK!
> >
> > Thanks
> >
> > Paolo
> >
> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >> On Wed, 30 Mar 2011, Paolo Santinelli wrote:
> >>
> >>> Hi Guennadi,
> >>>
> >>> Am I wrong or do  I have to add some functions ?
> >>>
> >>> I have hand applied the changes at the soc_camera.c and soc_camera.h
> >>> files. At a fist glance to these files seems that I have to add the
> >>> function:
> >>>
> >>>  .set_livecrop()
> >>
> >> Yes, I think, I mentioned this in my last mail, that's what my sh-ceu
> >> example should have illustrated.
> >>
> >>>
> >>> and probably even something more:
> >>>
> >>>   CC      drivers/media/video/soc_camera.o
> >>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
> >>> drivers/media/video/soc_camera.c:545: error: implicit declaration of
> >>> function 'vb2_is_streaming'
> >>> drivers/media/video/soc_camera.c:545: error: 'struct
> >>> soc_camera_device' has no member named 'vb2_vidq'
> >>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
> >>> drivers/media/video/soc_camera.c:799: error: 'struct
> >>> soc_camera_device' has no member named 'vb2_vidq'
> >>> make[3]: *** [drivers/media/video/soc_camera.o] Error 1
> >>
> >> You have to use current sources. 2.6.39-rc1 should be ok.
> >>
> >> Thanks
> >> Guennadi
> >>
> >>>
> >>> What about vb2_is_streaming and vb2_vidq ?
> >>>
> >>> Any tips regarding these functions ?
> >>>
> >>> Thanks
> >>>
> >>> Paolo
> >>> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
> >>> > Hi Guennadi,
> >>> >
> >>> > thank you very much for the patch. I am going to apply it in order to
> >>> > start toying with the new capability. I think it is a  useful
> >>> > capability.
> >>> >
> >>> > I'll let you know.
> >>> >
> >>> > Again thank you.
> >>> >
> >>> > Paolo
> >>> >
> >>> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >>> >> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
> >>> >>
> >>> >>> Hi Guennadi,
> >>> >>>
> >>> >>> thank you for the quick answer.
> >>> >>>
> >>> >>> Here is what I mean with dynamic: I take "live" one frame at high
> >>> >>> resolution, for example a picture at VGA or  QVGA resolution, then a
> >>> >>> sequence of frames that depict a cropped area (200x200 or 100x100)
> >>> >>> from the original full-resolution frame, and then a new full
> >>> >>> resolution image (VGA or QVGA) and again the sequence of frames  that
> >>> >>> depict a cropped area from the original full resolution, and so on.
> >>> >>> That means takes one frame in 640x480 and  than takes some frames at
> >>> >>> 100x100 (or 200x200) and so on.
> >>> >>
> >>> >> Ic, so, if you can live with a fixed output format and only change the
> >>> >> input cropping rectangle, the patch set, that I've just sent could give
> >>> >> you a hint, how this can be done. This would work if you're ok with first
> >>> >> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
> >>> >> cropped frames unscaled. But I'm not sure, this is something, that would
> >>> >> work for you. Otherwise, unless your sensor can upscale cropped images to
> >>> >> VGA output size, you'll also want fast switching between different output
> >>> >> sizes, which you'd have to wait for (or implement yourself;-))
> >>> >>
> >>> >> Thanks
> >>> >> Guennadi
> >>> >>
> >>> >>>
> >>> >>> The best would be have two different fixed-output image formats, the
> >>> >>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
> >>> >>> pictures obtained cropping the a region of the whole image. The
> >>> >>> cropping area could be even wider  than 100x100 and then scaled down
> >>> >>> to the 100x100 ROI format.
> >>> >>>
> >>> >>> Probably it is more simple have a cropping area of the same dimension
> >>> >>> of the ROI format, 100x100.
> >>> >>>
> >>> >>> In this way there is a reduction of the computation load of the CPU
> >>> >>> (smaller images).
> >>> >>>
> >>> >>> Thank you very much!
> >>> >>>
> >>> >>> Paolo
> >>> >>>
> >>> >>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >>> >>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
> >>> >>> >
> >>> >>> >> Hi all,
> >>> >>> >>
> >>> >>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
> >>> >>> >> Image sensor. I am able to use the cropping and scaling capabilities
> >>> >>> >> V4L2 driver.
> >>> >>> >> The question is :
> >>> >>> >>
> >>> >>> >> Is it possible dynamically change the cropping and scaling values
> >>> >>> >> without close and re-open  the camera every time ?
> >>> >>> >>
> >>> >>> >> Now I am using the streaming I/O memory mapping and to dynamically
> >>> >>> >> change the cropping and scaling values I do :
> >>> >>> >>
> >>> >>> >> 1) stop capturing using VIDIOC_STREAMOFF;
> >>> >>> >> 2) unmap all the buffers;
> >>> >>> >> 3) close the device;
> >>> >>> >> 4) open the device;
> >>> >>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
> >>> >>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
> >>> >>> >> the target image width and height, (scaling).
> >>> >>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
> >>> >>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
> >>> >>> >>
> >>> >>> >> this procedure works but take 400 ms.
> >>> >>> >>
> >>> >>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
> >>> >>> >>
> >>> >>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
> >>> >>> >> camera 0-0: S_FMT denied: queue initialised
> >>> >>> >> VIDIOC_S_FMT error 16, Device or resource busy
> >>> >>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
> >>> >>> >>
> >>> >>> >> Do you have some Idea regarding why I have to close and reopen the
> >>> >>> >> device and regarding a way to speed up these change?
> >>> >>> >
> >>> >>> > Yes, by chance I do;-) First of all you have to make it more precise -
> >>> >>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
> >>> >>> > If you want to change output format, that will not be easy ATM, that will
> >>> >>> > require the snapshot mode API, which is not yet even in an RFC state. If
> >>> >>> > you only want to change the cropping and keep the output format (zoom),
> >>> >>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
> >>> >>> > couple of extensions to the soc-camera core, which I can post tomorrow.
> >>> >>> > But in fact that is also a hack, because the proper way to implement this
> >>> >>> > is to port soc-camera to the Media Controller framework and use the
> >>> >>> > pad-level API. So, I am not sure, whether we want this in the mainline,
> >>> >>> > but if already two of us need it now - before the transition to pad-level
> >>> >>> > operations, maybe it would make sense to mainline this. If, however, you
> >>> >>> > do have to change your output window, maybe you could tell us your
> >>> >>> > use-case, so that we could consider, what's the best way to support that.
> >>> >>> >
> >>> >>> > Thanks
> >>> >>> > Guennadi
> >>> >>> > ---
> >>> >>> > Guennadi Liakhovetski, Ph.D.
> >>> >>> > Freelance Open-Source Software Developer
> >>> >>> > http://www.open-technology.de/
> >>> >>> >
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> --------------------------------------------------
> >>> >>> Paolo Santinelli
> >>> >>> ImageLab Computer Vision and Pattern Recognition Lab
> >>> >>> Dipartimento di Ingegneria dell'Informazione
> >>> >>> Universita' di Modena e Reggio Emilia
> >>> >>> via Vignolese 905/B, 41125, Modena, Italy
> >>> >>>
> >>> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> >>> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> >>> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> >>> >>> --------------------------------------------------
> >>> >>>
> >>> >>
> >>> >> ---
> >>> >> Guennadi Liakhovetski, Ph.D.
> >>> >> Freelance Open-Source Software Developer
> >>> >> http://www.open-technology.de/
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > --------------------------------------------------
> >>> > Paolo Santinelli
> >>> > ImageLab Computer Vision and Pattern Recognition Lab
> >>> > Dipartimento di Ingegneria dell'Informazione
> >>> > Universita' di Modena e Reggio Emilia
> >>> > via Vignolese 905/B, 41125, Modena, Italy
> >>> >
> >>> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> >>> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> >>> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> >>> > --------------------------------------------------
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> --------------------------------------------------
> >>> Paolo Santinelli
> >>> ImageLab Computer Vision and Pattern Recognition Lab
> >>> Dipartimento di Ingegneria dell'Informazione
> >>> Universita' di Modena e Reggio Emilia
> >>> via Vignolese 905/B, 41125, Modena, Italy
> >>>
> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> >>> --------------------------------------------------
> >>>
> >>
> >> ---
> >> Guennadi Liakhovetski, Ph.D.
> >> Freelance Open-Source Software Developer
> >> http://www.open-technology.de/
> >>
> >
> >
> >
> > --
> > --------------------------------------------------
> > Paolo Santinelli
> > ImageLab Computer Vision and Pattern Recognition Lab
> > Dipartimento di Ingegneria dell'Informazione
> > Universita' di Modena e Reggio Emilia
> > via Vignolese 905/B, 41125, Modena, Italy
> >
> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> > --------------------------------------------------
> >
> 
> 
> 
> -- 
> --------------------------------------------------
> Paolo Santinelli
> ImageLab Computer Vision and Pattern Recognition Lab
> Dipartimento di Ingegneria dell'Informazione
> Universita' di Modena e Reggio Emilia
> via Vignolese 905/B, 41125, Modena, Italy
> 
> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
> --------------------------------------------------
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: soc_camera dynamically cropping and scaling

[Paolo Santinelli]

Hi Guennadi,

thank you for the useful explanation. I will try!

Paolo.

2011/4/5 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Mon, 4 Apr 2011, Paolo Santinelli wrote:
>
>> Hi Guennadi,
>>
>> I have tried to implement the changes you suggested me in order to get
>> the new live cropping capabilities. I am able to successfully compile
>> the patched driver even though I still have problems  when an
>> application program tries to do live cropping (calling run time the
>> same ."ioctl VIDIOC_S_CROP" but with different cropping parameters).
>>
>> This is the run time error I get when  I call the "ioctl
>> VIDIOC_S_CROP" in order to change live the  cropping area:
>
> Yes, this is correct, my patches don't add any new ioctl()s.
>
>> ov9655 0-0030: Scaled for 320x240 (320x240) : 0x0
>> ov9655 0-0030: Adjusted for 320x240 : hstart 240, hstop 560 = 320,
>> vstart 11, vstop 252 = 241
>> pxa27x-camera pxa27x-camera.0: Output after crop: 320x240
>>
>> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
>> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
>> pxa27x-camera pxa27x-camera.0: DMA Bus Error IRQ!
>> select timeout
>
> Well, you certainly realise, that sh-mobile and PXA270 are two absolutely
> different architectures, using different IPs for there video capture
> interfaces and DMA engines. What I have provided to you is a generic patch
> for soc-camera, which you have to take as is, which you've also done. But
> the sh-mobile part was just an example, which you could only take as a
> basis for your PXA work. But for PXA you would actually have to _develop_
> (test and debug) your own implementation, which may well end up pretty
> different from the sh-mobile version. PXA270 doesn't implement any video
> scaling, and the driver currently doesn't implement any host-side
> cropping. So, you have to make sure, that your sensor produces exactly the
> same size output in both your cropped modes. And then just go wild and
> experiment with the driver until you get the desired result:-)
>
> Good luck
> Guennadi
>
>> Here is what I have done:
>>
>> 1) I have used the kernel  linux-2.6.39-rc1;
>> 2) I have carefully  added the patch at the  soc_camera.c and
>> soc_camera.h files as you indicated in your mail "[PATCH 1/2] V4L:
>> soc-camera: add a livecrop host operation";
>> 4) I have changed pxa_camera.c file trying to put the patch as you
>> have indicated for the sh_mobile_ceu_camera.c, that is:
>>
>> -I have changed the "pxa_camera_dev" struct  instead of
>> "sh_mobile_ceu_dev" adding the elements "struct completion complete"
>> and "unsigned int frozen:1;"
>>
>> -I have added the .set_livecrop() method. Inside this method I  have
>> replaced the  prefix "sh_mobile_ceu_"  with "pxa_camera_". This method
>> calls the function "sh_mobile_ceu_capture()" but doesn't exist an
>> equivalent function named "pxa_camera_capture()", instead there are
>> the two functions "pxa_camera_start_capture()" and
>> "pxa_camera_stop_capture()". So I call "pxa_camera_start_capture()"
>> instead of "sh_mobile_ceu_capture()", but I am not sure this is the
>> right solution;
>>
>> -I have changed the  "pxa_camera_start_capture()" function instead of
>> "sh_mobile_ceu_capture()" adding at the end "if (pcdev->frozen)
>> complete(&pcdev->complete);" even if I'm afraid that is not good.
>>
>> -Of course I have  added the line ".set_livecrop      =
>> sh_mobile_ceu_set_livecrop," at the structure  "static struct
>> soc_camera_host_ops pxa_soc_camera_host_ops" instead of  "static
>> struct soc_camera_host_ops sh_mobile_ceu_host_ops".
>>
>> -I didn't change anything else.
>>
>> How to do the live cropping from the application program ? Do I have
>> to invoke the usual "ioctl VIDIOC_S_CROP" or I have to invoke the
>> ".livecrop()" method adding a new "ioctl VIDIOC_S_LIVECROP" ?
>>
>>
>> Thank you very much.
>>
>> Paolo
>>
>>
>>
>> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
>> > OK!
>> >
>> > Thanks
>> >
>> > Paolo
>> >
>> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >> On Wed, 30 Mar 2011, Paolo Santinelli wrote:
>> >>
>> >>> Hi Guennadi,
>> >>>
>> >>> Am I wrong or do  I have to add some functions ?
>> >>>
>> >>> I have hand applied the changes at the soc_camera.c and soc_camera.h
>> >>> files. At a fist glance to these files seems that I have to add the
>> >>> function:
>> >>>
>> >>>  .set_livecrop()
>> >>
>> >> Yes, I think, I mentioned this in my last mail, that's what my sh-ceu
>> >> example should have illustrated.
>> >>
>> >>>
>> >>> and probably even something more:
>> >>>
>> >>>   CC      drivers/media/video/soc_camera.o
>> >>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_fmt_vid_cap':
>> >>> drivers/media/video/soc_camera.c:545: error: implicit declaration of
>> >>> function 'vb2_is_streaming'
>> >>> drivers/media/video/soc_camera.c:545: error: 'struct
>> >>> soc_camera_device' has no member named 'vb2_vidq'
>> >>> drivers/media/video/soc_camera.c: In function 'soc_camera_s_crop':
>> >>> drivers/media/video/soc_camera.c:799: error: 'struct
>> >>> soc_camera_device' has no member named 'vb2_vidq'
>> >>> make[3]: *** [drivers/media/video/soc_camera.o] Error 1
>> >>
>> >> You have to use current sources. 2.6.39-rc1 should be ok.
>> >>
>> >> Thanks
>> >> Guennadi
>> >>
>> >>>
>> >>> What about vb2_is_streaming and vb2_vidq ?
>> >>>
>> >>> Any tips regarding these functions ?
>> >>>
>> >>> Thanks
>> >>>
>> >>> Paolo
>> >>> 2011/3/30 Paolo Santinelli <paolo.santinelli@unimore.it>:
>> >>> > Hi Guennadi,
>> >>> >
>> >>> > thank you very much for the patch. I am going to apply it in order to
>> >>> > start toying with the new capability. I think it is a  useful
>> >>> > capability.
>> >>> >
>> >>> > I'll let you know.
>> >>> >
>> >>> > Again thank you.
>> >>> >
>> >>> > Paolo
>> >>> >
>> >>> > 2011/3/30 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >>> >> On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>> >>> >>
>> >>> >>> Hi Guennadi,
>> >>> >>>
>> >>> >>> thank you for the quick answer.
>> >>> >>>
>> >>> >>> Here is what I mean with dynamic: I take "live" one frame at high
>> >>> >>> resolution, for example a picture at VGA or  QVGA resolution, then a
>> >>> >>> sequence of frames that depict a cropped area (200x200 or 100x100)
>> >>> >>> from the original full-resolution frame, and then a new full
>> >>> >>> resolution image (VGA or QVGA) and again the sequence of frames  that
>> >>> >>> depict a cropped area from the original full resolution, and so on.
>> >>> >>> That means takes one frame in 640x480 and  than takes some frames at
>> >>> >>> 100x100 (or 200x200) and so on.
>> >>> >>
>> >>> >> Ic, so, if you can live with a fixed output format and only change the
>> >>> >> input cropping rectangle, the patch set, that I've just sent could give
>> >>> >> you a hint, how this can be done. This would work if you're ok with first
>> >>> >> obtaining VGA images scaled down to, say, 160x120, and then take 160x120
>> >>> >> cropped frames unscaled. But I'm not sure, this is something, that would
>> >>> >> work for you. Otherwise, unless your sensor can upscale cropped images to
>> >>> >> VGA output size, you'll also want fast switching between different output
>> >>> >> sizes, which you'd have to wait for (or implement yourself;-))
>> >>> >>
>> >>> >> Thanks
>> >>> >> Guennadi
>> >>> >>
>> >>> >>>
>> >>> >>> The best would be have two different fixed-output image formats, the
>> >>> >>> WHOLE IMAGE format ex. 640x480 and the ROI format, 100x100. The ROI
>> >>> >>> pictures obtained cropping the a region of the whole image. The
>> >>> >>> cropping area could be even wider  than 100x100 and then scaled down
>> >>> >>> to the 100x100 ROI format.
>> >>> >>>
>> >>> >>> Probably it is more simple have a cropping area of the same dimension
>> >>> >>> of the ROI format, 100x100.
>> >>> >>>
>> >>> >>> In this way there is a reduction of the computation load of the CPU
>> >>> >>> (smaller images).
>> >>> >>>
>> >>> >>> Thank you very much!
>> >>> >>>
>> >>> >>> Paolo
>> >>> >>>
>> >>> >>> 2011/3/29 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >>> >>> > On Tue, 29 Mar 2011, Paolo Santinelli wrote:
>> >>> >>> >
>> >>> >>> >> Hi all,
>> >>> >>> >>
>> >>> >>> >> I am using a PXA270 board running linux 2.6.37 equipped with an ov9655
>> >>> >>> >> Image sensor. I am able to use the cropping and scaling capabilities
>> >>> >>> >> V4L2 driver.
>> >>> >>> >> The question is :
>> >>> >>> >>
>> >>> >>> >> Is it possible dynamically change the cropping and scaling values
>> >>> >>> >> without close and re-open  the camera every time ?
>> >>> >>> >>
>> >>> >>> >> Now I am using the streaming I/O memory mapping and to dynamically
>> >>> >>> >> change the cropping and scaling values I do :
>> >>> >>> >>
>> >>> >>> >> 1) stop capturing using VIDIOC_STREAMOFF;
>> >>> >>> >> 2) unmap all the buffers;
>> >>> >>> >> 3) close the device;
>> >>> >>> >> 4) open the device;
>> >>> >>> >> 5) init the device: VIDIOC_CROPCAP and VIDIOC_S_CROP in order to set
>> >>> >>> >> the cropping parameters. VIDIOC_G_FMT and VIDIOC_S_FMT in order to set
>> >>> >>> >> the target image width and height, (scaling).
>> >>> >>> >> 6) Mapping the buffers: VIDIOC_REQBUFS in order to request buffers and
>> >>> >>> >> mmap each buffer using VIDIOC_QUERYBUF and mmap():
>> >>> >>> >>
>> >>> >>> >> this procedure works but take 400 ms.
>> >>> >>> >>
>> >>> >>> >> If I omit steps 3) and 4)  (close and re-open the device) I get this errors:
>> >>> >>> >>
>> >>> >>> >> camera 0-0: S_CROP denied: queue initialised and sizes differ
>> >>> >>> >> camera 0-0: S_FMT denied: queue initialised
>> >>> >>> >> VIDIOC_S_FMT error 16, Device or resource busy
>> >>> >>> >> pxa27x-camera pxa27x-camera.0: PXA Camera driver detached from camera 0
>> >>> >>> >>
>> >>> >>> >> Do you have some Idea regarding why I have to close and reopen the
>> >>> >>> >> device and regarding a way to speed up these change?
>> >>> >>> >
>> >>> >>> > Yes, by chance I do;-) First of all you have to make it more precise -
>> >>> >>> > what exactly do you mean - dynamic (I call it "live") scaling or cropping?
>> >>> >>> > If you want to change output format, that will not be easy ATM, that will
>> >>> >>> > require the snapshot mode API, which is not yet even in an RFC state. If
>> >>> >>> > you only want to change the cropping and keep the output format (zoom),
>> >>> >>> > then I've just implemented that for sh_mobile_ceu_camera. This requires a
>> >>> >>> > couple of extensions to the soc-camera core, which I can post tomorrow.
>> >>> >>> > But in fact that is also a hack, because the proper way to implement this
>> >>> >>> > is to port soc-camera to the Media Controller framework and use the
>> >>> >>> > pad-level API. So, I am not sure, whether we want this in the mainline,
>> >>> >>> > but if already two of us need it now - before the transition to pad-level
>> >>> >>> > operations, maybe it would make sense to mainline this. If, however, you
>> >>> >>> > do have to change your output window, maybe you could tell us your
>> >>> >>> > use-case, so that we could consider, what's the best way to support that.
>> >>> >>> >
>> >>> >>> > Thanks
>> >>> >>> > Guennadi
>> >>> >>> > ---
>> >>> >>> > Guennadi Liakhovetski, Ph.D.
>> >>> >>> > Freelance Open-Source Software Developer
>> >>> >>> > http://www.open-technology.de/
>> >>> >>> >
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> --
>> >>> >>> --------------------------------------------------
>> >>> >>> Paolo Santinelli
>> >>> >>> ImageLab Computer Vision and Pattern Recognition Lab
>> >>> >>> Dipartimento di Ingegneria dell'Informazione
>> >>> >>> Universita' di Modena e Reggio Emilia
>> >>> >>> via Vignolese 905/B, 41125, Modena, Italy
>> >>> >>>
>> >>> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> >>> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> >>> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> >>> >>> --------------------------------------------------
>> >>> >>>
>> >>> >>
>> >>> >> ---
>> >>> >> Guennadi Liakhovetski, Ph.D.
>> >>> >> Freelance Open-Source Software Developer
>> >>> >> http://www.open-technology.de/
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > --------------------------------------------------
>> >>> > Paolo Santinelli
>> >>> > ImageLab Computer Vision and Pattern Recognition Lab
>> >>> > Dipartimento di Ingegneria dell'Informazione
>> >>> > Universita' di Modena e Reggio Emilia
>> >>> > via Vignolese 905/B, 41125, Modena, Italy
>> >>> >
>> >>> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> >>> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> >>> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> >>> > --------------------------------------------------
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> --------------------------------------------------
>> >>> Paolo Santinelli
>> >>> ImageLab Computer Vision and Pattern Recognition Lab
>> >>> Dipartimento di Ingegneria dell'Informazione
>> >>> Universita' di Modena e Reggio Emilia
>> >>> via Vignolese 905/B, 41125, Modena, Italy
>> >>>
>> >>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> >>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> >>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> >>> --------------------------------------------------
>> >>>
>> >>
>> >> ---
>> >> Guennadi Liakhovetski, Ph.D.
>> >> Freelance Open-Source Software Developer
>> >> http://www.open-technology.de/
>> >>
>> >
>> >
>> >
>> > --
>> > --------------------------------------------------
>> > Paolo Santinelli
>> > ImageLab Computer Vision and Pattern Recognition Lab
>> > Dipartimento di Ingegneria dell'Informazione
>> > Universita' di Modena e Reggio Emilia
>> > via Vignolese 905/B, 41125, Modena, Italy
>> >
>> > Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> > email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> > URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> > --------------------------------------------------
>> >
>>
>>
>>
>> --
>> --------------------------------------------------
>> Paolo Santinelli
>> ImageLab Computer Vision and Pattern Recognition Lab
>> Dipartimento di Ingegneria dell'Informazione
>> Universita' di Modena e Reggio Emilia
>> via Vignolese 905/B, 41125, Modena, Italy
>>
>> Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
>> email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
>> URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
>> --------------------------------------------------
>>
>
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
>



-- 
--------------------------------------------------
Paolo Santinelli
ImageLab Computer Vision and Pattern Recognition Lab
Dipartimento di Ingegneria dell'Informazione
Universita' di Modena e Reggio Emilia
via Vignolese 905/B, 41125, Modena, Italy

Cell. +39 3472953357,  Office +39 059 2056270, Fax +39 059 2056129
email:  <mailto:paolo.santinelli@unimore.it> paolo.santinelli@unimore.it
URL:  <http://imagelab.ing.unimo.it/> http://imagelab.ing.unimo.it
--------------------------------------------------