Using UVC webcam gadget with a real v4l2 device

Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

I have been doing some experimentation with the UVC webcam gadget along with the UVC user-space
application which you have written.

The UVC webcam gadget works fine with the user space application handling the CONTROL events and
providing DATA events. Now, I wish to interface a real v4l2 device, for e.g. VIVI or more particularly
a soc_camera based host and subdev pair.

Now, I see that I can achieve this by opening the UVC and V4L2 devices and doing MMAP -> REQBUF ->
QBUF -> DQBUF calls on both the devices per the UVC control event received. But this will involve
copying the video buffer in the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT) domains,
which will significantly reduce the video capture performance.

Is there a better solution to this issue? Maybe doing something like a RNDIS gadget does with
the help of u_ether.c like helper routines. But if I remember well it also requires the BRCTL (Bridge
Control Utility) in userspace to route data arriving on usb0 to eth0 and vice-versa. Not sure though,
if it does copying of a skb buffer from ethernet to usb domain and vice-versa.

Any idea is much appreciated.

Thanks for your help,
Bhupesh

Re: Using UVC webcam gadget with a real v4l2 device

[Laurent Pinchart]

Hi Bhupesh,

On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> Hi Laurent,
> 
> I have been doing some experimentation with the UVC webcam gadget along with
> the UVC user-space application which you have written.
> 
> The UVC webcam gadget works fine with the user space application handling
> the CONTROL events and providing DATA events. Now, I wish to interface a
> real v4l2 device, for e.g. VIVI or more particularly a soc_camera based
> host and subdev pair.
> 
> Now, I see that I can achieve this by opening the UVC and V4L2 devices and
> doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per the UVC
> control event received. But this will involve copying the video buffer in
> the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT) domains,
> which will significantly reduce the video capture performance.
> 
> Is there a better solution to this issue? Maybe doing something like a RNDIS
> gadget does with the help of u_ether.c like helper routines. But if I
> remember well it also requires the BRCTL (Bridge Control Utility) in
> userspace to route data arriving on usb0 to eth0 and vice-versa. Not sure
> though, if it does copying of a skb buffer from ethernet to usb domain and
> vice-versa.

To avoid copying data between the two devices you should use USERPTR instead 
of MMAP on at least one of the two V4L2 devices. The UVC gadget driver doesn't 
support USERPTR yet though. This shouldn't be too difficult to fix, we need to 
replace the custom buffers queue implementation with videobuf2, as has been 
done in the uvcvideo driver.

I'll try to implement this. Would you then be able to test patches ?

-- 
Regards,

Laurent Pinchart

RE: Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

> -----Original Message-----
> From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> Sent: Monday, April 23, 2012 7:47 PM
> To: Bhupesh SHARMA
> Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> balbi@ti.com; g.liakhovetski@gmx.de
> Subject: Re: Using UVC webcam gadget with a real v4l2 device
> 
> Hi Bhupesh,
> 
> On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> > Hi Laurent,
> >
> > I have been doing some experimentation with the UVC webcam gadget
> along with
> > the UVC user-space application which you have written.
> >
> > The UVC webcam gadget works fine with the user space application
> handling
> > the CONTROL events and providing DATA events. Now, I wish to
> interface a
> > real v4l2 device, for e.g. VIVI or more particularly a soc_camera
> based
> > host and subdev pair.
> >
> > Now, I see that I can achieve this by opening the UVC and V4L2
> devices and
> > doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per
> the UVC
> > control event received. But this will involve copying the video
> buffer in
> > the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT)
> domains,
> > which will significantly reduce the video capture performance.
> >
> > Is there a better solution to this issue? Maybe doing something like
> a RNDIS
> > gadget does with the help of u_ether.c like helper routines. But if I
> > remember well it also requires the BRCTL (Bridge Control Utility) in
> > userspace to route data arriving on usb0 to eth0 and vice-versa. Not
> sure
> > though, if it does copying of a skb buffer from ethernet to usb
> domain and
> > vice-versa.
> 
> To avoid copying data between the two devices you should use USERPTR
> instead
> of MMAP on at least one of the two V4L2 devices. The UVC gadget driver
> doesn't
> support USERPTR yet though. This shouldn't be too difficult to fix, we
> need to
> replace the custom buffers queue implementation with videobuf2, as has
> been
> done in the uvcvideo driver.

I was thinking of using the USERPTR method too, but I realized that currently
neither UVC webcam gadget nor soc-camera subsystem supports this IO method.
They support only MMAP IO as of now :(

> 
> I'll try to implement this. Would you then be able to test patches ?

For sure, I can test your patches on my setup.

BTW, I was exploring GSTREAMER to use the data arriving from soc-camera (v4l2)
capture device '/dev/video1' via 'v4l2src' plugin and routing the same to
the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.

Don't know if this can work cleanly in my setup and whether GSTREAMER actually
performs a buffer copy internally. But I will at-least give it a try :)

Regards,
Bhupesh

Re: Using UVC webcam gadget with a real v4l2 device

[Laurent Pinchart]

Hi Bhupesh,

On Tuesday 24 April 2012 02:46:22 Bhupesh SHARMA wrote:
> On Monday, April 23, 2012 7:47 PM Laurent Pinchart wrote:
> > On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> > > Hi Laurent,
> > > 
> > > I have been doing some experimentation with the UVC webcam gadget along
> > > with the UVC user-space application which you have written.
> > > 
> > > The UVC webcam gadget works fine with the user space application
> > > handling the CONTROL events and providing DATA events. Now, I wish to
> > > interface a real v4l2 device, for e.g. VIVI or more particularly a
> > > soc_camera based host and subdev pair.
> > > 
> > > Now, I see that I can achieve this by opening the UVC and V4L2 devices
> > > and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per
> > > the UVC control event received. But this will involve copying the video
> > > buffer in the user-space application from v4l2 (_CAPTURE) to uvc
> > > (_OUTPUT) domains, which will significantly reduce the video capture
> > > performance.
> > > 
> > > Is there a better solution to this issue? Maybe doing something like a
> > > RNDIS gadget does with the help of u_ether.c like helper routines. But
> > > if I remember well it also requires the BRCTL (Bridge Control Utility)
> > > in userspace to route data arriving on usb0 to eth0 and vice-versa. Not
> > > sure though, if it does copying of a skb buffer from ethernet to usb
> > > domain and vice-versa.
> > 
> > To avoid copying data between the two devices you should use USERPTR
> > instead of MMAP on at least one of the two V4L2 devices. The UVC gadget
> > driver doesn't support USERPTR yet though. This shouldn't be too difficult
> > to fix, we need toreplace the custom buffers queue implementation with
> > videobuf2, as has been done in the uvcvideo driver.
> 
> I was thinking of using the USERPTR method too, but I realized that
> currently neither UVC webcam gadget nor soc-camera subsystem supports this
> IO method. They support only MMAP IO as of now :(

Both soc-camera and the UVC gadget driver should be ported to videobuf2 to fix 
the problem.

> > I'll try to implement this. Would you then be able to test patches ?
> 
> For sure, I can test your patches on my setup.

I had a quick look, but there's a bit more work than expected. The UVC gadget 
driver locking scheme needs to be revisited. I unfortunately won't have time 
to work on that in the next couple of weeks, and very probably not before end 
of June. Sorry.

If you want to give it a try, I can provide you with some pointers.

> BTW, I was exploring GSTREAMER to use the data arriving from soc-camera
> (v4l2) capture device '/dev/video1' via 'v4l2src' plugin and routing the
> same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
> 
> Don't know if this can work cleanly in my setup and whether GSTREAMER
> actually performs a buffer copy internally. But I will at-least give it a
> try :)

There will definitely be a buffer copy (and actually two copies, as the UVC 
gadget driver performs a second copy internally) if you don't use USERPTR.

-- 
Regards,

Laurent Pinchart

RE: Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

> -----Original Message-----
> From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> Sent: Tuesday, April 24, 2012 2:26 AM
> To: Bhupesh SHARMA
> Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> balbi@ti.com; g.liakhovetski@gmx.de
> Subject: Re: Using UVC webcam gadget with a real v4l2 device
> 
> Hi Bhupesh,
> 
> On Tuesday 24 April 2012 02:46:22 Bhupesh SHARMA wrote:
> > On Monday, April 23, 2012 7:47 PM Laurent Pinchart wrote:
> > > On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> > > > Hi Laurent,
> > > >
> > > > I have been doing some experimentation with the UVC webcam gadget
> along
> > > > with the UVC user-space application which you have written.
> > > >
> > > > The UVC webcam gadget works fine with the user space application
> > > > handling the CONTROL events and providing DATA events. Now, I
> wish to
> > > > interface a real v4l2 device, for e.g. VIVI or more particularly
> a
> > > > soc_camera based host and subdev pair.
> > > >
> > > > Now, I see that I can achieve this by opening the UVC and V4L2
> devices
> > > > and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
> devices per
> > > > the UVC control event received. But this will involve copying the
> video
> > > > buffer in the user-space application from v4l2 (_CAPTURE) to uvc
> > > > (_OUTPUT) domains, which will significantly reduce the video
> capture
> > > > performance.
> > > >
> > > > Is there a better solution to this issue? Maybe doing something
> like a
> > > > RNDIS gadget does with the help of u_ether.c like helper
> routines. But
> > > > if I remember well it also requires the BRCTL (Bridge Control
> Utility)
> > > > in userspace to route data arriving on usb0 to eth0 and vice-
> versa. Not
> > > > sure though, if it does copying of a skb buffer from ethernet to
> usb
> > > > domain and vice-versa.
> > >
> > > To avoid copying data between the two devices you should use
> USERPTR
> > > instead of MMAP on at least one of the two V4L2 devices. The UVC
> gadget
> > > driver doesn't support USERPTR yet though. This shouldn't be too
> difficult
> > > to fix, we need toreplace the custom buffers queue implementation
> with
> > > videobuf2, as has been done in the uvcvideo driver.
> >
> > I was thinking of using the USERPTR method too, but I realized that
> > currently neither UVC webcam gadget nor soc-camera subsystem supports
> this
> > IO method. They support only MMAP IO as of now :(
> 
> Both soc-camera and the UVC gadget driver should be ported to videobuf2
> to fix
> the problem.
> 
> > > I'll try to implement this. Would you then be able to test patches
> ?
> >
> > For sure, I can test your patches on my setup.
> 
> I had a quick look, but there's a bit more work than expected. The UVC
> gadget
> driver locking scheme needs to be revisited. I unfortunately won't have
> time
> to work on that in the next couple of weeks, and very probably not
> before end
> of June. Sorry.

> If you want to give it a try, I can provide you with some pointers.

It's  a pity. You are the best person to do it as you have in-depth know
-how of both v4l2 and UVC webcam gadget. But I can give it a try if you
can provide me some pointers..

 
> > BTW, I was exploring GSTREAMER to use the data arriving from soc-
> camera
> > (v4l2) capture device '/dev/video1' via 'v4l2src' plugin and routing
> the
> > same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
> >
> > Don't know if this can work cleanly in my setup and whether GSTREAMER
> > actually performs a buffer copy internally. But I will at-least give
> it a
> > try :)
> 
> There will definitely be a buffer copy (and actually two copies, as the
> UVC
> gadget driver performs a second copy internally) if you don't use
> USERPTR.

That's what I was afraid of. But can you let me know where the gadget driver
performs a second copy internally, so that I can also start exploring the
USERPTR method using the pointers provided by you..

Regards,
Bhupesh

RE: Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

Sorry to jump-in before your reply on my previous mail,
but as I was studying the USERPTR stuff in more detail, I have a few more
queries which I believe you can include in your reply as well..

> -----Original Message-----
> From: Bhupesh SHARMA
> Sent: Wednesday, April 25, 2012 8:37 PM
> To: 'Laurent Pinchart'
> Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> balbi@ti.com; g.liakhovetski@gmx.de
> Subject: RE: Using UVC webcam gadget with a real v4l2 device
> 
> Hi Laurent,
> 
> > -----Original Message-----
> > From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> > Sent: Tuesday, April 24, 2012 2:26 AM
> > To: Bhupesh SHARMA
> > Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> > balbi@ti.com; g.liakhovetski@gmx.de
> > Subject: Re: Using UVC webcam gadget with a real v4l2 device
> >
> > Hi Bhupesh,
> >
> > On Tuesday 24 April 2012 02:46:22 Bhupesh SHARMA wrote:
> > > On Monday, April 23, 2012 7:47 PM Laurent Pinchart wrote:
> > > > On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> > > > > Hi Laurent,
> > > > >
> > > > > I have been doing some experimentation with the UVC webcam
> gadget
> > along
> > > > > with the UVC user-space application which you have written.
> > > > >
> > > > > The UVC webcam gadget works fine with the user space
> application
> > > > > handling the CONTROL events and providing DATA events. Now, I
> > wish to
> > > > > interface a real v4l2 device, for e.g. VIVI or more
> particularly
> > a
> > > > > soc_camera based host and subdev pair.
> > > > >
> > > > > Now, I see that I can achieve this by opening the UVC and V4L2
> > devices
> > > > > and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
> > devices per
> > > > > the UVC control event received. But this will involve copying
> the
> > video
> > > > > buffer in the user-space application from v4l2 (_CAPTURE) to
> uvc
> > > > > (_OUTPUT) domains, which will significantly reduce the video
> > capture
> > > > > performance.
> > > > >
> > > > > Is there a better solution to this issue? Maybe doing something
> > like a
> > > > > RNDIS gadget does with the help of u_ether.c like helper
> > routines. But
> > > > > if I remember well it also requires the BRCTL (Bridge Control
> > Utility)
> > > > > in userspace to route data arriving on usb0 to eth0 and vice-
> > versa. Not
> > > > > sure though, if it does copying of a skb buffer from ethernet
> to
> > usb
> > > > > domain and vice-versa.
> > > >
> > > > To avoid copying data between the two devices you should use
> > USERPTR
> > > > instead of MMAP on at least one of the two V4L2 devices. The UVC
> > gadget
> > > > driver doesn't support USERPTR yet though. This shouldn't be too
> > difficult
> > > > to fix, we need toreplace the custom buffers queue implementation
> > with
> > > > videobuf2, as has been done in the uvcvideo driver.
> > >
> > > I was thinking of using the USERPTR method too, but I realized that
> > > currently neither UVC webcam gadget nor soc-camera subsystem
> supports
> > this
> > > IO method. They support only MMAP IO as of now :(
> >
> > Both soc-camera and the UVC gadget driver should be ported to
> videobuf2
> > to fix
> > the problem.

I am now a bit confused on how the entire system will work now:
	- Does USERPTR method needs to be supported both in UVC gadget and soc-camera side,
	  or one can still support the MMAP method and the other can now be changed to support USERPTR method
	  and we can achieve a ZERO buffer copy operation using this method?

	- More specifically, I would like to keep the soc-camera still using MMAP (and hence still using video-buf)
	  and make changes at the UVC gadget side to support USERPTR and videobuf2. Will this work?

	- At the application side how should we design the flow in case both support USERPTR, i.e. the buffer needs
	  to be protected from simultaneous access from the UVC gadget driver and soc-camera driver (to ensure that
	  a single buffer can be shared across them). Also in case we keep soc-camera still using MMAP and UVC gadget
	  side supporting USERPTR, how can we share a common buffer across the UVC gadget and soc-camera driver.

	- In case of USERPTR method the camera capture hardware should be able to DMA the received data to the user
	  space buffers. Are there any specific requirements on the DMA capability of these use-space buffers
	  (scatter-gather or contiguous?).

Regards,
Bhupesh

> > > > I'll try to implement this. Would you then be able to test
> patches
> > ?
> > >
> > > For sure, I can test your patches on my setup.
> >
> > I had a quick look, but there's a bit more work than expected. The
> UVC
> > gadget
> > driver locking scheme needs to be revisited. I unfortunately won't
> have
> > time
> > to work on that in the next couple of weeks, and very probably not
> > before end
> > of June. Sorry.
> 
> > If you want to give it a try, I can provide you with some pointers.
> 
> It's  a pity. You are the best person to do it as you have in-depth
> know
> -how of both v4l2 and UVC webcam gadget. But I can give it a try if you
> can provide me some pointers..
> 
> 
> > > BTW, I was exploring GSTREAMER to use the data arriving from soc-
> > camera
> > > (v4l2) capture device '/dev/video1' via 'v4l2src' plugin and
> routing
> > the
> > > same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
> > >
> > > Don't know if this can work cleanly in my setup and whether
> GSTREAMER
> > > actually performs a buffer copy internally. But I will at-least
> give
> > it a
> > > try :)
> >
> > There will definitely be a buffer copy (and actually two copies, as
> the
> > UVC
> > gadget driver performs a second copy internally) if you don't use
> > USERPTR.
> 
> That's what I was afraid of. But can you let me know where the gadget
> driver
> performs a second copy internally, so that I can also start exploring
> the
> USERPTR method using the pointers provided by you..
> 
> Regards,
> Bhupesh

Re: Using UVC webcam gadget with a real v4l2 device

[Laurent Pinchart]

Hi Bhupesh,

On Thursday 26 April 2012 13:23:59 Bhupesh SHARMA wrote:
> Hi Laurent,
> 
> Sorry to jump-in before your reply on my previous mail,
> but as I was studying the USERPTR stuff in more detail, I have a few more
> queries which I believe you can include in your reply as well..

[snip]

> I am now a bit confused on how the entire system will work now:
> 	- Does USERPTR method needs to be supported both in UVC gadget and
> soc-camera side, or one can still support the MMAP method and the other can
> now be changed to support USERPTR method and we can achieve a ZERO buffer
> copy operation using this method?

You need USERPTR support on one side only. In practice many (all?) soc-camera 
drivers require physically contiguous memory, so you will need to use MMAP on 
the soc-camera side and USERPTR on the UVC gadget side. DMABUF, when merged in 
the kernel, will be a better solution (but will require all drivers to use 
vb2).

> 	- More specifically, I would like to keep the soc-camera still using MMAP
> (and hence still using video-buf) and make changes at the UVC gadget side
> to support USERPTR and videobuf2. Will this work?

Please see above :-)

> 	- At the application side how should we design the flow in case both
> support USERPTR, i.e. the buffer needs to be protected from simultaneous
> access from the UVC gadget driver and soc-camera driver (to ensure that a
> single buffer can be shared across them). Also in case we keep soc-camera
> still using MMAP and UVC gadget side supporting USERPTR, how can we share a
> common buffer across the UVC gadget and soc-camera driver.

That's easy. Request the same number of buffers on both sides with REQBUFS, 
mmap() them to userspace on the soc-camera side, and then use the user pointer 
to queue them with QBUF on the UVC side. You just need to ensure that a buffer 
is never enqueued to two drivers at the same time. Wait for buffers to be 
ready on both sides with select(), and when a buffer is ready dequeue it and 
requeue it on the other side.

> 	- In case of USERPTR method the camera capture hardware should be able to
> DMA the received data to the user space buffers. Are there any specific
> requirements on the DMA capability of these use-space buffers
> (scatter-gather or contiguous?).

DMA to userspace is quite hackish. You should use the MMAP method on the soc-
camera side.

-- 
Regards,

Laurent Pinchart

RE: Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

> -----Original Message-----
> From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> Sent: Monday, April 30, 2012 3:51 PM
> To: Bhupesh SHARMA
> Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> balbi@ti.com; g.liakhovetski@gmx.de
> Subject: Re: Using UVC webcam gadget with a real v4l2 device
> 
> Hi Bhupesh,
> 
> On Thursday 26 April 2012 13:23:59 Bhupesh SHARMA wrote:
> > Hi Laurent,
> >
> > Sorry to jump-in before your reply on my previous mail,
> > but as I was studying the USERPTR stuff in more detail, I have a few
> more
> > queries which I believe you can include in your reply as well..
> 
> [snip]
> 
> > I am now a bit confused on how the entire system will work now:
> > 	- Does USERPTR method needs to be supported both in UVC gadget
> and
> > soc-camera side, or one can still support the MMAP method and the
> other can
> > now be changed to support USERPTR method and we can achieve a ZERO
> buffer
> > copy operation using this method?
> 
> You need USERPTR support on one side only. In practice many (all?) soc-
> camera
> drivers require physically contiguous memory, so you will need to use
> MMAP on
> the soc-camera side and USERPTR on the UVC gadget side. DMABUF, when
> merged in
> the kernel, will be a better solution (but will require all drivers to
> use
> vb2).

Perfect. So, I plan now to add vb2 support for uvc-gadget and leave soc-camera
side to use the mmap stuff.

Now, waiting for your pointers for managing the race-conditions in the UVC gadget
and also avoiding the memcpy that is happening in the QBUF call on the UVC gadget,
before I start the actual work.

Thanks for your help.

Regards,
Bhupesh

> > 	- More specifically, I would like to keep the soc-camera still
> using MMAP
> > (and hence still using video-buf) and make changes at the UVC gadget
> side
> > to support USERPTR and videobuf2. Will this work?
> 
> Please see above :-)
> 
> > 	- At the application side how should we design the flow in case
> both
> > support USERPTR, i.e. the buffer needs to be protected from
> simultaneous
> > access from the UVC gadget driver and soc-camera driver (to ensure
> that a
> > single buffer can be shared across them). Also in case we keep soc-
> camera
> > still using MMAP and UVC gadget side supporting USERPTR, how can we
> share a
> > common buffer across the UVC gadget and soc-camera driver.
> 
> That's easy. Request the same number of buffers on both sides with
> REQBUFS,
> mmap() them to userspace on the soc-camera side, and then use the user
> pointer
> to queue them with QBUF on the UVC side. You just need to ensure that a
> buffer
> is never enqueued to two drivers at the same time. Wait for buffers to
> be
> ready on both sides with select(), and when a buffer is ready dequeue
> it and
> requeue it on the other side.
> 
> > 	- In case of USERPTR method the camera capture hardware should be
> able to
> > DMA the received data to the user space buffers. Are there any
> specific
> > requirements on the DMA capability of these use-space buffers
> > (scatter-gather or contiguous?).
> 
> DMA to userspace is quite hackish. You should use the MMAP method on
> the soc-
> camera side.
> 
> --
> Regards,
> 
> Laurent Pinchart

Re: Using UVC webcam gadget with a real v4l2 device

[Laurent Pinchart]

Hi Bhupesh,

On Monday 30 April 2012 18:47:24 Bhupesh SHARMA wrote:
> On Monday, April 30, 2012 3:51 PM Laurent Pinchart wrote:
> > On Thursday 26 April 2012 13:23:59 Bhupesh SHARMA wrote:
> > > Hi Laurent,
> > > 
> > > Sorry to jump-in before your reply on my previous mail, but as I was
> > > studying the USERPTR stuff in more detail, I have a few more queries
> > > which I believe you can include in your reply as well..
> > 
> > [snip]
> > 
> > > I am now a bit confused on how the entire system will work now:
> > > - Does USERPTR method needs to be supported both in UVC gadget and
> > > soc-camera side, or one can still support the MMAP method and the other
> > > can now be changed to support USERPTR method and we can achieve a ZERO
> > > buffer copy operation using this method?
> > 
> > You need USERPTR support on one side only. In practice many (all?) soc-
> > camera drivers require physically contiguous memory, so you will need to
> > use MMAP on the soc-camera side and USERPTR on the UVC gadget side.
> > DMABUF, when merged in the kernel, will be a better solution (but will
> > require all drivers to use vb2).
> 
> Perfect. So, I plan now to add vb2 support for uvc-gadget and leave soc-
> camera side to use the mmap stuff.
> 
> Now, waiting for your pointers for managing the race-conditions in the UVC
> gadget and also avoiding the memcpy that is happening in the QBUF call on
> the UVC gadget, before I start the actual work.

The memcpy doesn't happen at QBUF time, but when filling the URBs. Avoiding it 
will be pretty difficult, as the driver needs to add packet headers. I would 
leave that out for now.

Regarding videobuf2 support, the main issue comes from race conditions between 
stream start, buffer queueing and URB completion. Unlike the UVC host driver 
where URBs can be resubmitted immediately, the gadget driver can only resubmit 
URBs (in uvc_video_complete()) when there is data to be sent. Otherwise the 
URB is put on a free URBs list (video->req_free) and enqueued in 
uvc_video_pump() the next time a buffer is queued. This requires taking 
various locks and must thus be considered with care. I'm pretty unhappy with 
calling video->encode with the queue irqlock held, I would like to change 
that, but I don't expect to to be an easy task.

-- 
Regards,

Laurent Pinchart

RE: Using UVC webcam gadget with a real v4l2 device

[Bhupesh SHARMA]

Hi Laurent,

> -----Original Message-----
> From: Bhupesh SHARMA
> Sent: Thursday, April 26, 2012 10:54 AM
> To: 'Laurent Pinchart'
> Cc: 'linux-usb@vger.kernel.org'; 'linux-media@vger.kernel.org';
> 'balbi@ti.com'; 'g.liakhovetski@gmx.de'
> Subject: RE: Using UVC webcam gadget with a real v4l2 device
> 
> Hi Laurent,
> 
> Sorry to jump-in before your reply on my previous mail,
> but as I was studying the USERPTR stuff in more detail, I have a few
> more
> queries which I believe you can include in your reply as well..
> 
> > -----Original Message-----
> > From: Bhupesh SHARMA
> > Sent: Wednesday, April 25, 2012 8:37 PM
> > To: 'Laurent Pinchart'
> > Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> > balbi@ti.com; g.liakhovetski@gmx.de
> > Subject: RE: Using UVC webcam gadget with a real v4l2 device
> >
> > Hi Laurent,
> >
> > > -----Original Message-----
> > > From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> > > Sent: Tuesday, April 24, 2012 2:26 AM
> > > To: Bhupesh SHARMA
> > > Cc: linux-usb@vger.kernel.org; linux-media@vger.kernel.org;
> > > balbi@ti.com; g.liakhovetski@gmx.de
> > > Subject: Re: Using UVC webcam gadget with a real v4l2 device
> > >
> > > Hi Bhupesh,
> > >
> > > On Tuesday 24 April 2012 02:46:22 Bhupesh SHARMA wrote:
> > > > On Monday, April 23, 2012 7:47 PM Laurent Pinchart wrote:
> > > > > On Monday 23 April 2012 02:24:53 Bhupesh SHARMA wrote:
> > > > > > Hi Laurent,
> > > > > >
> > > > > > I have been doing some experimentation with the UVC webcam
> > gadget
> > > along
> > > > > > with the UVC user-space application which you have written.
> > > > > >
> > > > > > The UVC webcam gadget works fine with the user space
> > application
> > > > > > handling the CONTROL events and providing DATA events. Now, I
> > > wish to
> > > > > > interface a real v4l2 device, for e.g. VIVI or more
> > particularly
> > > a
> > > > > > soc_camera based host and subdev pair.
> > > > > >
> > > > > > Now, I see that I can achieve this by opening the UVC and
> V4L2
> > > devices
> > > > > > and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
> > > devices per
> > > > > > the UVC control event received. But this will involve copying
> > the
> > > video
> > > > > > buffer in the user-space application from v4l2 (_CAPTURE) to
> > uvc
> > > > > > (_OUTPUT) domains, which will significantly reduce the video
> > > capture
> > > > > > performance.
> > > > > >
> > > > > > Is there a better solution to this issue? Maybe doing
> something
> > > like a
> > > > > > RNDIS gadget does with the help of u_ether.c like helper
> > > routines. But
> > > > > > if I remember well it also requires the BRCTL (Bridge Control
> > > Utility)
> > > > > > in userspace to route data arriving on usb0 to eth0 and vice-
> > > versa. Not
> > > > > > sure though, if it does copying of a skb buffer from ethernet
> > to
> > > usb
> > > > > > domain and vice-versa.
> > > > >
> > > > > To avoid copying data between the two devices you should use
> > > USERPTR
> > > > > instead of MMAP on at least one of the two V4L2 devices. The
> UVC
> > > gadget
> > > > > driver doesn't support USERPTR yet though. This shouldn't be
> too
> > > difficult
> > > > > to fix, we need toreplace the custom buffers queue
> implementation
> > > with
> > > > > videobuf2, as has been done in the uvcvideo driver.
> > > >
> > > > I was thinking of using the USERPTR method too, but I realized
> that
> > > > currently neither UVC webcam gadget nor soc-camera subsystem
> > supports
> > > this
> > > > IO method. They support only MMAP IO as of now :(
> > >
> > > Both soc-camera and the UVC gadget driver should be ported to
> > videobuf2
> > > to fix
> > > the problem.
> 
> I am now a bit confused on how the entire system will work now:
> 	- Does USERPTR method needs to be supported both in UVC gadget
> and soc-camera side,
> 	  or one can still support the MMAP method and the other can now
> be changed to support USERPTR method
> 	  and we can achieve a ZERO buffer copy operation using this
> method?
> 
> 	- More specifically, I would like to keep the soc-camera still
> using MMAP (and hence still using video-buf)
> 	  and make changes at the UVC gadget side to support USERPTR and
> videobuf2. Will this work?
> 
> 	- At the application side how should we design the flow in case
> both support USERPTR, i.e. the buffer needs
> 	  to be protected from simultaneous access from the UVC gadget
> driver and soc-camera driver (to ensure that
> 	  a single buffer can be shared across them). Also in case we
> keep soc-camera still using MMAP and UVC gadget
> 	  side supporting USERPTR, how can we share a common buffer
> across the UVC gadget and soc-camera driver.
> 
> 	- In case of USERPTR method the camera capture hardware should be
> able to DMA the received data to the user
> 	  space buffers. Are there any specific requirements on the DMA
> capability of these use-space buffers
> 	  (scatter-gather or contiguous?).


Can you help me with these queries.

Thanks,
Bhupesh
 
> Regards,
> Bhupesh
> 
> > > > > I'll try to implement this. Would you then be able to test
> > patches
> > > ?
> > > >
> > > > For sure, I can test your patches on my setup.
> > >
> > > I had a quick look, but there's a bit more work than expected. The
> > UVC
> > > gadget
> > > driver locking scheme needs to be revisited. I unfortunately won't
> > have
> > > time
> > > to work on that in the next couple of weeks, and very probably not
> > > before end
> > > of June. Sorry.
> >
> > > If you want to give it a try, I can provide you with some pointers.
> >
> > It's  a pity. You are the best person to do it as you have in-depth
> > know
> > -how of both v4l2 and UVC webcam gadget. But I can give it a try if
> you
> > can provide me some pointers..
> >
> >
> > > > BTW, I was exploring GSTREAMER to use the data arriving from soc-
> > > camera
> > > > (v4l2) capture device '/dev/video1' via 'v4l2src' plugin and
> > routing
> > > the
> > > > same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
> > > >
> > > > Don't know if this can work cleanly in my setup and whether
> > GSTREAMER
> > > > actually performs a buffer copy internally. But I will at-least
> > give
> > > it a
> > > > try :)
> > >
> > > There will definitely be a buffer copy (and actually two copies, as
> > the
> > > UVC
> > > gadget driver performs a second copy internally) if you don't use
> > > USERPTR.
> >
> > That's what I was afraid of. But can you let me know where the gadget
> > driver
> > performs a second copy internally, so that I can also start exploring
> > the
> > USERPTR method using the pointers provided by you..
> >
> > Regards,
> > Bhupesh

Re: Using UVC webcam gadget with a real v4l2 device

[Peter Chen]

On Mon, Apr 23, 2012 at 2:24 AM, Bhupesh SHARMA <bhupesh.sharma@st.com> wrote:
> Hi Laurent,
>
> I have been doing some experimentation with the UVC webcam gadget along with the UVC user-space
> application which you have written.
>
I have tried UVC webcam gadget at Freescale i.mx platform,
unfortunately,  It can't work
properly with my demo application. Would you tell me where I can get
Laurent's user-space
application? Thanks.

-- 
BR,
Peter Chen

Re: Using UVC webcam gadget with a real v4l2 device

[Laurent Pinchart]

On Tuesday 24 April 2012 08:36:34 Peter Chen wrote:
> On Mon, Apr 23, 2012 at 2:24 AM, Bhupesh SHARMA wrote:
> > Hi Laurent,
> > 
> > I have been doing some experimentation with the UVC webcam gadget along
> > with the UVC user-space application which you have written.
> 
> I have tried UVC webcam gadget at Freescale i.mx platform, unfortunately, 
> It can't work properly with my demo application. Would you tell me where I
> can get Laurent's user-space application? Thanks.

Here you go: http://git.ideasonboard.org/uvc-gadget.git

-- 
Regards,

Laurent Pinchart