* USB mini-summit at LinuxCon Vancouver @ 2011-06-10 0:21 Sarah Sharp 2011-06-10 3:18 ` Greg KH ` (3 more replies) 0 siblings, 4 replies; 90+ messages in thread From: Sarah Sharp @ 2011-06-10 0:21 UTC (permalink / raw) To: linux-usb, linux-media, linux-kernel, libusb-devel Cc: Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede I'm pleased to announce a USB mini-summit at LinuxCon Vancouver. What: USB mini-summit When: Tuesday, August 16th, all day Where: At the conference venue, room TBD pending confirmation from Angela Brown. Proposed topics include USB virtualization, and improved bandwidth APIs between the USB core and drivers (especially webcam drivers). See the detailed topic list below. Anyone is also welcome to propose or show up with a USB related topic. MUSB? USB 3.0 gadget drivers? USB-IP? The USB mini-summit does overlap with the virtualization mini-summit by a day, but I'm hoping we can schedule talks so some of the virtualization folks can make it to the USB mini-summit. The other option was on Friday during the conference which was not ideal. Proposed topics: Topic 1 ------- The KVM folks suggested that it would be good to get USB and virtualization developers together to talk about how to virtualize the xHCI host controller. The xHCI spec architect worked closely with VMWare to get some extra goodies in the spec to help virtualization, and I'd like to see the other virtualization developers take advantage of that. I'd also like us to hash out any issues they have been finding in the USB core or xHCI driver during the virtualization effort. Topic 2 ------- I'd also like to get the V4L and audio developers who work with USB devices together with the core USB folks to talk about bandwidth management under xHCI. One of the issues is that since the xHCI hardware does bandwidth management, not the xHCI driver, a schedule that will take too much bandwidth will get rejected much sooner than any USB driver currently expects (during a call to usb_set_interface). This poses issues, since most USB video drivers negotiate the video size and frame rate after they call usb_set_interface, so they don't know whether they can fall back to a less bandwidth-intensive setting. Currently, they just submit URBs with less and less bandwidth until one interval setting gets accepted that won't work under xHCI. A second issue is that that some drivers need less bandwidth than the device advertises, and the xHCI driver currently uses whatever periodic interval the device advertises in its descriptors. This is not what the video/audio driver wants, especially in the case of buggy high speed devices that advertise the interval in frames, not microframes. There needs to be some way for the drivers to communicate their bandwidth needs to the USB core. We've known about this issue for a while, and I think it's time to get everyone in the same room and hash out an API. (I will send out an API proposal later this month.) Hope to see you there! Sarah Sharp ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 0:21 USB mini-summit at LinuxCon Vancouver Sarah Sharp @ 2011-06-10 3:18 ` Greg KH 2011-06-10 6:59 ` Gerd Hoffmann 2011-06-10 19:48 ` Sarah Sharp 2011-06-10 7:19 ` Hans de Goede ` (2 subsequent siblings) 3 siblings, 2 replies; 90+ messages in thread From: Greg KH @ 2011-06-10 3:18 UTC (permalink / raw) To: Sarah Sharp Cc: linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: > Topic 1 > ------- > > The KVM folks suggested that it would be good to get USB and > virtualization developers together to talk about how to virtualize the > xHCI host controller. The xHCI spec architect worked closely with > VMWare to get some extra goodies in the spec to help virtualization, and > I'd like to see the other virtualization developers take advantage of > that. I'd also like us to hash out any issues they have been finding in > the USB core or xHCI driver during the virtualization effort. Do people really want to virtualize the whole xHCI controller, or just specific ports or devices to the guest operating system? If just specific ports, would something like usbip be better for virtual machines, with the USB traffic going over the network connection between the guest/host? > Hope to see you there! Thanks for putting this together. Do we need to sign up somewhere to give the organizers a sense of how many people will be attending? thanks, greg k-h ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 3:18 ` Greg KH @ 2011-06-10 6:59 ` Gerd Hoffmann 2011-06-10 19:48 ` Sarah Sharp 1 sibling, 0 replies; 90+ messages in thread From: Gerd Hoffmann @ 2011-06-10 6:59 UTC (permalink / raw) To: Greg KH Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede Hi, >> The KVM folks suggested that it would be good to get USB and >> virtualization developers together to talk about how to virtualize the >> xHCI host controller. The xHCI spec architect worked closely with >> VMWare to get some extra goodies in the spec to help virtualization, and >> I'd like to see the other virtualization developers take advantage of >> that. I'd also like us to hash out any issues they have been finding in >> the USB core or xHCI driver during the virtualization effort. > > Do people really want to virtualize the whole xHCI controller, or just > specific ports or devices to the guest operating system? SR/IOV support is an optional xHCI feature. As I understand it you can create a VF which looks like a real xHCI controller. This is partly done in hardware and partly by software. Then you can assign it some ressources (specific ports) and pass it to the guest. > If just specific ports, would something like usbip be better for virtual > machines, with the USB traffic going over the network connection between > the guest/host? There are several ways depending on the use case. Usually the guest sees a (fully software emulated) host adapter with usb devices connected, where the usb devices can be (a) emulated too or (b) real usb devices passed through to the guest. The later is done by passing the guests requests to the real device via usbfs. One problem with emulating usb fully in software is the polling design of the hardware which makes the emulation quite cpu intensive. Using a xHCI VF should help here alot, but works for the pass through use case only of course. cheers, Gerd ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 3:18 ` Greg KH 2011-06-10 6:59 ` Gerd Hoffmann @ 2011-06-10 19:48 ` Sarah Sharp 2011-06-10 20:50 ` Greg KH 1 sibling, 1 reply; 90+ messages in thread From: Sarah Sharp @ 2011-06-10 19:48 UTC (permalink / raw) To: Greg KH Cc: linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: > On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: > > Topic 1 > > ------- > > > > The KVM folks suggested that it would be good to get USB and > > virtualization developers together to talk about how to virtualize the > > xHCI host controller. The xHCI spec architect worked closely with > > VMWare to get some extra goodies in the spec to help virtualization, and > > I'd like to see the other virtualization developers take advantage of > > that. I'd also like us to hash out any issues they have been finding in > > the USB core or xHCI driver during the virtualization effort. > > Do people really want to virtualize the whole xHCI controller, or just > specific ports or devices to the guest operating system? A host OS could chose to virtualize the whole xHCI controller if it wanted to. That's part of the reason why xHCI does all the bandwidth checking in hardware, not in software. > If just specific ports, would something like usbip be better for virtual > machines, with the USB traffic going over the network connection between > the guest/host? It could be done that way too. But that doesn't help if you're trying to run Windows under Linux, right? Only if all the guest OSes use the same USB IP protocol then it would work. > > Hope to see you there! > > Thanks for putting this together. > > Do we need to sign up somewhere to give the organizers a sense of how > many people will be attending? I private ack by email would be great. Or you can ack by facebook: https://www.facebook.com/event.php?eid=229532200405657 I could add the event in upcoming.yahoo.com if anyone uses that. Sarah Sharp ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 19:48 ` Sarah Sharp @ 2011-06-10 20:50 ` Greg KH 2011-06-13 10:44 ` Alexander Graf 0 siblings, 1 reply; 90+ messages in thread From: Greg KH @ 2011-06-10 20:50 UTC (permalink / raw) To: Sarah Sharp Cc: linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: > On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: > > On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: > > > Topic 1 > > > ------- > > > > > > The KVM folks suggested that it would be good to get USB and > > > virtualization developers together to talk about how to virtualize the > > > xHCI host controller. The xHCI spec architect worked closely with > > > VMWare to get some extra goodies in the spec to help virtualization, and > > > I'd like to see the other virtualization developers take advantage of > > > that. I'd also like us to hash out any issues they have been finding in > > > the USB core or xHCI driver during the virtualization effort. > > > > Do people really want to virtualize the whole xHCI controller, or just > > specific ports or devices to the guest operating system? > > A host OS could chose to virtualize the whole xHCI controller if it > wanted to. That's part of the reason why xHCI does all the bandwidth > checking in hardware, not in software. And here I thought it did that so it would be "correct" :) > > > If just specific ports, would something like usbip be better for virtual > > machines, with the USB traffic going over the network connection between > > the guest/host? > > It could be done that way too. But that doesn't help if you're trying > to run Windows under Linux, right? Only if all the guest OSes use the > same USB IP protocol then it would work. usbip works on Windows as well as Linux. But how could you run Windows with a xHCI controller in a guest, as Windows has no xHCI driver? What would it expect to see? > > > Hope to see you there! > > > > Thanks for putting this together. > > > > Do we need to sign up somewhere to give the organizers a sense of how > > many people will be attending? > > I private ack by email would be great. Or you can ack by facebook: > https://www.facebook.com/event.php?eid=229532200405657 I could add the > event in upcoming.yahoo.com if anyone uses that. Nice try, but I'm still not going to join facebook... ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 20:50 ` Greg KH @ 2011-06-13 10:44 ` Alexander Graf 0 siblings, 0 replies; 90+ messages in thread From: Alexander Graf @ 2011-06-13 10:44 UTC (permalink / raw) To: Greg KH Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: > On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: >> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: >>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: >>>> Topic 1 >>>> ------- >>>> >>>> The KVM folks suggested that it would be good to get USB and >>>> virtualization developers together to talk about how to virtualize the >>>> xHCI host controller. The xHCI spec architect worked closely with >>>> VMWare to get some extra goodies in the spec to help virtualization, and >>>> I'd like to see the other virtualization developers take advantage of >>>> that. I'd also like us to hash out any issues they have been finding in >>>> the USB core or xHCI driver during the virtualization effort. >>> >>> Do people really want to virtualize the whole xHCI controller, or just >>> specific ports or devices to the guest operating system? >> >> A host OS could chose to virtualize the whole xHCI controller if it >> wanted to. That's part of the reason why xHCI does all the bandwidth >> checking in hardware, not in software. > > And here I thought it did that so it would be "correct" :) > >> >>> If just specific ports, would something like usbip be better for virtual >>> machines, with the USB traffic going over the network connection between >>> the guest/host? >> >> It could be done that way too. But that doesn't help if you're trying >> to run Windows under Linux, right? Only if all the guest OSes use the >> same USB IP protocol then it would work. > > usbip works on Windows as well as Linux. Do you have a reliable, working usbip solution at hand that work on Windows and Linux and doesn't require real network access, which can be a no-go for some scenarios? > > But how could you run Windows with a xHCI controller in a guest, as > Windows has no xHCI driver? What would it expect to see? There are drivers for xhci adapters on Windows. Also, this whole discussion is pretty much future oriented - which most likely means built-in xhci drivers anywhere. No, in all seriousity, the main reason to go for FV vs PV is that so far, the best tested and working drivers are built for real hardware. I'm still unsatisfied with the PV driver situation for virtio on Windows. It's just incredibly hard to get Windows drivers right - and open source developers sure are not good at it :) But sure, let's talk about it during LinuxCon as well - there's a good chance you know more there than me :) > Alex ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver @ 2011-06-13 10:44 ` Alexander Graf 0 siblings, 0 replies; 90+ messages in thread From: Alexander Graf @ 2011-06-13 10:44 UTC (permalink / raw) To: Greg KH Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: > On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: >> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: >>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: >>>> Topic 1 >>>> ------- >>>> >>>> The KVM folks suggested that it would be good to get USB and >>>> virtualization developers together to talk about how to virtualize the >>>> xHCI host controller. The xHCI spec architect worked closely with >>>> VMWare to get some extra goodies in the spec to help virtualization, and >>>> I'd like to see the other virtualization developers take advantage of >>>> that. I'd also like us to hash out any issues they have been finding in >>>> the USB core or xHCI driver during the virtualization effort. >>> >>> Do people really want to virtualize the whole xHCI controller, or just >>> specific ports or devices to the guest operating system? >> >> A host OS could chose to virtualize the whole xHCI controller if it >> wanted to. That's part of the reason why xHCI does all the bandwidth >> checking in hardware, not in software. > > And here I thought it did that so it would be "correct" :) > >> >>> If just specific ports, would something like usbip be better for virtual >>> machines, with the USB traffic going over the network connection between >>> the guest/host? >> >> It could be done that way too. But that doesn't help if you're trying >> to run Windows under Linux, right? Only if all the guest OSes use the >> same USB IP protocol then it would work. > > usbip works on Windows as well as Linux. Do you have a reliable, working usbip solution at hand that work on Windows and Linux and doesn't require real network access, which can be a no-go for some scenarios? > > But how could you run Windows with a xHCI controller in a guest, as > Windows has no xHCI driver? What would it expect to see? There are drivers for xhci adapters on Windows. Also, this whole discussion is pretty much future oriented - which most likely means built-in xhci drivers anywhere. No, in all seriousity, the main reason to go for FV vs PV is that so far, the best tested and working drivers are built for real hardware. I'm still unsatisfied with the PV driver situation for virtio on Windows. It's just incredibly hard to get Windows drivers right - and open source developers sure are not good at it :) But sure, let's talk about it during LinuxCon as well - there's a good chance you know more there than me :) > Alex ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-13 10:44 ` Alexander Graf @ 2011-06-13 16:29 ` Greg KH -1 siblings, 0 replies; 90+ messages in thread From: Greg KH @ 2011-06-13 16:29 UTC (permalink / raw) To: Alexander Graf Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On Mon, Jun 13, 2011 at 12:44:57PM +0200, Alexander Graf wrote: > > Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: > > > On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: > >> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: > >>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: > >>>> Topic 1 > >>>> ------- > >>>> > >>>> The KVM folks suggested that it would be good to get USB and > >>>> virtualization developers together to talk about how to virtualize the > >>>> xHCI host controller. The xHCI spec architect worked closely with > >>>> VMWare to get some extra goodies in the spec to help virtualization, and > >>>> I'd like to see the other virtualization developers take advantage of > >>>> that. I'd also like us to hash out any issues they have been finding in > >>>> the USB core or xHCI driver during the virtualization effort. > >>> > >>> Do people really want to virtualize the whole xHCI controller, or just > >>> specific ports or devices to the guest operating system? > >> > >> A host OS could chose to virtualize the whole xHCI controller if it > >> wanted to. That's part of the reason why xHCI does all the bandwidth > >> checking in hardware, not in software. > > > > And here I thought it did that so it would be "correct" :) > > > >> > >>> If just specific ports, would something like usbip be better for virtual > >>> machines, with the USB traffic going over the network connection between > >>> the guest/host? > >> > >> It could be done that way too. But that doesn't help if you're trying > >> to run Windows under Linux, right? Only if all the guest OSes use the > >> same USB IP protocol then it would work. > > > > usbip works on Windows as well as Linux. > > Do you have a reliable, working usbip solution at hand that work on > Windows and Linux and doesn't require real network access, which can > be a no-go for some scenarios? What do you mean by "real network access"? A virtual network connection to the guest and host should be sufficient, right? > > But how could you run Windows with a xHCI controller in a guest, as > > Windows has no xHCI driver? What would it expect to see? > > There are drivers for xhci adapters on Windows. Not well-working ones, see the windows driver mailing list as well as the libusb mailing list for problems that people are having due to there not being any "official" Microsoft xhci drivers shipping yet. > Also, this whole discussion is pretty much future oriented - which > most likely means built-in xhci drivers anywhere. Agreed. thanks, greg k-h ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver @ 2011-06-13 16:29 ` Greg KH 0 siblings, 0 replies; 90+ messages in thread From: Greg KH @ 2011-06-13 16:29 UTC (permalink / raw) To: Alexander Graf Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On Mon, Jun 13, 2011 at 12:44:57PM +0200, Alexander Graf wrote: > > Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: > > > On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: > >> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: > >>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: > >>>> Topic 1 > >>>> ------- > >>>> > >>>> The KVM folks suggested that it would be good to get USB and > >>>> virtualization developers together to talk about how to virtualize the > >>>> xHCI host controller. The xHCI spec architect worked closely with > >>>> VMWare to get some extra goodies in the spec to help virtualization, and > >>>> I'd like to see the other virtualization developers take advantage of > >>>> that. I'd also like us to hash out any issues they have been finding in > >>>> the USB core or xHCI driver during the virtualization effort. > >>> > >>> Do people really want to virtualize the whole xHCI controller, or just > >>> specific ports or devices to the guest operating system? > >> > >> A host OS could chose to virtualize the whole xHCI controller if it > >> wanted to. That's part of the reason why xHCI does all the bandwidth > >> checking in hardware, not in software. > > > > And here I thought it did that so it would be "correct" :) > > > >> > >>> If just specific ports, would something like usbip be better for virtual > >>> machines, with the USB traffic going over the network connection between > >>> the guest/host? > >> > >> It could be done that way too. But that doesn't help if you're trying > >> to run Windows under Linux, right? Only if all the guest OSes use the > >> same USB IP protocol then it would work. > > > > usbip works on Windows as well as Linux. > > Do you have a reliable, working usbip solution at hand that work on > Windows and Linux and doesn't require real network access, which can > be a no-go for some scenarios? What do you mean by "real network access"? A virtual network connection to the guest and host should be sufficient, right? > > But how could you run Windows with a xHCI controller in a guest, as > > Windows has no xHCI driver? What would it expect to see? > > There are drivers for xhci adapters on Windows. Not well-working ones, see the windows driver mailing list as well as the libusb mailing list for problems that people are having due to there not being any "official" Microsoft xhci drivers shipping yet. > Also, this whole discussion is pretty much future oriented - which > most likely means built-in xhci drivers anywhere. Agreed. thanks, greg k-h ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-13 16:29 ` Greg KH @ 2011-06-13 17:11 ` Alexander Graf -1 siblings, 0 replies; 90+ messages in thread From: Alexander Graf @ 2011-06-13 17:11 UTC (permalink / raw) To: Greg KH Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On 13.06.2011, at 18:29, Greg KH wrote: > On Mon, Jun 13, 2011 at 12:44:57PM +0200, Alexander Graf wrote: >> >> Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: >> >>> On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: >>>> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: >>>>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: >>>>>> Topic 1 >>>>>> ------- >>>>>> >>>>>> The KVM folks suggested that it would be good to get USB and >>>>>> virtualization developers together to talk about how to virtualize the >>>>>> xHCI host controller. The xHCI spec architect worked closely with >>>>>> VMWare to get some extra goodies in the spec to help virtualization, and >>>>>> I'd like to see the other virtualization developers take advantage of >>>>>> that. I'd also like us to hash out any issues they have been finding in >>>>>> the USB core or xHCI driver during the virtualization effort. >>>>> >>>>> Do people really want to virtualize the whole xHCI controller, or just >>>>> specific ports or devices to the guest operating system? >>>> >>>> A host OS could chose to virtualize the whole xHCI controller if it >>>> wanted to. That's part of the reason why xHCI does all the bandwidth >>>> checking in hardware, not in software. >>> >>> And here I thought it did that so it would be "correct" :) >>> >>>> >>>>> If just specific ports, would something like usbip be better for virtual >>>>> machines, with the USB traffic going over the network connection between >>>>> the guest/host? >>>> >>>> It could be done that way too. But that doesn't help if you're trying >>>> to run Windows under Linux, right? Only if all the guest OSes use the >>>> same USB IP protocol then it would work. >>> >>> usbip works on Windows as well as Linux. >> >> Do you have a reliable, working usbip solution at hand that work on >> Windows and Linux and doesn't require real network access, which can >> be a no-go for some scenarios? > > What do you mean by "real network access"? A virtual network connection > to the guest and host should be sufficient, right? Yes, which would either mean we inject a second network adapter into the guest which needs to be configured to an IP address, hence possibly colliding with some other IP range that's in use outside, or reuse some real network connection, which is neither always given, nor always desired. We have a full new PV serial protocol (virtio-serial) just for exactly that reason: To be able to run a daemon inside the guest that does not require IP access to talk to the host. USB/IP really has its own huge set of issues. Sure, it might be good for certain use cases, but it's definitely not a one-size-fits-all solution. But so far, doing FV emulation right has worked out a lot better for most use cases than inventing PV solutions. Real hardware simply seems to be supported by others, while we can't afford to write drivers for all OSs :). So IMHO, PV should be the last resort, when emulating real hardware utterly fails. In this case, I believe the hardware designers finally got the spec usable for us, so it should be good! :) > >>> But how could you run Windows with a xHCI controller in a guest, as >>> Windows has no xHCI driver? What would it expect to see? >> >> There are drivers for xhci adapters on Windows. > > Not well-working ones, see the windows driver mailing list as well as > the libusb mailing list for problems that people are having due to there > not being any "official" Microsoft xhci drivers shipping yet. Yes, which is an issue that's hopefully gone when we're finally having xHCI actually properly emulated and all the infrastructure for it laid out. We still have UHCI and OHCI emulation today already to fall back to if really necessary, but installing an xHCI driver inside the guest really isn't more work than installing USB/IP inside the guest. Alex ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver @ 2011-06-13 17:11 ` Alexander Graf 0 siblings, 0 replies; 90+ messages in thread From: Alexander Graf @ 2011-06-13 17:11 UTC (permalink / raw) To: Greg KH Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede On 13.06.2011, at 18:29, Greg KH wrote: > On Mon, Jun 13, 2011 at 12:44:57PM +0200, Alexander Graf wrote: >> >> Am 10.06.2011 um 22:50 schrieb Greg KH <greg@kroah.com>: >> >>> On Fri, Jun 10, 2011 at 12:48:15PM -0700, Sarah Sharp wrote: >>>> On Thu, Jun 09, 2011 at 08:18:05PM -0700, Greg KH wrote: >>>>> On Thu, Jun 09, 2011 at 05:21:03PM -0700, Sarah Sharp wrote: >>>>>> Topic 1 >>>>>> ------- >>>>>> >>>>>> The KVM folks suggested that it would be good to get USB and >>>>>> virtualization developers together to talk about how to virtualize the >>>>>> xHCI host controller. The xHCI spec architect worked closely with >>>>>> VMWare to get some extra goodies in the spec to help virtualization, and >>>>>> I'd like to see the other virtualization developers take advantage of >>>>>> that. I'd also like us to hash out any issues they have been finding in >>>>>> the USB core or xHCI driver during the virtualization effort. >>>>> >>>>> Do people really want to virtualize the whole xHCI controller, or just >>>>> specific ports or devices to the guest operating system? >>>> >>>> A host OS could chose to virtualize the whole xHCI controller if it >>>> wanted to. That's part of the reason why xHCI does all the bandwidth >>>> checking in hardware, not in software. >>> >>> And here I thought it did that so it would be "correct" :) >>> >>>> >>>>> If just specific ports, would something like usbip be better for virtual >>>>> machines, with the USB traffic going over the network connection between >>>>> the guest/host? >>>> >>>> It could be done that way too. But that doesn't help if you're trying >>>> to run Windows under Linux, right? Only if all the guest OSes use the >>>> same USB IP protocol then it would work. >>> >>> usbip works on Windows as well as Linux. >> >> Do you have a reliable, working usbip solution at hand that work on >> Windows and Linux and doesn't require real network access, which can >> be a no-go for some scenarios? > > What do you mean by "real network access"? A virtual network connection > to the guest and host should be sufficient, right? Yes, which would either mean we inject a second network adapter into the guest which needs to be configured to an IP address, hence possibly colliding with some other IP range that's in use outside, or reuse some real network connection, which is neither always given, nor always desired. We have a full new PV serial protocol (virtio-serial) just for exactly that reason: To be able to run a daemon inside the guest that does not require IP access to talk to the host. USB/IP really has its own huge set of issues. Sure, it might be good for certain use cases, but it's definitely not a one-size-fits-all solution. But so far, doing FV emulation right has worked out a lot better for most use cases than inventing PV solutions. Real hardware simply seems to be supported by others, while we can't afford to write drivers for all OSs :). So IMHO, PV should be the last resort, when emulating real hardware utterly fails. In this case, I believe the hardware designers finally got the spec usable for us, so it should be good! :) > >>> But how could you run Windows with a xHCI controller in a guest, as >>> Windows has no xHCI driver? What would it expect to see? >> >> There are drivers for xhci adapters on Windows. > > Not well-working ones, see the windows driver mailing list as well as > the libusb mailing list for problems that people are having due to there > not being any "official" Microsoft xhci drivers shipping yet. Yes, which is an issue that's hopefully gone when we're finally having xHCI actually properly emulated and all the infrastructure for it laid out. We still have UHCI and OHCI emulation today already to fall back to if really necessary, but installing an xHCI driver inside the guest really isn't more work than installing USB/IP inside the guest. Alex ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 0:21 USB mini-summit at LinuxCon Vancouver Sarah Sharp 2011-06-10 3:18 ` Greg KH @ 2011-06-10 7:19 ` Hans de Goede 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede 2011-08-04 22:21 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab 3 siblings, 0 replies; 90+ messages in thread From: Hans de Goede @ 2011-06-10 7:19 UTC (permalink / raw) To: Sarah Sharp Cc: linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi, On 06/10/2011 02:21 AM, Sarah Sharp wrote: > I'm pleased to announce a USB mini-summit at LinuxCon Vancouver. > > What: USB mini-summit > When: Tuesday, August 16th, all day > Where: At the conference venue, room TBD pending confirmation from > Angela Brown. > > Proposed topics include USB virtualization, and improved bandwidth APIs > between the USB core and drivers (especially webcam drivers). See the > detailed topic list below. Anyone is also welcome to propose or show up > with a USB related topic. MUSB? USB 3.0 gadget drivers? USB-IP? > I would like to give a short presentation on / demo off the usb redirection over the network (USB-IP replacement with a less dumb protocol) I've been working on, followed by a questions and answers session. For those who want to give this a try now, see: http://hansdegoede.livejournal.com/9682.html Sheets from the presentation I gave on this at FOSDEM: http://people.fedoraproject.org/~jwrdegoede/usb-redir.pdf > The USB mini-summit does overlap with the virtualization mini-summit by > a day, but I'm hoping we can schedule talks so some of the > virtualization folks can make it to the USB mini-summit. The other > option was on Friday during the conference which was not ideal. > > Proposed topics: > > Topic 1 > ------- > > The KVM folks suggested that it would be good to get USB and > virtualization developers together to talk about how to virtualize the > xHCI host controller. The xHCI spec architect worked closely with > VMWare to get some extra goodies in the spec to help virtualization, and > I'd like to see the other virtualization developers take advantage of > that. I'd also like us to hash out any issues they have been finding in > the USB core or xHCI driver during the virtualization effort. I'm not really happy with how the management / hand over of userspace to kernel space drivers and vice versa works. This is a problem when doing usb-redirection to a virtual machine. I think this is best discussed in a separate thread, and then if needed further discussed during the mini summit. I'll send a mail on this shortly. I'll use the same address list as this mail, except that I'm going to cut linux-kernel from the CC list. > Topic 2 > ------- > > I'd also like to get the V4L and audio developers who work with USB > devices together with the core USB folks to talk about bandwidth > management under xHCI. > > One of the issues is that since the xHCI hardware does bandwidth > management, not the xHCI driver, a schedule that will take too much > bandwidth will get rejected much sooner than any USB driver currently > expects (during a call to usb_set_interface). This poses issues, since > most USB video drivers negotiate the video size and frame rate after > they call usb_set_interface, so they don't know whether they can fall > back to a less bandwidth-intensive setting. Currently, they just submit > URBs with less and less bandwidth until one interval setting gets > accepted that won't work under xHCI. > > A second issue is that that some drivers need less bandwidth than the > device advertises, and the xHCI driver currently uses whatever periodic > interval the device advertises in its descriptors. This is not what the > video/audio driver wants, especially in the case of buggy high speed > devices that advertise the interval in frames, not microframes. There > needs to be some way for the drivers to communicate their bandwidth > needs to the USB core. We've known about this issue for a while, and I > think it's time to get everyone in the same room and hash out an API. Interesting, being able to tell the core how much bandwidth a device will actually use (versus what it claims as maxpacketsize in its descriptors) is something which we really need. I know of at least 2 usb1 webcam chipsets (and drivers) which have only 1 altsetting which claim 1023 bytes maxpacketsize. But they also have a register which allows the driver to configure how large the largest (iso) packet it sends will actually be. Currently these drivers have this "beauty" to be able to tell the linux usb core that they won't be using 1023 as maxpacketsize but something else, and thus function without needing full usb1 bandwidth: struct usb_host_interface *alt; alt = &gspca_dev->dev->config->intf_cache[0]->altsetting[1]; alt->endpoint[0].desc.wMaxPacketSize = cpu_to_le16(packet_size); ret = usb_set_interface(gspca_dev->dev, gspca_dev->iface, 1); Yes they are overwriting the kernels cached descriptors ... ### Something which I would also like to bring to everyone's attention is that we really need to fix the ehci schedule code wrt scheduling usb1 transfers over usb2 hubs. The current code is very broken when it comes to periodic transfers, it basically disallows using the last microframe (let alone the crossing of the frame boundary and using the first microframe of the next frame). This means that trying to submit isoc transfers with a size of 1023 will just plain fail, even if this is the only device on the entire bus. This is one of the reasons I ended up doing the hack above, so that these devices will at least work (be it at a decreased framerate) through a usb2 hub. Things become even messier when the device has a build in usb audio microphone, often this will just not work. Finally fixing this has become more important now then ever since sandybridge machines (and maybe generation one core i# machines too, I don't know) no longer have a companion controller, but instead have an integrated usb2 hub. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 0:21 USB mini-summit at LinuxCon Vancouver Sarah Sharp 2011-06-10 3:18 ` Greg KH 2011-06-10 7:19 ` Hans de Goede @ 2011-06-10 7:55 ` Hans de Goede 2011-06-10 8:22 ` Felipe Balbi ` (2 more replies) 2011-08-04 22:21 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab 3 siblings, 3 replies; 90+ messages in thread From: Hans de Goede @ 2011-06-10 7:55 UTC (permalink / raw) To: linux-usb Cc: Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi all, The current API for managing kernel -> userspace is a bit rough around the edges, so I would like to discuss extending the API. First of all an example use case scenarios where the current API falls short. 1) Redirection of USB devices to a virtual machine, qemu, vbox, etc. all have the ability to redirect a USB device to the virtual machine, and they all use usbfs for this. The first thing which will happen here when the user selects a device to redirect is a IOCTL_USBFS_DISCONNECT. This causes the kernel driver to see a device unplug, with no chances for the kernel driver to do anything against this. Now lets say the user does the following: -write a file to a usb flash disk -redirect the flash disk to a vm Currently this will cause the usb mass storage driver to see a disconnect, and any possible still pending writes are lost ... This is IMHO unacceptable, but currently there is nothing we can do to avoid this. 2) So called dual mode cameras are (cheap) stillcams often even without an lcdscreen viewfinder, and battery backed sram instead of flash, which double as a webcam. We have drivers for both the stillcam function of these (in libgphoto2, so using usbfs) as well as for the webcam function (v4l2 kernel drivers). These drivers work well, and are mature. Yet the user experience is rather poor. Under gnome the still-cam contents will be automatically be made available as a "drive" using a gvfs-gphoto2 fuse mount. This however involves sending a disconnect to the v4l2 kernel driver, and thus the /dev/video# node disappearing. So if a user wants to use the device as a webcam he/she needs to first go to nautilus and unmount the gvfs mount. Until that is done the user will simply get a message from an app like cheese that he has no webcam, not even an ebusy error, just that he has no such device. Again not good. ### So what do we need to make this situation better: 1) A usb_driver callback alternative to the disconnect callback, I propose to call this soft_disconnect. This serves 2 purposes a) It will allow the driver to tell the caller that that is not a good idea by returning an error code (think usb mass storage driver and mounted filesystem b) It will allow for example a v4l2 driver to keep its /dev/video node around Note that b) means that the normal disconnect handler should still be called after a soft reconnect on a real disconnect. 2) A usb_driver soft_reconnect callback to match the soft_disconnect 3) A mechanism for a usb_driver to signal a usbfs fd owner of the device it would like the device back. So for example the gvfs mount can be automatically unmounted (if not busy). 4) A IOCTL_USBFS_SOFT_DISCONNECT ioctl which will call the drivers soft_disconnect if it has one, and otherwise fall back to the regular disconnect. 5) A method for a usbfs fd owning app to know the device driver would like the device back. I suggest using poll with POLLIN to signal this. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede @ 2011-06-10 8:22 ` Felipe Balbi 2011-06-10 8:36 ` Hans de Goede 2011-06-10 14:48 ` Alan Stern 2011-06-10 18:16 ` Theodore Kilgore 2 siblings, 1 reply; 90+ messages in thread From: Felipe Balbi @ 2011-06-10 8:22 UTC (permalink / raw) To: Hans de Goede Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 2090 bytes --] Hi, On Fri, Jun 10, 2011 at 09:55:13AM +0200, Hans de Goede wrote: > Currently this will cause the usb mass storage driver to see a > disconnect, and any possible still pending writes are lost ... > > This is IMHO unacceptable, but currently there is nothing we can > do to avoid this. > > 2) So called dual mode cameras are (cheap) stillcams often even > without an lcdscreen viewfinder, and battery backed sram instead > of flash, which double as a webcam. We have drivers for both the > stillcam function of these (in libgphoto2, so using usbfs) as > well as for the webcam function (v4l2 kernel drivers). > > These drivers work well, and are mature. Yet the user experience > is rather poor. Under gnome the still-cam contents will be > automatically be made available as a "drive" using a gvfs-gphoto2 fuse > mount. This however involves sending a disconnect to the v4l2 kernel > driver, and thus the /dev/video# node disappearing. So if a user > wants to use the device as a webcam he/she needs to first go to > nautilus and unmount the gvfs mount. Until that is done the user will > simply get a message from an app like cheese that he has no webcam, > not even an ebusy error, just that he has no such device. that sounds quite weird. Should only happen if still image and video functions are on different configurations or different alt-settings of the same interface. But if they are on same configurations and separate interfaces, you should be able to bind gphoto to the still image interface and v4l2 to the camera interface. How's the device setup ? > So what do we need to make this situation better: > 1) A usb_driver callback alternative to the disconnect callback, > I propose to call this soft_disconnect. This serves 2 purposes > a) It will allow the driver to tell the caller that that is not > a good idea by returning an error code (think usb mass storage > driver and mounted filesystem I'm not sure you even need a driver callback for that. Should we leave that to Desktop manager ? -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 8:22 ` Felipe Balbi @ 2011-06-10 8:36 ` Hans de Goede 2011-06-10 8:42 ` Felipe Balbi 0 siblings, 1 reply; 90+ messages in thread From: Hans de Goede @ 2011-06-10 8:36 UTC (permalink / raw) To: balbi Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi, On 06/10/2011 10:22 AM, Felipe Balbi wrote: > Hi, > > On Fri, Jun 10, 2011 at 09:55:13AM +0200, Hans de Goede wrote: >> Currently this will cause the usb mass storage driver to see a >> disconnect, and any possible still pending writes are lost ... >> >> This is IMHO unacceptable, but currently there is nothing we can >> do to avoid this. >> >> 2) So called dual mode cameras are (cheap) stillcams often even >> without an lcdscreen viewfinder, and battery backed sram instead >> of flash, which double as a webcam. We have drivers for both the >> stillcam function of these (in libgphoto2, so using usbfs) as >> well as for the webcam function (v4l2 kernel drivers). >> >> These drivers work well, and are mature. Yet the user experience >> is rather poor. Under gnome the still-cam contents will be >> automatically be made available as a "drive" using a gvfs-gphoto2 fuse >> mount. This however involves sending a disconnect to the v4l2 kernel >> driver, and thus the /dev/video# node disappearing. So if a user >> wants to use the device as a webcam he/she needs to first go to >> nautilus and unmount the gvfs mount. Until that is done the user will >> simply get a message from an app like cheese that he has no webcam, >> not even an ebusy error, just that he has no such device. > > that sounds quite weird. Should only happen if still image and video > functions are on different configurations or different alt-settings of > the same interface. But if they are on same configurations and separate > interfaces, you should be able to bind gphoto to the still image > interface and v4l2 to the camera interface. > > How's the device setup ? > These are very cheap devices, and as such poorly designed. There still and webcam functionality is on the same interface. This is likely done this way because the devices cannot handle both functions at the same time. >> So what do we need to make this situation better: >> 1) A usb_driver callback alternative to the disconnect callback, >> I propose to call this soft_disconnect. This serves 2 purposes >> a) It will allow the driver to tell the caller that that is not >> a good idea by returning an error code (think usb mass storage >> driver and mounted filesystem > > I'm not sure you even need a driver callback for that. Should we leave > that to Desktop manager ? Not sure what you mean here, but we need for a way for drivers to say no to a software caused disconnection. See my usb mass storage device which is still mounted getting redirected to a vm example. This cannot be reliably done from userspace. Where as it is trivial to do this from kernel space. One could advocate to make the existing disconnect ioctl use the new soft_disconnect usb_driver callback instead of adding a new usbfs ioctl for this, but that means that a driver can block any and all userspace triggered disconnects. Where as having a new ioctl, means that apps which want to play nice can play nice, while keeping the possibility of a hard userspace initiated disconnect. One could also argue that making the existing disconnect ioctl return -EBUSY in some cases now is an ABI change. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 8:36 ` Hans de Goede @ 2011-06-10 8:42 ` Felipe Balbi 2011-06-10 12:19 ` Hans de Goede 0 siblings, 1 reply; 90+ messages in thread From: Felipe Balbi @ 2011-06-10 8:42 UTC (permalink / raw) To: Hans de Goede Cc: balbi, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 3867 bytes --] Hi, On Fri, Jun 10, 2011 at 10:36:47AM +0200, Hans de Goede wrote: > >On Fri, Jun 10, 2011 at 09:55:13AM +0200, Hans de Goede wrote: > >>Currently this will cause the usb mass storage driver to see a > >>disconnect, and any possible still pending writes are lost ... > >> > >>This is IMHO unacceptable, but currently there is nothing we can > >>do to avoid this. > >> > >>2) So called dual mode cameras are (cheap) stillcams often even > >>without an lcdscreen viewfinder, and battery backed sram instead > >>of flash, which double as a webcam. We have drivers for both the > >>stillcam function of these (in libgphoto2, so using usbfs) as > >>well as for the webcam function (v4l2 kernel drivers). > >> > >>These drivers work well, and are mature. Yet the user experience > >>is rather poor. Under gnome the still-cam contents will be > >>automatically be made available as a "drive" using a gvfs-gphoto2 fuse > >>mount. This however involves sending a disconnect to the v4l2 kernel > >>driver, and thus the /dev/video# node disappearing. So if a user > >>wants to use the device as a webcam he/she needs to first go to > >>nautilus and unmount the gvfs mount. Until that is done the user will > >>simply get a message from an app like cheese that he has no webcam, > >>not even an ebusy error, just that he has no such device. > > > >that sounds quite weird. Should only happen if still image and video > >functions are on different configurations or different alt-settings of > >the same interface. But if they are on same configurations and separate > >interfaces, you should be able to bind gphoto to the still image > >interface and v4l2 to the camera interface. > > > >How's the device setup ? > > > > These are very cheap devices, and as such poorly designed. There still > and webcam functionality is on the same interface. This is likely done > this way because the devices cannot handle both functions at the same > time. ok got it. > >>So what do we need to make this situation better: > >>1) A usb_driver callback alternative to the disconnect callback, > >> I propose to call this soft_disconnect. This serves 2 purposes > >> a) It will allow the driver to tell the caller that that is not > >> a good idea by returning an error code (think usb mass storage > >> driver and mounted filesystem > > > >I'm not sure you even need a driver callback for that. Should we leave > >that to Desktop manager ? > > Not sure what you mean here, but we need for a way for drivers to say > no to a software caused disconnection. See my usb mass storage device > which is still mounted getting redirected to a vm example. This cannot in that case, why don't you just flush all data and continue ? Also, desktop manager knows that a particular device mounted, so it could also ask the user if s/he wants to continue. I'm not sure preventing a disconnect is a good thing. > be reliably done from userspace. Where as it is trivial to do this > from kernel space. One could advocate to make the existing disconnect > ioctl use the new soft_disconnect usb_driver callback instead of > adding a new usbfs ioctl for this, but that means that a driver > can block any and all userspace triggered disconnects. Where as > having a new ioctl, means that apps which want to play nice can play > nice, while keeping the possibility of a hard userspace initiated > disconnect. > > One could also argue that making the existing disconnect ioctl return > -EBUSY in some cases now is an ABI change. OTOH, if the user really wants to move the usb device to the guest OS, he has just requested for that, so should we prevent it ? what we need is for the applications to be notified to exit cleanly and release the device because the user has requested to do so. No ? -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 8:42 ` Felipe Balbi @ 2011-06-10 12:19 ` Hans de Goede 2011-06-10 12:28 ` Felipe Balbi 0 siblings, 1 reply; 90+ messages in thread From: Hans de Goede @ 2011-06-10 12:19 UTC (permalink / raw) To: balbi Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi, On 06/10/2011 10:42 AM, Felipe Balbi wrote: > Hi, > > On Fri, Jun 10, 2011 at 10:36:47AM +0200, Hans de Goede wrote: >>> On Fri, Jun 10, 2011 at 09:55:13AM +0200, Hans de Goede wrote: <snip> >>>> So what do we need to make this situation better: >>>> 1) A usb_driver callback alternative to the disconnect callback, >>>> I propose to call this soft_disconnect. This serves 2 purposes >>>> a) It will allow the driver to tell the caller that that is not >>>> a good idea by returning an error code (think usb mass storage >>>> driver and mounted filesystem >>> >>> I'm not sure you even need a driver callback for that. Should we leave >>> that to Desktop manager ? >> >> Not sure what you mean here, but we need for a way for drivers to say >> no to a software caused disconnection. See my usb mass storage device >> which is still mounted getting redirected to a vm example. This cannot > > in that case, why don't you just flush all data and continue ? Also, > desktop manager knows that a particular device mounted, so it could also > ask the user if s/he wants to continue. > > I'm not sure preventing a disconnect is a good thing. > I assume you are sure preventing data loss is a good thing? Because in this example the 2 are the same. Also note I'm not suggestion at always preventing the disconnect, I'm suggesting to add a new try_disconnect ioctl, which apps which want to behave nicely can use instead of the regular disconnect ioctl, and then drivers can prevent the disconnect. Apps using the old ioctl will still get an unconditional disconnect as before. >> be reliably done from userspace. Where as it is trivial to do this >> from kernel space. One could advocate to make the existing disconnect >> ioctl use the new soft_disconnect usb_driver callback instead of >> adding a new usbfs ioctl for this, but that means that a driver >> can block any and all userspace triggered disconnects. Where as >> having a new ioctl, means that apps which want to play nice can play >> nice, while keeping the possibility of a hard userspace initiated >> disconnect. >> >> One could also argue that making the existing disconnect ioctl return >> -EBUSY in some cases now is an ABI change. > > OTOH, if the user really wants to move the usb device to the guest OS, > he has just requested for that, so should we prevent it ? what we need > is for the applications to be notified to exit cleanly and release the > device because the user has requested to do so. No ? We are talking about a device with a mounted file system on it here, any process could be holding files open on there, and there currently exists no mechanism to notify all apps to exit cleanly and release the files. Even if there were some method for a desktop environment like gnome to ask apps to release those files, and all gnome apps where to be modified to support that mechanism, then there are still 1000-s of non gnome (or kde, xfce, whatever) apps which will not support that. We already have a mechanism to cleanly close down a filesystem, it is called unmount. And it will fail if apps have files open. All I'm suggesting is forwarding this ebusy failure to the application trying to disconnect the driver from the usb mass storage interface. Simply removing the filesystem from under apps holding files open will lead to io errors, and very unhappy apps. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 12:19 ` Hans de Goede @ 2011-06-10 12:28 ` Felipe Balbi 0 siblings, 0 replies; 90+ messages in thread From: Felipe Balbi @ 2011-06-10 12:28 UTC (permalink / raw) To: Hans de Goede Cc: balbi, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 3971 bytes --] Hi, On Fri, Jun 10, 2011 at 02:19:20PM +0200, Hans de Goede wrote: > >>>>So what do we need to make this situation better: > >>>>1) A usb_driver callback alternative to the disconnect callback, > >>>> I propose to call this soft_disconnect. This serves 2 purposes > >>>> a) It will allow the driver to tell the caller that that is not > >>>> a good idea by returning an error code (think usb mass storage > >>>> driver and mounted filesystem > >>> > >>>I'm not sure you even need a driver callback for that. Should we leave > >>>that to Desktop manager ? > >> > >>Not sure what you mean here, but we need for a way for drivers to say > >>no to a software caused disconnection. See my usb mass storage device > >>which is still mounted getting redirected to a vm example. This cannot > > > >in that case, why don't you just flush all data and continue ? Also, > >desktop manager knows that a particular device mounted, so it could also > >ask the user if s/he wants to continue. > > > >I'm not sure preventing a disconnect is a good thing. > > I assume you are sure preventing data loss is a good thing? Because in > this example the 2 are the same. > > Also note I'm not suggestion at always preventing the disconnect, I'm > suggesting to add a new try_disconnect ioctl, which apps which want > to behave nicely can use instead of the regular disconnect ioctl, and > then drivers can prevent the disconnect. Apps using the old ioctl will > still get an unconditional disconnect as before. I'm just not sure if this should sit in the kernel. You can easily have some sort of device manager (much like udisks for storage) to track this down for ya. So instead of calling usbfs straight, you need to go via the "device manager". This device manager should know that before calling 'disconnect' it must be sure there's nothing pending. > >>One could also argue that making the existing disconnect ioctl return > >>-EBUSY in some cases now is an ABI change. > > > >OTOH, if the user really wants to move the usb device to the guest OS, > >he has just requested for that, so should we prevent it ? what we need > >is for the applications to be notified to exit cleanly and release the > >device because the user has requested to do so. No ? > > We are talking about a device with a mounted file system on it here, > any process could be holding files open on there, and there currently > exists no mechanism to notify all apps to exit cleanly and release > the files. Even if there were some method for a desktop environment > like gnome to ask apps to release those files, and all gnome apps > where to be modified to support that mechanism, then there are still > 1000-s of non gnome (or kde, xfce, whatever) apps which will not > support that. Still, I'm not sure this is argument good enough to add another sort of conflicting ioctl to the kernel. It won't prevent apps from calling the other disconnect anyway. And besides, if an app can be modified to use the try_disconnect ioctl, they can be modified for the system notification (on userland) too, can't they ? > We already have a mechanism to cleanly close down a filesystem, it > is called unmount. And it will fail if apps have files open. All I'm > suggesting is forwarding this ebusy failure to the application > trying to disconnect the driver from the usb mass storage interface. > > Simply removing the filesystem from under apps holding files open will > lead to io errors, and very unhappy apps. Correct, but does the solution _have_ to sit in kernel ? I'm not sure. If you add the try_disconnect ioctl and all applications one your desktop, except one, are playing nice, you will still have problems. To me, it sounds much nicer to add such things on userland alone, in a way such that it doesn't matter if it's a usb storage, mmc card, or eSATA drive. That should be hidden from apps IMHO. -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede 2011-06-10 8:22 ` Felipe Balbi @ 2011-06-10 14:48 ` Alan Stern 2011-06-10 15:07 ` Mauro Carvalho Chehab 2011-06-11 9:15 ` Hans de Goede 2011-06-10 18:16 ` Theodore Kilgore 2 siblings, 2 replies; 90+ messages in thread From: Alan Stern @ 2011-06-10 14:48 UTC (permalink / raw) To: Hans de Goede Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Hans de Goede wrote: > Hi all, > > The current API for managing kernel -> userspace is a bit > rough around the edges, so I would like to discuss extending > the API. > > First of all an example use case scenarios where the current API > falls short. > > 1) Redirection of USB devices to a virtual machine, qemu, vbox, etc. > all have the ability to redirect a USB device to the virtual machine, > and they all use usbfs for this. The first thing which will happen > here when the user selects a device to redirect is a > IOCTL_USBFS_DISCONNECT. This causes the kernel driver to see a > device unplug, with no chances for the kernel driver to do anything > against this. > > Now lets say the user does the following: > -write a file to a usb flash disk > -redirect the flash disk to a vm > > Currently this will cause the usb mass storage driver to see a > disconnect, and any possible still pending writes are lost ... > > This is IMHO unacceptable, but currently there is nothing we can > do to avoid this. You haven't given a proper description of the problem. See below. > 2) So called dual mode cameras are (cheap) stillcams often even > without an lcdscreen viewfinder, and battery backed sram instead > of flash, which double as a webcam. We have drivers for both the > stillcam function of these (in libgphoto2, so using usbfs) as > well as for the webcam function (v4l2 kernel drivers). > > These drivers work well, and are mature. Yet the user experience > is rather poor. Under gnome the still-cam contents will be > automatically be made available as a "drive" using a gvfs-gphoto2 fuse > mount. This however involves sending a disconnect to the v4l2 kernel > driver, and thus the /dev/video# node disappearing. So if a user > wants to use the device as a webcam he/she needs to first go to > nautilus and unmount the gvfs mount. Until that is done the user will > simply get a message from an app like cheese that he has no webcam, > not even an ebusy error, just that he has no such device. > > Again not good. As Felipe has mentioned, this sounds like the sort of problem that can better be solved in userspace. A dual-mode device like the one you describe really is either a still-cam or a webcam, never both at the same time. Hence what users need is a utility program to switch modes (by loading/unloading the appropriate programs or drivers). Or maybe a desktop daemon that could accomplish the same result automatically, based on requests from user programs. > ### > > So what do we need to make this situation better: > 1) A usb_driver callback alternative to the disconnect callback, > I propose to call this soft_disconnect. This serves 2 purposes > a) It will allow the driver to tell the caller that that is not > a good idea by returning an error code (think usb mass storage > driver and mounted filesystem Not feasible. usb-storage has no idea whether or not a device it controls has a mounted filesystem. (All it does is send SCSI commands to a device and get back the results.) Since that's the main use case you're interested in, this part of the proposal seems destined to fail. But userspace _does_ know where the mounted filesystems are. Therefore userspace should be responsible for avoiding programs that want to take control of devices holding these filesystems. That's the reason why usbfs device nodes are owned by root and have 0644 mode; there're can be written to only by programs with superuser privileges -- and such programs are supposed to be careful about what they do. > b) It will allow for example a v4l2 driver to keep its /dev/video > node around > Note that b) means that the normal disconnect handler should still > be called after a soft reconnect on a real disconnect. In some sense the disconnect callback for usb-storage already _is_ "soft". Although the driver cannot refuse the disconnect, it _can_ continue to communicate with the device until the callback returns. (As it happens, usb-storage _doesn't_ do any further communication with the device. This is mostly for historical reasons, to compensate for shortcomings in the SCSI stack in earlier kernel versions. On the other hand, there really isn't much that you would want to send to a mass-storage device during a soft disconnect. Perhaps tell it to flush its cache out to the storage medium -- but if there are no open file handles for the device and no mounted filesystems then the cache will already be flushed.) > 2) A usb_driver soft_reconnect callback to match the soft_disconnect > 3) A mechanism for a usb_driver to signal a usbfs fd owner of the device > it would like the device back. So for example the gvfs mount can be > automatically unmounted (if not busy). This also should be handled in userspace. USB drivers never "want back" a device they are no longer bound to -- in fact, the device model used throughout the kernel makes this whole idea meaningless. Device drivers don't want devices back. Rather, _users_ want to turn control of devices over to specific drivers. That's why this problem needs to be handled in userspace. > 4) A IOCTL_USBFS_SOFT_DISCONNECT ioctl which will call the drivers > soft_disconnect if it has one, and otherwise fall back to the > regular disconnect. > 5) A method for a usbfs fd owning app to know the device driver would > like the device back. I suggest using poll with POLLIN to signal this. It seems as if you're trying to implement some notion of allowing a device to have more than one driver at the same time. This is so far out from the way the kernel behaves now, adopting it would be very difficult if not impossible. Certainly the USB stack isn't the place to start. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 14:48 ` Alan Stern @ 2011-06-10 15:07 ` Mauro Carvalho Chehab 2011-06-10 15:21 ` Alan Stern 2011-06-11 9:15 ` Hans de Goede 1 sibling, 1 reply; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-06-10 15:07 UTC (permalink / raw) To: Alan Stern Cc: Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Em 10-06-2011 11:48, Alan Stern escreveu: > On Fri, 10 Jun 2011, Hans de Goede wrote: > > > As Felipe has mentioned, this sounds like the sort of problem that > can better be solved in userspace. A dual-mode device like the one > you describe really is either a still-cam or a webcam, never both at > the same time. Hence what users need is a utility program to switch > modes (by loading/unloading the appropriate programs or drivers). Unloading a driver in order to access the hardware via userspace? This sounds a very bad idea do me. What happens if another hardware is using the same driver? Cheers, Mauro. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 15:07 ` Mauro Carvalho Chehab @ 2011-06-10 15:21 ` Alan Stern 0 siblings, 0 replies; 90+ messages in thread From: Alan Stern @ 2011-06-10 15:21 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Mauro Carvalho Chehab wrote: > Em 10-06-2011 11:48, Alan Stern escreveu: > > On Fri, 10 Jun 2011, Hans de Goede wrote: > > > > > > As Felipe has mentioned, this sounds like the sort of problem that > > can better be solved in userspace. A dual-mode device like the one > > you describe really is either a still-cam or a webcam, never both at > > the same time. Hence what users need is a utility program to switch > > modes (by loading/unloading the appropriate programs or drivers). > > Unloading a driver in order to access the hardware via userspace? > This sounds a very bad idea do me. What happens if another hardware > is using the same driver? A kernel driver wouldn't have to be unloaded. It could simply be unbound from the device via sysfs or usbfs. Alan STern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 14:48 ` Alan Stern 2011-06-10 15:07 ` Mauro Carvalho Chehab @ 2011-06-11 9:15 ` Hans de Goede 2011-06-11 16:19 ` Theodore Kilgore 2011-06-11 16:57 ` Alan Stern 1 sibling, 2 replies; 90+ messages in thread From: Hans de Goede @ 2011-06-11 9:15 UTC (permalink / raw) To: Alan Stern Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi, Given the many comments in this thread, I'm just going reply to this one, and try to also answer any other ones in this mail. As far as the dual mode camera is involved, I agree that that should be fixed in the existing v4l2 drivers + libgphoto. I think that Felipe's solution to also handle the stillcam part in kernel space for dual mode cameras (and add a libgphoto cam driver which knows how to talk the new kernel API for this), is the best solution. Unfortunately this will involve quite a bit of work, but so be it. On 06/10/2011 04:48 PM, Alan Stern wrote: > On Fri, 10 Jun 2011, Hans de Goede wrote: > >> Hi all, >> >> The current API for managing kernel -> userspace is a bit >> rough around the edges, so I would like to discuss extending >> the API. >> >> First of all an example use case scenarios where the current API >> falls short. >> >> 1) Redirection of USB devices to a virtual machine, qemu, vbox, etc. >> all have the ability to redirect a USB device to the virtual machine, >> and they all use usbfs for this. The first thing which will happen >> here when the user selects a device to redirect is a >> IOCTL_USBFS_DISCONNECT. This causes the kernel driver to see a >> device unplug, with no chances for the kernel driver to do anything >> against this. >> >> Now lets say the user does the following: >> -write a file to a usb flash disk >> -redirect the flash disk to a vm >> >> Currently this will cause the usb mass storage driver to see a >> disconnect, and any possible still pending writes are lost ... >> >> This is IMHO unacceptable, but currently there is nothing we can >> do to avoid this. > > You haven't given a proper description of the problem. See below. I think I've given an excellent description of the problem, drivers can be unbound from devices, and there is no way for drivers to block this. All I'm asking for is for a new usbfs try_disconnect ioctl which the currently bound driver has a chance blocking nothing more nothing less. <snip dual mode camera stuff> >> So what do we need to make this situation better: >> 1) A usb_driver callback alternative to the disconnect callback, >> I propose to call this soft_disconnect. This serves 2 purposes >> a) It will allow the driver to tell the caller that that is not >> a good idea by returning an error code (think usb mass storage >> driver and mounted filesystem > > Not feasible. usb-storage has no idea whether or not a device it > controls has a mounted filesystem. (All it does is send SCSI commands > to a device and get back the results.) Since that's the main use > case you're interested in, this part of the proposal seems destined to > fail. > This is not completely true, I cannot rmmod usb-storage as long as disks using it are mounted. I know this is done through the global module usage count, so this is not per usb-storage device. But extending the ref counting to be per usb-storage device should not be hard. All the accounting is already done for this. > But userspace _does_ know where the mounted filesystems are. > Therefore userspace should be responsible for avoiding programs that > want to take control of devices holding these filesystems. That's the > reason why usbfs device nodes are owned by root and have 0644 mode; > there're can be written to only by programs with superuser privileges > -- and such programs are supposed to be careful about what they do. > Yes, and what I'm asking for is for an easy way for these programs to be careful. A way for them to ask the kernel, which in general is responsible for things like this and traditionally does resource management and things which come with that like refcounting: "unbind the driver from this device unless the device is currently in use". Why can't this be done in userspace, simply put: 1) Process a checks if the usb-storage device is not used 2) Process b mounts it after the check 3) Process a unbinds the driver Yes this can be avoided if all binding/unbinding and all mounting happens under control of a single instance. However this requires completely re-inventing userspace... Also note that the notion of this can be extended to other devices, want to ubs-redirect a usb printer to a vm better not do it halfway through a printjob being spooled. Want to usb-redirect a usb webcam to a vm, better not do it while something is streaming video from the webcam, etc. etc. And please don't come with the inevitable "if it hurts do not do that" argument. We want to provide this kind of functionality to non tech savy computer users, and things should just work, including telling the user that the device is currently in use rather then wrecking his printjob or filesystem. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-11 9:15 ` Hans de Goede @ 2011-06-11 16:19 ` Theodore Kilgore 2011-06-12 11:43 ` Hans de Goede 2011-06-11 16:57 ` Alan Stern 1 sibling, 1 reply; 90+ messages in thread From: Theodore Kilgore @ 2011-06-11 16:19 UTC (permalink / raw) To: Hans de Goede Cc: Alan Stern, linux-usb, Sarah Sharp, linux-media, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Sat, 11 Jun 2011, Hans de Goede wrote: > Hi, > > Given the many comments in this thread, I'm just > going reply to this one, and try to also answer any > other ones in this mail. > > As far as the dual mode camera is involved, I agree > that that should be fixed in the existing v4l2 > drivers + libgphoto. I think that Felipe's solution > to also handle the stillcam part in kernel space for > dual mode cameras (and add a libgphoto cam driver which > knows how to talk the new kernel API for this), is > the best solution. Unfortunately this will involve > quite a bit of work, but so be it. Hans, It appears to me that the solution ought to be at hand, actually. I was not aware of the recent changes in libusb, which I understand are supposed to allow a kernel driver to be hooked up again. To review the situation: 1. As of approximately 2 years ago, libusb already was so configured as to suspend the kernel module for a dual-mode device if a userspace-based program tried to claim the device. 2. At this point with the more recent versions of libusb (see the last message from yesterday, from Xiaofan Chen), we are supposed to be able to re-activate the kernel module for the device when it is relinquished by userspace. This ought to take care of the problems completely, provided that the new capabilities of libusb are actually used and called upon in libgphoto2. I have checked on what is happening, just now, on my own machine. I have libusb version 1.08 which ought to be recent enough. The advertised abilities did not work, however. Presumably, what is missing is on the other end of the problem, most likely in the functions in libgphoto2 which hook up a camera. That code would presumably need to call upon the new functionality of libusb. My currently installed version of libgphoto2 (from svn, but several months old) clearly does not contain the needed functionality. But it might have been put in recently and I did not notice. I guess that the first thing to do is to update my gphoto tree and then to see what happens. If things still don't work, then something needs to be updated and then things ought to work. I will try to see that something gets done about this. Thank you for raising the old issue of dual-mode devices yet again, and thanks to Xiaofan Chen for pointing out that the needed missing half of the functionality is supposed to exist now in libusb. That had escaped my attention. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-11 16:19 ` Theodore Kilgore @ 2011-06-12 11:43 ` Hans de Goede 2011-06-12 21:20 ` Theodore Kilgore 0 siblings, 1 reply; 90+ messages in thread From: Hans de Goede @ 2011-06-12 11:43 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Stern, linux-usb, Sarah Sharp, linux-media, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Hi, On 06/11/2011 06:19 PM, Theodore Kilgore wrote: > > > On Sat, 11 Jun 2011, Hans de Goede wrote: > >> Hi, >> >> Given the many comments in this thread, I'm just >> going reply to this one, and try to also answer any >> other ones in this mail. >> >> As far as the dual mode camera is involved, I agree >> that that should be fixed in the existing v4l2 >> drivers + libgphoto. I think that Felipe's solution >> to also handle the stillcam part in kernel space for >> dual mode cameras (and add a libgphoto cam driver which >> knows how to talk the new kernel API for this), is >> the best solution. Unfortunately this will involve >> quite a bit of work, but so be it. > > Hans, > > It appears to me that the solution ought to be at hand, actually. > > I was not aware of the recent changes in libusb, which I understand are > supposed to allow a kernel driver to be hooked up again. > > To review the situation: > > 1. As of approximately 2 years ago, libusb already was so configured as to > suspend the kernel module for a dual-mode device if a userspace-based > program tried to claim the device. > > 2. At this point with the more recent versions of libusb (see the last > message from yesterday, from Xiaofan Chen), we are supposed to be able to > re-activate the kernel module for the device when it is relinquished by > userspace. > > This ought to take care of the problems completely, provided that the new > capabilities of libusb are actually used and called upon in libgphoto2. > > I have checked on what is happening, just now, on my own machine. I have > libusb version 1.08 which ought to be recent enough. The advertised > abilities did not work, however. Presumably, what is missing is on the > other end of the problem, most likely in the functions in libgphoto2 which > hook up a camera. That code would presumably need to call upon the new > functionality of libusb. My currently installed version of libgphoto2 > (from svn, but several months old) clearly does not contain the needed > functionality. But it might have been put in recently and I did not > notice. I guess that the first thing to do is to update my gphoto tree and > then to see what happens. If things still don't work, then something needs > to be updated and then things ought to work. > > I will try to see that something gets done about this. Thank you for > raising the old issue of dual-mode devices yet again, and thanks to > Xiaofan Chen for pointing out that the needed missing half of the > functionality is supposed to exist now in libusb. That had escaped my > attention. Actually libusb and libgphoto have been using the rebind orginal driver functionality of the code for quite a while now, unfortunately this does not solve the problem, unless we somehow move to 1 central coordinator for the device the user experience will stay subpar. Example, user downloads pictures from the camera using shotwell, gthumb, fspot or whatever, keeps the app in question open and the app in question keeps the gphoto2 device handle open. User wants to do some skyping with video chat, skype complains it cannot find the device, since the kernel driver currently is unbound. -> Poor user experience. With having both functions in the kernel, the kernel could actually allow skype to use the dual mode cameras as video source, and if the user then were to switch to f-spot and try to import more photo's then he will get an -ebusy in f-spot. If he finishes skyping and then returns to f-spot everything will just continue working. This is the kind of "seamless" user experience I'm aiming for here. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-12 11:43 ` Hans de Goede @ 2011-06-12 21:20 ` Theodore Kilgore 2011-06-13 2:03 ` Xiaofan Chen 0 siblings, 1 reply; 90+ messages in thread From: Theodore Kilgore @ 2011-06-12 21:20 UTC (permalink / raw) To: Hans de Goede Cc: Alan Stern, linux-usb, Sarah Sharp, linux-media, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Sun, 12 Jun 2011, Hans de Goede wrote: > Hi, > > On 06/11/2011 06:19 PM, Theodore Kilgore wrote: > > > > > > On Sat, 11 Jun 2011, Hans de Goede wrote: > > > > > Hi, > > > > > > Given the many comments in this thread, I'm just > > > going reply to this one, and try to also answer any > > > other ones in this mail. > > > > > > As far as the dual mode camera is involved, I agree > > > that that should be fixed in the existing v4l2 > > > drivers + libgphoto. I think that Felipe's solution > > > to also handle the stillcam part in kernel space for > > > dual mode cameras (and add a libgphoto cam driver which > > > knows how to talk the new kernel API for this), is > > > the best solution. Unfortunately this will involve > > > quite a bit of work, but so be it. > > > > Hans, > > > > It appears to me that the solution ought to be at hand, actually. > > > > I was not aware of the recent changes in libusb, which I understand are > > supposed to allow a kernel driver to be hooked up again. > > > > To review the situation: > > > > 1. As of approximately 2 years ago, libusb already was so configured as to > > suspend the kernel module for a dual-mode device if a userspace-based > > program tried to claim the device. > > > > 2. At this point with the more recent versions of libusb (see the last > > message from yesterday, from Xiaofan Chen), we are supposed to be able to > > re-activate the kernel module for the device when it is relinquished by > > userspace. > > > > This ought to take care of the problems completely, provided that the new > > capabilities of libusb are actually used and called upon in libgphoto2. > > > > I have checked on what is happening, just now, on my own machine. I have > > libusb version 1.08 which ought to be recent enough. The advertised > > abilities did not work, however. Presumably, what is missing is on the > > other end of the problem, most likely in the functions in libgphoto2 which > > hook up a camera. That code would presumably need to call upon the new > > functionality of libusb. My currently installed version of libgphoto2 > > (from svn, but several months old) clearly does not contain the needed > > functionality. But it might have been put in recently and I did not > > notice. I guess that the first thing to do is to update my gphoto tree and > > then to see what happens. If things still don't work, then something needs > > to be updated and then things ought to work. > > > > I will try to see that something gets done about this. Thank you for > > raising the old issue of dual-mode devices yet again, and thanks to > > Xiaofan Chen for pointing out that the needed missing half of the > > functionality is supposed to exist now in libusb. That had escaped my > > attention. > > Actually libusb and libgphoto have been using the rebind orginal driver > functionality of the code for quite a while now, Oh? I can see that libusb is doing that, and I can also see that there is a "public" function for _unbinding_ a kernel driver, namely int usb_detach_kernel_driver_np() found in usb.h and it is used in libgphoto, as well. I am not sure that there is any corresponding rebind function which is public. Is it perhaps int usb_get_driver_np() ??? By context (looking at libgphoto2-port/usb/libusb.c) I would think that this function is not the rebind function, but is only checking whether or not there is any potential conflict with a kernel driver. If I am right, then where is the publicly exported rebind function, and where does it currently get used in libgphoto2? So frankly after my eagerness yesterday I do not see how it can easily be made to work, after all. unfortunately this > does not solve the problem, unless we somehow move to 1 central > coordinator for the device the user experience will stay subpar. > > Example, user downloads pictures from the camera using shotwell, > gthumb, fspot or whatever, keeps the app in question open and the app > in question keeps the gphoto2 device handle open. > > User wants to do some skyping with video chat, skype complains it > cannot find the device, since the kernel driver currently is unbound. > > -> Poor user experience. Poor user experience, or merely poor user? The user ought to know better. Of course, I do agree that there are lots of such people, and it is a good idea to try to put up warning signs. > > With having both functions in the kernel, the kernel could actually > allow skype to use the dual mode cameras as video source, and if > the user then were to switch to f-spot and try to import more photo's > then he will get an -ebusy in f-spot. If he finishes skyping and > then returns to f-spot everything will just continue working. > > This is the kind of "seamless" user experience I'm aiming for here. > > Regards, > > Hans Yes, I can see where you are coming from. But if the camera really will not let you run skype and fspot at the same time, which I do not believe it would allow on _any_ operating system, then each app should give an error message which says it cannot be run unless and until the other app has been closed. If that has to happen at the kernel level, then OK. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-12 21:20 ` Theodore Kilgore @ 2011-06-13 2:03 ` Xiaofan Chen 2011-06-13 2:27 ` [Libusb-devel] " Michael Bender 0 siblings, 1 reply; 90+ messages in thread From: Xiaofan Chen @ 2011-06-13 2:03 UTC (permalink / raw) To: Theodore Kilgore; +Cc: Hans de Goede, linux-usb, linux-media, libusb-devel On Mon, Jun 13, 2011 at 5:20 AM, Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote: > On Sun, 12 Jun 2011, Hans de Goede wrote: >> Actually libusb and libgphoto have been using the rebind orginal driver >> functionality of the code for quite a while now, > > Oh? I can see that libusb is doing that, and I can also see that there is > a "public" function for _unbinding_ a kernel driver, namely > > int usb_detach_kernel_driver_np() > > found in usb.h > > and it is used in libgphoto, as well. > > I am not sure that there is any corresponding rebind function which is > public. Is it perhaps > > int usb_get_driver_np() > > ??? > > By context (looking at libgphoto2-port/usb/libusb.c) I would think that > this function is not the rebind function, but is only checking whether or > not there is any potential conflict with a kernel driver. If I am right, > then where is the publicly exported rebind function, and where does it > currently get used in libgphoto2? http://gphoto.svn.sourceforge.net/viewvc/gphoto/trunk/libgphoto2/libgphoto2_port/usb/libusb.c?revision=13652&view=markup The rebind happened under the function "static int gp_port_usb_close (GPPort *port)". Since libgphoto2 is still using libusb-0.1, the unbind is using usbfs IOCTL directly (USBDEVFS_CONNECT). > So frankly after my eagerness yesterday I do not see how it can easily be > made to work, after all. > >> unfortunately this >> does not solve the problem, unless we somehow move to 1 central >> coordinator for the device the user experience will stay subpar. Now I understand what Hans is saying. It will be a lot of work trying to sort out this issue in userspace. What can be the single central coordinator? A device manager applet listing the program or service which hold the device? >> Example, user downloads pictures from the camera using shotwell, >> gthumb, fspot or whatever, keeps the app in question open and the app >> in question keeps the gphoto2 device handle open. >> >> User wants to do some skyping with video chat, skype complains it >> cannot find the device, since the kernel driver currently is unbound. >> >> -> Poor user experience. > > Poor user experience, or merely poor user? The user ought to know better. > Of course, I do agree that there are lots of such people, and it is a good > idea to try to put up warning signs. It is difficult to call the users "poor users" in this case. Since they may not know that the other open program is holding the device. Some warning message may help, not "I can not find the device" though. It would be better to pinpoint which program is holding the device and then ask the user to close that program. I understand this is easily said than done... Similar experiences for Windows about the serial port, sometimes it is difficult for the user to know that some program or service are holding the serial port so that the other program or will fail or Windows complain that it is still open when you want to undock the computer. >> >> With having both functions in the kernel, the kernel could actually >> allow skype to use the dual mode cameras as video source, and if >> the user then were to switch to f-spot and try to import more photo's >> then he will get an -ebusy in f-spot. If he finishes skyping and >> then returns to f-spot everything will just continue working. >> >> This is the kind of "seamless" user experience I'm aiming for here. >> > Yes, I can see where you are coming from. But if the camera really will > not let you run skype and fspot at the same time, which I do not believe > it would allow on _any_ operating system, then each app should give an > error message which says it cannot be run unless and until the other app > has been closed. If that has to happen at the kernel level, then OK. > Yes. From what I read, to solve it in kernel or to solve it in user space are both a lot of work. Personally I tend to think to solve it in user space is more feasible. -- Xiaofan ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [Libusb-devel] Improving kernel -> userspace (usbfs) usb device hand off 2011-06-13 2:03 ` Xiaofan Chen @ 2011-06-13 2:27 ` Michael Bender 0 siblings, 0 replies; 90+ messages in thread From: Michael Bender @ 2011-06-13 2:27 UTC (permalink / raw) To: Libusb Mailing List; +Cc: Theodore Kilgore, linux-usb, linux-media On Jun 12, 2011, at 7:03 PM, Xiaofan Chen wrote: > On Mon, Jun 13, 2011 at 5:20 AM, Theodore Kilgore > <kilgota@banach.math.auburn.edu> wrote: >> On Sun, 12 Jun 2011, Hans de Goede wrote: >>> Actually libusb and libgphoto have been using the rebind orginal >>> driver >>> functionality of the code for quite a while now, >> >> Oh? I can see that libusb is doing that, and I can also see that >> there is >> a "public" function for _unbinding_ a kernel driver, namely >> >> int usb_detach_kernel_driver_np() >> >> found in usb.h >> >> and it is used in libgphoto, as well. >> >> I am not sure that there is any corresponding rebind function which >> is >> public. Is it perhaps >> >> int usb_get_driver_np() >> >> ??? >> >> By context (looking at libgphoto2-port/usb/libusb.c) I would think >> that >> this function is not the rebind function, but is only checking >> whether or >> not there is any potential conflict with a kernel driver. If I am >> right, >> then where is the publicly exported rebind function, and where does >> it >> currently get used in libgphoto2? > > http://gphoto.svn.sourceforge.net/viewvc/gphoto/trunk/libgphoto2/libgphoto2_port/usb/libusb.c?revision=13652&view=markup > > The rebind happened under the function "static int gp_port_usb_close > (GPPort *port)". > Since libgphoto2 is still using libusb-0.1, the unbind is using > usbfs IOCTL > directly (USBDEVFS_CONNECT). > >> So frankly after my eagerness yesterday I do not see how it can >> easily be >> made to work, after all. >> >>> unfortunately this >>> does not solve the problem, unless we somehow move to 1 central >>> coordinator for the device the user experience will stay subpar. > > Now I understand what Hans is saying. It will be a lot of work trying > to sort out this issue in userspace. What can be the single central > coordinator? A device manager applet listing the program or service > which hold the device? Something like that. Have a user-space device allocation mechanism so that apps are not constrained by a particular interface semantic (i.e. not required to open /dev/video and have a kernel driver deliver pixels to the apps). Or hid that behind a library API and let instances of the library coordinate with each other; as long as an app uses the documented public library API to access a webcam, then everyone will play fair with each other. Look at the mess that this userspace/kernel driver issue has for code in an application like gphoto - that's insane that a photo manipulation app needs to know anything about userspace/kernel switching! >>> Example, user downloads pictures from the camera using shotwell, >>> gthumb, fspot or whatever, keeps the app in question open and the >>> app >>> in question keeps the gphoto2 device handle open. >>> >>> User wants to do some skyping with video chat, skype complains it >>> cannot find the device, since the kernel driver currently is >>> unbound. >>> >>> -> Poor user experience. >> >> Poor user experience, or merely poor user? The user ought to know >> better. >> Of course, I do agree that there are lots of such people, and it is >> a good >> idea to try to put up warning signs. > > It is difficult to call the users "poor users" in this case. Since > they may > not know that the other open program is holding the device. Some > warning message may help, not "I can not find the device" though. It > would be better to pinpoint which program is holding the device > and then ask the user to close that program. I understand this is > easily said than done... > > Similar experiences for Windows about the serial port, sometimes > it is difficult for the user to know that some program or service > are holding the serial port so that the other program or will fail or > Windows complain that it is still open when you want to undock > the computer. > >>> >>> With having both functions in the kernel, the kernel could actually >>> allow skype to use the dual mode cameras as video source, and if >>> the user then were to switch to f-spot and try to import more >>> photo's >>> then he will get an -ebusy in f-spot. If he finishes skyping and >>> then returns to f-spot everything will just continue working. >>> >>> This is the kind of "seamless" user experience I'm aiming for here. >>> >> Yes, I can see where you are coming from. But if the camera really >> will >> not let you run skype and fspot at the same time, which I do not >> believe >> it would allow on _any_ operating system, then each app should give >> an >> error message which says it cannot be run unless and until the >> other app >> has been closed. If that has to happen at the kernel level, then OK. >> > > Yes. From what I read, to solve it in kernel or to solve it in user > space > are both a lot of work. Yes and a kernel-based solution locks you into a kernel-based webcam driver paradigm, or an even uglier loopback driver. > Personally I tend to think to solve it in user space is more feasible. I agree. mike ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-11 9:15 ` Hans de Goede 2011-06-11 16:19 ` Theodore Kilgore @ 2011-06-11 16:57 ` Alan Stern 1 sibling, 0 replies; 90+ messages in thread From: Alan Stern @ 2011-06-11 16:57 UTC (permalink / raw) To: Hans de Goede Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Sat, 11 Jun 2011, Hans de Goede wrote: > >> So what do we need to make this situation better: > >> 1) A usb_driver callback alternative to the disconnect callback, > >> I propose to call this soft_disconnect. This serves 2 purposes > >> a) It will allow the driver to tell the caller that that is not > >> a good idea by returning an error code (think usb mass storage > >> driver and mounted filesystem > > > > Not feasible. usb-storage has no idea whether or not a device it > > controls has a mounted filesystem. (All it does is send SCSI commands > > to a device and get back the results.) Since that's the main use > > case you're interested in, this part of the proposal seems destined to > > fail. > > > > This is not completely true, I cannot rmmod usb-storage as long as > disks using it are mounted. I know this is done through the global > module usage count, so this is not per usb-storage device. But extending > the ref counting to be per usb-storage device should not be hard. > > All the accounting is already done for this. It would be harder than you think. All the accounting is _not_ already being done. What you're talking about would amount to a significant change in the driver model core and the SCSI core. It isn't just a USB thing. > > But userspace _does_ know where the mounted filesystems are. > > Therefore userspace should be responsible for avoiding programs that > > want to take control of devices holding these filesystems. That's the > > reason why usbfs device nodes are owned by root and have 0644 mode; > > there're can be written to only by programs with superuser privileges > > -- and such programs are supposed to be careful about what they do. > > > > Yes, and what I'm asking for is for an easy way for these programs to > be careful. A way for them to ask the kernel, which in general is > responsible for things like this and traditionally does resource > management and things which come with that like refcounting: "unbind > the driver from this device unless the device is currently in use". Sure. At the moment the kernel does not keep track of whether a device is currently in use -- at least, not in the way you mean. I'm not saying this can't be done. But it would be a bigger job than you think, and this isn't the appropriate thread to discuss it. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede 2011-06-10 8:22 ` Felipe Balbi 2011-06-10 14:48 ` Alan Stern @ 2011-06-10 18:16 ` Theodore Kilgore 2011-06-10 18:34 ` Felipe Balbi 2 siblings, 1 reply; 90+ messages in thread From: Theodore Kilgore @ 2011-06-10 18:16 UTC (permalink / raw) To: Hans de Goede Cc: linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Hans de Goede wrote: > Hi all, > > The current API for managing kernel -> userspace is a bit > rough around the edges, so I would like to discuss extending > the API. [...] > 2) So called dual mode cameras are (cheap) stillcams often even > without an lcdscreen viewfinder, and battery backed sram instead > of flash, which double as a webcam. We have drivers for both the > stillcam function of these (in libgphoto2, so using usbfs) as > well as for the webcam function (v4l2 kernel drivers). > > These drivers work well, and are mature. Yet the user experience > is rather poor. Under gnome the still-cam contents will be > automatically be made available as a "drive" using a gvfs-gphoto2 fuse > mount. This however involves sending a disconnect to the v4l2 kernel > driver, and thus the /dev/video# node disappearing. So if a user > wants to use the device as a webcam he/she needs to first go to > nautilus and unmount the gvfs mount. Until that is done the user will > simply get a message from an app like cheese that he has no webcam, > not even an ebusy error, just that he has no such device. > > Again not good. [...] As I have been involved in writing the drivers (both the kernel and the libgphoto2 drivers) for many of the affected cameras, perhaps I should expand on this problem. There are lots of responses to this original message of Hans. I will try to take some of their comments into account, below. First, some background. 1. All of the cameras in question have only one USB Vendor:Product number. In this sense, they are not "good citizens" which have different Product numbers for the two distinct modes of functioning. Thus, the problems are from that standpoint unavoidable. 2. Until recently in the history of Linux, there was an irreconcilable conflict. If a kernel driver for the video streaming mode was present and installed, it was not possible to use the camera in stillcam mode at all. Thus the only solution to the problem was to blacklist the kernel module so that it would not get loaded automatically and only to install said module by hand if the camera were to be used in video streaming mode, then to rmmod it immediately afterwards. Very cumbersome, obviously. 3. The current state of affairs is an advance on (2) but it is still inelegant. What happens now is, libusb has been revised in such a way that the kernel module is disabled (though still present) if a userspace driver (libgphoto2) wants to access the device (the camera). If it is desired to do video streaming after that, the camera needs to be re-plugged, which then causes the module to be automatically re-loaded. 4. Hans is absolutely correct about the problem with certain Gnome apps (and certain distros which blindly use them), which load up the libgphoto2 driver for the camera as soon as it is detected. The consequence (cf. item 3) is that the camera can never be used as a webcam. The only solution to that problem is to disable the automatic loading of the libgphoto2 driver. 5. It could be said that those who came up with the "user-friendly" "solution" described in (4) were not very clever, and perhaps they ought to fix their own mess. I would strongly agree that they ought to have thought before coding, as the result is not user-friendly in the least. 6. The question has been asked whether the cameras are always using the same interface. Typically, yes. The same altsetting? That depends on the camera. Some of them use isochronous transport for streaming, and some of them rely exclusively upon bulk transport. Those which use bulk transport only are confined to altsetting 0. Some possible solutions? Well, first of all it needs to be understood that the problem originates as a bad feature of something good, namely the rigid separation of kernelspace and userspace functionality which is an integral part of the Linux security model. Some other operating systems are not so careful, and thus they do not have a problem about supporting dual-mode hardware. Second, it appears to me that the problem is most appropriately addressed from the userspace side and perhaps also from the kernelspace side. In userspace, it would be a really good idea if those who are attempting to write user-friendly apps and to create user-friendly distros would actually learn some of the basics of Linux, such as the rudiments of the security model. Since dual-mode cameras are known to exist, it would have been appropriate, when one is detected, to pop up a dialog window which asks the user to choose a webcam app or a stillcam app. Even this minor innovation would stop a lot of user grief. Frankly, I am mystified that this was not done. This still leaves the problem (see item 3, above) that a dual-mode camera needs to be re-plugged in order to re-enable the access to /dev/video* if it has been used in stillcam mode. If it is possible to do a re-plug in software, this would help a lot. It does not matter if the re-plug is done in userspace or in kernelspace, so long as it can be done, somehow, after libusb relinquishes the camera. Or, fix things up somehow so that the kernel module automatically re-activates itself when the userspace app (using libgphoto2) is closed down. Finally, another possible alternative would be to figure out how to do something in the kernel module which permits the camera to be accessed by libusb. If there is going to be discussion about this problem in Vancouver, I am really sorry that I will probably be able to attend. Classes for the Fall semester begin at Auburn University on August 17. Nevertheless, this is a problem in which I have been interested for a long time. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 18:16 ` Theodore Kilgore @ 2011-06-10 18:34 ` Felipe Balbi 2011-06-10 21:18 ` Alan Stern 2011-06-10 22:43 ` Theodore Kilgore 0 siblings, 2 replies; 90+ messages in thread From: Felipe Balbi @ 2011-06-10 18:34 UTC (permalink / raw) To: Theodore Kilgore Cc: Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 6682 bytes --] Hi, On Fri, Jun 10, 2011 at 01:16:47PM -0500, Theodore Kilgore wrote: > As I have been involved in writing the drivers (both the kernel and the > libgphoto2 drivers) for many of the affected cameras, perhaps I should > expand on this problem. There are lots of responses to this original > message of Hans. I will try to take some of their comments into account, > below. First, some background. > > 1. All of the cameras in question have only one USB Vendor:Product number. > In this sense, they are not "good citizens" which have different Product there's nothing in the USB spec that says you need different product IDs for different modes of operation. No matter if it's still or webcam configuration, the underlying function is the same: capture images using a set of lenses and image sensor. > numbers for the two distinct modes of functioning. Thus, the problems are > from that standpoint unavoidable. I don't see any problems in this situation. If, for that particular product, webcam and still image functionality are mutually exclusive, then that's how the product (and their drivers) have to work. If the linux community decided to put webcam functionality in kernel and still image functionality on a completely separate driver, that's entirely our problem. > 2. Until recently in the history of Linux, there was an irreconcilable > conflict. If a kernel driver for the video streaming mode was present and > installed, it was not possible to use the camera in stillcam mode at all. > Thus the only solution to the problem was to blacklist the kernel module > so that it would not get loaded automatically and only to install said > module by hand if the camera were to be used in video streaming mode, then > to rmmod it immediately afterwards. Very cumbersome, obviously. true... but why couldn't we combine both in kernel or in userspace altogether ? Why do we have this split ? (words from a newbie in V4L2, go easy :-p) > 3. The current state of affairs is an advance on (2) but it is still > inelegant. What happens now is, libusb has been revised in such a way that > the kernel module is disabled (though still present) if a userspace driver > (libgphoto2) wants to access the device (the camera). If it is desired to > do video streaming after that, the camera needs to be re-plugged, which > then causes the module to be automatically re-loaded. It's still wrong. This should be just another USB_REQ_SET_CONFIGURATION(). If this is was just one single driver, you could easily do that. > 4. Hans is absolutely correct about the problem with certain Gnome apps > (and certain distros which blindly use them), which load up the libgphoto2 > driver for the camera as soon as it is detected. The consequence (cf. item > 3) is that the camera can never be used as a webcam. The only solution to > that problem is to disable the automatic loading of the libgphoto2 driver. Or, to move the libgphoto2 driver to kernel, combine it in the same driver that handles streaming. No ? > 5. It could be said that those who came up with the "user-friendly" > "solution" described in (4) were not very clever, and perhaps they ought > to fix their own mess. I would strongly agree that they ought to have > thought before coding, as the result is not user-friendly in the least. I agree here > 6. The question has been asked whether the cameras are always using the > same interface. Typically, yes. The same altsetting? That depends on the > camera. Some of them use isochronous transport for streaming, and some of > them rely exclusively upon bulk transport. Those which use bulk transport > only are confined to altsetting 0. the transfer type or the way the configurations are setup shouldn't matter much for the end functionality. > Some possible solutions? > > Well, first of all it needs to be understood that the problem originates > as a bad feature of something good, namely the rigid separation of > kernelspace and userspace functionality which is an integral part of the > Linux security model. Some other operating systems are not so careful, and > thus they do not have a problem about supporting dual-mode hardware. You'd need to describe this fully. What's the problem in having one driver handle all modes of operation of that particular camera ? Sounds like a decision from V4L2 folks not to take still image cameras in kernel. Am I wrong ? > Second, it appears to me that the problem is most appropriately addressed > from the userspace side and perhaps also from the kernelspace side. > > In userspace, it would be a really good idea if those who are attempting > to write user-friendly apps and to create user-friendly distros would > actually learn some of the basics of Linux, such as the rudiments of the > security model. Since dual-mode cameras are known to exist, it would have > been appropriate, when one is detected, to pop up a dialog window which > asks the user to choose a webcam app or a stillcam app. Even this minor > innovation would stop a lot of user grief. Frankly, I am mystified that > this was not done. I think this is still not good. It should be more "fluid". If the camera has separate alternate settings or different configurations, all it takes (from a USB standpoint) is to send the correct SetConfiguration or SetInterface command and the camera should happily change its mode of operation. Which means that a simple ioctl() to change the mode of operation would suffice should the driver be combined. > This still leaves the problem (see item 3, above) that a dual-mode camera > needs to be re-plugged in order to re-enable the access to /dev/video* if > it has been used in stillcam mode. If it is possible to do a re-plug in > software, this would help a lot. It does not matter if the re-plug is done > in userspace or in kernelspace, so long as it can be done, somehow, after > libusb relinquishes the camera. Or, fix things up somehow so that the > kernel module automatically re-activates itself when the userspace app > (using libgphoto2) is closed down. > > Finally, another possible alternative would be to figure out how to do > something in the kernel module which permits the camera to be accessed by > libusb. I don't see the possibility of combining both drivers and use /dev/videoX for streaming and something like /dev/camX for still image. Now, the only thing you have to be sure in kernel space is that you don't allow both to be accessed together, but there are ways to handle it. -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 18:34 ` Felipe Balbi @ 2011-06-10 21:18 ` Alan Stern 2011-06-10 21:46 ` Felipe Balbi 2011-06-10 22:46 ` Theodore Kilgore 2011-06-10 22:43 ` Theodore Kilgore 1 sibling, 2 replies; 90+ messages in thread From: Alan Stern @ 2011-06-10 21:18 UTC (permalink / raw) To: Felipe Balbi Cc: Theodore Kilgore, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Felipe Balbi wrote: > I don't see any problems in this situation. If, for that particular > product, webcam and still image functionality are mutually exclusive, > then that's how the product (and their drivers) have to work. > > If the linux community decided to put webcam functionality in kernel and > still image functionality on a completely separate driver, that's > entirely our problem. And the problem is how to coordinate the two of them. > > 2. Until recently in the history of Linux, there was an irreconcilable > > conflict. If a kernel driver for the video streaming mode was present and > > installed, it was not possible to use the camera in stillcam mode at all. > > Thus the only solution to the problem was to blacklist the kernel module > > so that it would not get loaded automatically and only to install said > > module by hand if the camera were to be used in video streaming mode, then > > to rmmod it immediately afterwards. Very cumbersome, obviously. > > true... but why couldn't we combine both in kernel or in userspace > altogether ? Why do we have this split ? (words from a newbie in V4L2, > go easy :-p) I think the problem may be that the PTP protocol used in the still-cam mode isn't suitable for a kernel driver. Or if it is suitable, it would have to be something like a shared-filesystem driver -- nothing like a video driver. You certainly wouldn't want to put it in V4L2. > Or, to move the libgphoto2 driver to kernel, combine it in the same > driver that handles streaming. No ? No. Something else is needed. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 21:18 ` Alan Stern @ 2011-06-10 21:46 ` Felipe Balbi 2011-06-10 22:46 ` Theodore Kilgore 1 sibling, 0 replies; 90+ messages in thread From: Felipe Balbi @ 2011-06-10 21:46 UTC (permalink / raw) To: Alan Stern Cc: Felipe Balbi, Theodore Kilgore, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 1167 bytes --] Hi, On Fri, Jun 10, 2011 at 05:18:39PM -0400, Alan Stern wrote: > > > 2. Until recently in the history of Linux, there was an irreconcilable > > > conflict. If a kernel driver for the video streaming mode was present and > > > installed, it was not possible to use the camera in stillcam mode at all. > > > Thus the only solution to the problem was to blacklist the kernel module > > > so that it would not get loaded automatically and only to install said > > > module by hand if the camera were to be used in video streaming mode, then > > > to rmmod it immediately afterwards. Very cumbersome, obviously. > > > > true... but why couldn't we combine both in kernel or in userspace > > altogether ? Why do we have this split ? (words from a newbie in V4L2, > > go easy :-p) > > I think the problem may be that the PTP protocol used in the still-cam > mode isn't suitable for a kernel driver. Or if it is suitable, it > would have to be something like a shared-filesystem driver -- nothing > like a video driver. You certainly wouldn't want to put it in V4L2. ach true.. Had forgotten that detail. Needs more thinking. -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 21:18 ` Alan Stern 2011-06-10 21:46 ` Felipe Balbi @ 2011-06-10 22:46 ` Theodore Kilgore 1 sibling, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-06-10 22:46 UTC (permalink / raw) To: Alan Stern Cc: Felipe Balbi, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Alan Stern wrote: > On Fri, 10 Jun 2011, Felipe Balbi wrote: > > > I don't see any problems in this situation. If, for that particular > > product, webcam and still image functionality are mutually exclusive, > > then that's how the product (and their drivers) have to work. > > > > If the linux community decided to put webcam functionality in kernel and > > still image functionality on a completely separate driver, that's > > entirely our problem. > > And the problem is how to coordinate the two of them. > > > > 2. Until recently in the history of Linux, there was an irreconcilable > > > conflict. If a kernel driver for the video streaming mode was present and > > > installed, it was not possible to use the camera in stillcam mode at all. > > > Thus the only solution to the problem was to blacklist the kernel module > > > so that it would not get loaded automatically and only to install said > > > module by hand if the camera were to be used in video streaming mode, then > > > to rmmod it immediately afterwards. Very cumbersome, obviously. > > > > true... but why couldn't we combine both in kernel or in userspace > > altogether ? Why do we have this split ? (words from a newbie in V4L2, > > go easy :-p) > > I think the problem may be that the PTP protocol used in the still-cam > mode isn't suitable for a kernel driver. Or if it is suitable, it > would have to be something like a shared-filesystem driver -- nothing > like a video driver. You certainly wouldn't want to put it in V4L2. > > > Or, to move the libgphoto2 driver to kernel, combine it in the same > > driver that handles streaming. No ? > > No. Something else is needed. > > Alan Stern > Agreed. Something else is needed. But what? Also, very good point about PTP. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 18:34 ` Felipe Balbi 2011-06-10 21:18 ` Alan Stern @ 2011-06-10 22:43 ` Theodore Kilgore 2011-06-11 1:30 ` Xiaofan Chen 2011-06-13 9:05 ` Felipe Balbi 1 sibling, 2 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-06-10 22:43 UTC (permalink / raw) To: Felipe Balbi Cc: Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart On Fri, 10 Jun 2011, Felipe Balbi wrote: > Hi, > > On Fri, Jun 10, 2011 at 01:16:47PM -0500, Theodore Kilgore wrote: > > As I have been involved in writing the drivers (both the kernel and the > > libgphoto2 drivers) for many of the affected cameras, perhaps I should > > expand on this problem. There are lots of responses to this original > > message of Hans. I will try to take some of their comments into account, > > below. First, some background. > > > > 1. All of the cameras in question have only one USB Vendor:Product number. > > In this sense, they are not "good citizens" which have different Product > > there's nothing in the USB spec that says you need different product IDs > for different modes of operation. No matter if it's still or webcam > configuration, the underlying function is the same: capture images using > a set of lenses and image sensor. True, true. But I will add that most of these cameras are Class 255, Subclass 255, Protocol 255 (Proprietary, Proprietary, Proprietary). > > > numbers for the two distinct modes of functioning. Thus, the problems are > > from that standpoint unavoidable. > > I don't see any problems in this situation. If, for that particular > product, webcam and still image functionality are mutually exclusive, > then that's how the product (and their drivers) have to work. Yes. > > If the linux community decided to put webcam functionality in kernel and > still image functionality on a completely separate driver, that's > entirely our problem. As I understand, the basic reason why webcam functionality needs to be in the kernel is speed. Quick reaction time, and faster data transmission. Most but not all webcams use isochronous data transport. The ones which do not, cannot get as high a rate of frames per second. Isochronous data transport has historically seemed to need kernel support in order to work properly. Related to this, libusb has not supported isochronous data transport. But still cameras use bulk transport. This is natural because one needs to move data which is stored on the camera to a place on the computer where it can be processed (if needed) and stored. There is no impediment to userspace talking to a USB device, so why not go ahead and do the job that way if the kernel is not needed? The second thing to mention is that libgphoto2 at this point is supporting well over 1,000 cameras. True, there are not 1,000 driver libraries, but there are quite a few. My impression is that nobody wants to put stuff like that in the kernel unless it is absolutely necessary, just because some of those cameras are dual-mode cameras. Don't people complain on a fairly regular basis about kernel code bloat? > > > 2. Until recently in the history of Linux, there was an irreconcilable > > conflict. If a kernel driver for the video streaming mode was present and > > installed, it was not possible to use the camera in stillcam mode at all. > > Thus the only solution to the problem was to blacklist the kernel module > > so that it would not get loaded automatically and only to install said > > module by hand if the camera were to be used in video streaming mode, then > > to rmmod it immediately afterwards. Very cumbersome, obviously. > > true... but why couldn't we combine both in kernel or in userspace > altogether ? Why do we have this split ? (words from a newbie in V4L2, > go easy :-p) See above. > > 3. The current state of affairs is an advance on (2) but it is still > > inelegant. What happens now is, libusb has been revised in such a way that > > the kernel module is disabled (though still present) if a userspace driver > > (libgphoto2) wants to access the device (the camera). If it is desired to > > do video streaming after that, the camera needs to be re-plugged, which > > then causes the module to be automatically re-loaded. > > It's still wrong. This should be just another > USB_REQ_SET_CONFIGURATION(). If this is was just one single driver, you > could easily do that. Well, I think it ought to be possible anyway. Up to that point, I agree. Though I confess I do not exactly know how. > > 4. Hans is absolutely correct about the problem with certain Gnome apps > > (and certain distros which blindly use them), which load up the libgphoto2 > > driver for the camera as soon as it is detected. The consequence (cf. item > > 3) is that the camera can never be used as a webcam. The only solution to > > that problem is to disable the automatic loading of the libgphoto2 driver. I would only add to this, that it is not pleasant to have to deal with frustrated users. At this point, my usual advice is that this is a distro problem. Ask your distro for help. > > Or, to move the libgphoto2 driver to kernel, combine it in the same > driver that handles streaming. No ? See above. > > > 5. It could be said that those who came up with the "user-friendly" > > "solution" described in (4) were not very clever, and perhaps they ought > > to fix their own mess. I would strongly agree that they ought to have > > thought before coding, as the result is not user-friendly in the least. > > I agree here Excellent. I really did have to restrain myself when writing that, too. > > > 6. The question has been asked whether the cameras are always using the > > same interface. Typically, yes. The same altsetting? That depends on the > > camera. Some of them use isochronous transport for streaming, and some of > > them rely exclusively upon bulk transport. Those which use bulk transport > > only are confined to altsetting 0. > > the transfer type or the way the configurations are setup shouldn't > matter much for the end functionality. > > > Some possible solutions? > > > > Well, first of all it needs to be understood that the problem originates > > as a bad feature of something good, namely the rigid separation of > > kernelspace and userspace functionality which is an integral part of the > > Linux security model. Some other operating systems are not so careful, and > > thus they do not have a problem about supporting dual-mode hardware. > > You'd need to describe this fully. Certain other operating systems are not fussy at all about their device drivers. Those operating systems are also notorious for bad security and for crashing, and one of the things which gets blamed for that is all those hardware drivers from the hardware manufacturers. Linux tries to do things right, which clearly can also cause problems. What's the problem in having one > driver handle all modes of operation of that particular camera ? Sounds > like a decision from V4L2 folks not to take still image cameras in > kernel. Am I wrong ? Still image cameras can be handled perfectly well in userspace. There is no need for kernel support, at all. Moreover, many still cameras will not stream at all; they simply were not designed, built, or sold to do that. As far as V4L2 was concerned, it was developed in order to meet the need for supporting webcams. Clearly, that was a conscious decision. But you ask whether it is possible to support a dual-mode camera entirely in V4L2. I would say that of course it ought to be possible, but that would require a large portion of the functionality of Gphoto to be duplicated in V4L2, just to support those dual-mode cameras. Having some experience with both Gphoto and V4L2, I have to say that I am not very taken with the idea. There has to be a way which is not so arduous, one would hope. > > > Second, it appears to me that the problem is most appropriately addressed > > from the userspace side and perhaps also from the kernelspace side. > > > > In userspace, it would be a really good idea if those who are attempting > > to write user-friendly apps and to create user-friendly distros would > > actually learn some of the basics of Linux, such as the rudiments of the > > security model. Since dual-mode cameras are known to exist, it would have > > been appropriate, when one is detected, to pop up a dialog window which > > asks the user to choose a webcam app or a stillcam app. Even this minor > > innovation would stop a lot of user grief. Frankly, I am mystified that > > this was not done. > > I think this is still not good. It should be more "fluid". If the camera > has separate alternate settings or different configurations, all it > takes (from a USB standpoint) is to send the correct SetConfiguration or > SetInterface command and the camera should happily change its mode of > operation. Often true, but not necessarily. > > Which means that a simple ioctl() to change the mode of operation would > suffice should the driver be combined. > > > This still leaves the problem (see item 3, above) that a dual-mode camera > > needs to be re-plugged in order to re-enable the access to /dev/video* if > > it has been used in stillcam mode. If it is possible to do a re-plug in > > software, this would help a lot. It does not matter if the re-plug is done > > in userspace or in kernelspace, so long as it can be done, somehow, after > > libusb relinquishes the camera. Or, fix things up somehow so that the > > kernel module automatically re-activates itself when the userspace app > > (using libgphoto2) is closed down. > > > > Finally, another possible alternative would be to figure out how to do > > something in the kernel module which permits the camera to be accessed by > > libusb. > > I don't see the possibility of combining both drivers and use > /dev/videoX for streaming and something like /dev/camX for still image. > Now, the only thing you have to be sure in kernel space is that you > don't allow both to be accessed together, but there are ways to handle > it. Which leaves a problem for still cameras. Most of them, in fact, do not need any such /dev/camX as you are proposing, at all. The only cameras which need the kernel are those which need /dev/videoX. For pure still cameras, /dev/videoX is entirely superfluous, at best. Dual-mode cameras only need /dev/videoX when streaming and otherwise need for /dev/videoX to go away. I do not believe that we have found the optimal solution, yet. The ideal thing would be some kind of hack which allows the kernel to be used when it is needed, and when it is not needed it does not interfere. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 22:43 ` Theodore Kilgore @ 2011-06-11 1:30 ` Xiaofan Chen 2011-06-11 4:17 ` Theodore Kilgore 2011-06-13 9:05 ` Felipe Balbi 1 sibling, 1 reply; 90+ messages in thread From: Xiaofan Chen @ 2011-06-11 1:30 UTC (permalink / raw) To: Theodore Kilgore; +Cc: linux-usb, linux-media, libusb-devel On Sat, Jun 11, 2011 at 6:43 AM, Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote: > I do not believe that we have found the optimal solution, yet. The ideal > thing would be some kind of hack which allows the kernel to be used when > it is needed, and when it is not needed it does not interfere. Just wondering if you can use libusb-1.0 for the user space still image functionality. libusb-1.0 offers the following functions to do that for you under Linux. int libusb_kernel_driver_active (libusb_device_handle *dev, int interface) Determine if a kernel driver is active on an interface. int libusb_detach_kernel_driver (libusb_device_handle *dev, int interface) Detach a kernel driver from an interface. int libusb_attach_kernel_driver (libusb_device_handle *dev, int interface) Re-attach an interface's kernel driver, which was previously detached using libusb_detach_kernel_driver(). So you can detach the kernel v4l2 driver at the beginning and later re-attach it when you finish. -- Xiaofan ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-11 1:30 ` Xiaofan Chen @ 2011-06-11 4:17 ` Theodore Kilgore 0 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-06-11 4:17 UTC (permalink / raw) To: Xiaofan Chen; +Cc: linux-usb, linux-media, libusb-devel On Sat, 11 Jun 2011, Xiaofan Chen wrote: > On Sat, Jun 11, 2011 at 6:43 AM, Theodore Kilgore > <kilgota@banach.math.auburn.edu> wrote: > > I do not believe that we have found the optimal solution, yet. The ideal > > thing would be some kind of hack which allows the kernel to be used when > > it is needed, and when it is not needed it does not interfere. > > Just wondering if you can use libusb-1.0 for the user space still image > functionality. > > libusb-1.0 offers the following functions to do that for you under Linux. > > int libusb_kernel_driver_active (libusb_device_handle *dev, int interface) > Determine if a kernel driver is active on an interface. > int libusb_detach_kernel_driver (libusb_device_handle *dev, int interface) > Detach a kernel driver from an interface. > int libusb_attach_kernel_driver (libusb_device_handle *dev, int interface) > Re-attach an interface's kernel driver, which was previously > detached using libusb_detach_kernel_driver(). > > So you can detach the kernel v4l2 driver at the beginning and later > re-attach it when you finish. Well, then, this solves the problem, doesn't it? Of course, those who deal with creating those "simple" and "user-friendly" GUI environments would probably still do well if they would open a dialog box for dual-mode hardware. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-10 22:43 ` Theodore Kilgore 2011-06-11 1:30 ` Xiaofan Chen @ 2011-06-13 9:05 ` Felipe Balbi 2011-06-13 13:06 ` Mauro Carvalho Chehab 1 sibling, 1 reply; 90+ messages in thread From: Felipe Balbi @ 2011-06-13 9:05 UTC (permalink / raw) To: Theodore Kilgore Cc: Felipe Balbi, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Mauro Carvalho Chehab, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 603 bytes --] Hi, On Fri, Jun 10, 2011 at 05:43:06PM -0500, Theodore Kilgore wrote: > > there's nothing in the USB spec that says you need different product IDs > > for different modes of operation. No matter if it's still or webcam > > configuration, the underlying function is the same: capture images using > > a set of lenses and image sensor. > > True, true. But I will add that most of these cameras are Class 255, > Subclass 255, Protocol 255 (Proprietary, Proprietary, Proprietary). well, if the manufacturer doesn't want to implement UVC for whatever reason, it's his call ;-) -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-13 9:05 ` Felipe Balbi @ 2011-06-13 13:06 ` Mauro Carvalho Chehab 2011-06-13 13:12 ` Felipe Balbi 0 siblings, 1 reply; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-06-13 13:06 UTC (permalink / raw) To: balbi Cc: Theodore Kilgore, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart Em 13-06-2011 06:05, Felipe Balbi escreveu: > Hi, > > On Fri, Jun 10, 2011 at 05:43:06PM -0500, Theodore Kilgore wrote: >>> there's nothing in the USB spec that says you need different product IDs >>> for different modes of operation. No matter if it's still or webcam >>> configuration, the underlying function is the same: capture images using >>> a set of lenses and image sensor. >> >> True, true. But I will add that most of these cameras are Class 255, >> Subclass 255, Protocol 255 (Proprietary, Proprietary, Proprietary). > > well, if the manufacturer doesn't want to implement UVC for whatever > reason, it's his call ;-) This argument is bogus. UVC were implemented too late. There are lots of chipsets that are not UVC-compliant, simply because there were no UVC at the time those chipsets were designed. Still today, newer devices using those chipsets are still at the market. This is the same as saying that we should not support USB 1.1 or USB 2.0 because they're not fully USB 3.0 compliant. Mauro. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: Improving kernel -> userspace (usbfs) usb device hand off 2011-06-13 13:06 ` Mauro Carvalho Chehab @ 2011-06-13 13:12 ` Felipe Balbi 0 siblings, 0 replies; 90+ messages in thread From: Felipe Balbi @ 2011-06-13 13:12 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: balbi, Theodore Kilgore, Hans de Goede, linux-usb, Sarah Sharp, linux-media, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart [-- Attachment #1: Type: text/plain, Size: 1283 bytes --] Hi, On Mon, Jun 13, 2011 at 10:06:58AM -0300, Mauro Carvalho Chehab wrote: > Em 13-06-2011 06:05, Felipe Balbi escreveu: > > Hi, > > > > On Fri, Jun 10, 2011 at 05:43:06PM -0500, Theodore Kilgore wrote: > >>> there's nothing in the USB spec that says you need different product IDs > >>> for different modes of operation. No matter if it's still or webcam > >>> configuration, the underlying function is the same: capture images using > >>> a set of lenses and image sensor. > >> > >> True, true. But I will add that most of these cameras are Class 255, > >> Subclass 255, Protocol 255 (Proprietary, Proprietary, Proprietary). > > > > well, if the manufacturer doesn't want to implement UVC for whatever > > reason, it's his call ;-) > > This argument is bogus. > > UVC were implemented too late. There are lots of chipsets that are not UVC-compliant, > simply because there were no UVC at the time those chipsets were designed. > > Still today, newer devices using those chipsets are still at the market. > > This is the same as saying that we should not support USB 1.1 or USB 2.0 > because they're not fully USB 3.0 compliant. I would think the small wink at the end was enough to label the reply as a joke. Apparently not :-) -- balbi [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-06-10 0:21 USB mini-summit at LinuxCon Vancouver Sarah Sharp ` (2 preceding siblings ...) 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede @ 2011-08-04 22:21 ` Mauro Carvalho Chehab 2011-08-04 22:56 ` Greg KH 3 siblings, 1 reply; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-04 22:21 UTC (permalink / raw) To: Sarah Sharp Cc: linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Greg KH, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Theodore Kilgore, Adam Baker Hi Sarah/Greg, Em 09-06-2011 21:21, Sarah Sharp escreveu: > I'm pleased to announce a USB mini-summit at LinuxCon Vancouver. > > What: USB mini-summit > When: Tuesday, August 16th, all day > Where: At the conference venue, room TBD pending confirmation from > Angela Brown. > > Proposed topics include USB virtualization, and improved bandwidth APIs > between the USB core and drivers (especially webcam drivers). See the > detailed topic list below. Anyone is also welcome to propose or show up > with a USB related topic. MUSB? USB 3.0 gadget drivers? USB-IP? > > The USB mini-summit does overlap with the virtualization mini-summit by > a day, but I'm hoping we can schedule talks so some of the > virtualization folks can make it to the USB mini-summit. The other > option was on Friday during the conference which was not ideal. > > Proposed topics: > > Topic 1 > ------- > > The KVM folks suggested that it would be good to get USB and > virtualization developers together to talk about how to virtualize the > xHCI host controller. The xHCI spec architect worked closely with > VMWare to get some extra goodies in the spec to help virtualization, and > I'd like to see the other virtualization developers take advantage of > that. I'd also like us to hash out any issues they have been finding in > the USB core or xHCI driver during the virtualization effort. > > > Topic 2 > ------- > > I'd also like to get the V4L and audio developers who work with USB > devices together with the core USB folks to talk about bandwidth > management under xHCI. > > One of the issues is that since the xHCI hardware does bandwidth > management, not the xHCI driver, a schedule that will take too much > bandwidth will get rejected much sooner than any USB driver currently > expects (during a call to usb_set_interface). This poses issues, since > most USB video drivers negotiate the video size and frame rate after > they call usb_set_interface, so they don't know whether they can fall > back to a less bandwidth-intensive setting. Currently, they just submit > URBs with less and less bandwidth until one interval setting gets > accepted that won't work under xHCI. > > A second issue is that that some drivers need less bandwidth than the > device advertises, and the xHCI driver currently uses whatever periodic > interval the device advertises in its descriptors. This is not what the > video/audio driver wants, especially in the case of buggy high speed > devices that advertise the interval in frames, not microframes. There > needs to be some way for the drivers to communicate their bandwidth > needs to the USB core. We've known about this issue for a while, and I > think it's time to get everyone in the same room and hash out an API. > > (I will send out an API proposal later this month.) While discussing the topics for the media workshop that will happen together with this year's KS/2011, one issue related to the USB stack came on our discussions: it is related to multi-function USB devices, as you can see on this thread: http://www.spinics.net/lists/linux-media/msg36195.html We have several cases of multi-function devices at the media subsystem. For example, most TV grabber devices provides Remote Controller, Video, audio and MPEG streaming capabilities, either implementing the standard USB API's, or the vendor class API's. There are even some devices that support USB storage, 3G modem and Digital TV. Some of those devices have some resources that are mutually exclusive. For example, some Digital Cameras can provide either access to the stored images, or can be used as webcams. Using them as webcams automatically deletes all stored images from it. So, a properly implemented driver should be returning -EBUSY (or -EPERM?) if someone tries to stream for such devices if is there any pictures stored on it. Symmetrically, accessing the stored pictures there should return -EBUSY if the device is streaming. Currently, the data access for Digital Cameras is implemented via libusb, libgphoto and gvfs, while the streaming interface is implemented via a gspca Kernel driver. I know that this problem were somewhat solved for 3G modems, with the usage of the userspace problem usb_modeswitch, and with some quirks for the USB storage driver, but I'm not sure if such tricks will scale forever, as more functions are seen on some USB devices. So, if we have some time, maybe we could start some discussions about that during the USB mini-summit. It should be noticed that the media subsystem has currently similar problems on devices that provide both analog and digital TV, as some resources, like the tuner can't be used simultaneously by the two API's. This happens even with PCI devices. So, maybe, in the end, we'll be writing some kernel library for resource locking, but it would be good if we can have preliminary discussions there, and let the final discussions to happen during KS/2011. What do you think? Thanks! Mauro ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-04 22:21 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab @ 2011-08-04 22:56 ` Greg KH [not found] ` <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com> ` (3 more replies) 0 siblings, 4 replies; 90+ messages in thread From: Greg KH @ 2011-08-04 22:56 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Theodore Kilgore, Adam Baker On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > I know that this problem were somewhat solved for 3G modems, with the usage > of the userspace problem usb_modeswitch, and with some quirks for the USB > storage driver, but I'm not sure if such tricks will scale forever, as more > functions are seen on some USB devices. Well, no matter how it "scales" it needs to be done in userspace, like usb_modeswitch does. We made that decision a while ago, and it is working out very well. I see no reason why you can't do it in userspace as well as that is the easiest place to control this type of thing. I thought we had a long discussion about this topic a while ago and came to this very conclusion. Or am I mistaken? thanks, greg k-h ^ permalink raw reply [flat|nested] 90+ messages in thread
[parent not found: <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com>]
* Re: USB mini-summit at LinuxCon Vancouver [not found] ` <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com> @ 2011-08-05 0:33 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-05 0:33 UTC (permalink / raw) To: Matthew Dharm Cc: Greg KH, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Theodore Kilgore, Adam Baker Em 04-08-2011 20:22, Matthew Dharm escreveu: > On Thu, Aug 4, 2011 at 3:56 PM, Greg KH <greg@kroah.com <mailto:greg@kroah.com>> wrote: > > On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > I know that this problem were somewhat solved for 3G modems, with the usage > > of the userspace problem usb_modeswitch, and with some quirks for the USB > > storage driver, but I'm not sure if such tricks will scale forever, as more > > functions are seen on some USB devices. > > Well, no matter how it "scales" it needs to be done in userspace, like > usb_modeswitch does. We made that decision a while ago, and it is > working out very well. I see no reason why you can't do it in userspace > as well as that is the easiest place to control this type of thing. > > I thought we had a long discussion about this topic a while ago and came > to this very conclusion. Or am I mistaken? > > > We keep having the discussion over and over again. But, you are correct: the conclusion was that this all needs to live in userspace. In the case of 3G modem x USB storage only, it is possible to handle it on userspace. However, when there are more functions added, an they're not (completely) mutually exclusive, then I don't see an easy way (if is there a way) for doing it at userspace. Several devices offer more than one function at the same time, but some resources are mutually exclusive. A TV stick with just one tuner, and both analog and digital demods offer both analog and digital TV at the same time. So, both analog and digital parts of the driver will offer the device to userspace. However, unpredictable results will happen if userspace tries to use both at the same time. The Digital camera devices that offer PTP transfers and V4L support also fall at the same type of trouble. Some of those devices just delete all pictures from the memory, if streaming is started. So, receiving an automatic Skype video call may delete all pictures you took. Worse than that, currently, the PTP protocol is handled via libusb, while streaming is done via V4L2 API. The best technical approach, IMO, is to implement the PTP protocol in kernelspace, and do some sort of inter-subsystem locking to prevent such troubles. Regards, Mauro ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-04 22:56 ` Greg KH [not found] ` <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com> @ 2011-08-05 2:56 ` Theodore Kilgore 2011-08-05 6:57 ` Oliver Neukum 2011-08-05 7:45 ` Hans de Goede 3 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-05 2:56 UTC (permalink / raw) To: Greg KH Cc: Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Adam Baker On Thu, 4 Aug 2011, Greg KH wrote: > On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > I know that this problem were somewhat solved for 3G modems, with the usage > > of the userspace problem usb_modeswitch, and with some quirks for the USB > > storage driver, but I'm not sure if such tricks will scale forever, as more > > functions are seen on some USB devices. > > Well, no matter how it "scales" it needs to be done in userspace, like > usb_modeswitch does. We made that decision a while ago, and it is > working out very well. I see no reason why you can't do it in userspace > as well as that is the easiest place to control this type of thing. > > I thought we had a long discussion about this topic a while ago and came > to this very conclusion. Or am I mistaken? > > thanks, > > greg k-h > Greg, A little bit more of precision would help me a lot, here. Precisely what is one supposed to do in userspace? I would naturally assume that the user will control the device by announcing what the user wants to do with the device. Such control would therefore have to start in userspace. But this is such an obvious truism that you must mean something else. What, then? To say to do things "like usb_modeswitch" is rather vague. For one thing, usb_modeswitch, being something out there in userspace, does _not_ affect what is in the kernel. However, what is (or is not) in the kernel might make it possible (or, in the alternative, impossible) to facilitate the action of some kind of userspace function-switching program similar to usb_modeswitch, not so? As to the "long discussion about this topic a while ago" that may have been a discussion in which I was also involved. For, I do remember a discussion about this topic a few months ago. In one respect my memory differs from yours, however: It was not my impression that any definite conclusions were reached, certainly not a consensus. BTW, I did try to lay out what I remembered as three alternatives that came up, along with arguments for and against each of them, in my message which kicked off this thread today. It is of course a real possibility that you have seen the perfect solution in your mind to these problems, which have vexed a lot of people for years, and others have not. If that is the way it is, then perhaps you just wish that we would all shut up, implement what is obvious, and we could all be happy. This could certainly be the case. You have written a lot more code than I have, and you obviously must have started doing that when you were at least 30 years younger, perhaps even 40 years younger, than I was when I started. So I would be the first to say that you are much better at it than I am. But there are others here, too, who seem actively to have been sucked into this discussion, and they are similarly younger and more clever than I am, too. Giving due consideration to this, it would probably get the job done a lot faster if you just take the time to describe what grand vision you yourself have in mind for the solution of the problem, with a sufficient accounting of the details that people can all see what it is and exactly how and why it would work absolutely perfectly, and then we can just get busy and do it. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-04 22:56 ` Greg KH [not found] ` <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com> 2011-08-05 2:56 ` Theodore Kilgore @ 2011-08-05 6:57 ` Oliver Neukum 2011-08-05 17:38 ` Theodore Kilgore 2011-08-05 7:45 ` Hans de Goede 3 siblings, 1 reply; 90+ messages in thread From: Oliver Neukum @ 2011-08-05 6:57 UTC (permalink / raw) To: Greg KH Cc: Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Theodore Kilgore, Adam Baker Am Freitag, 5. August 2011, 00:56:03 schrieb Greg KH: > On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > I know that this problem were somewhat solved for 3G modems, with the usage > > of the userspace problem usb_modeswitch, and with some quirks for the USB > > storage driver, but I'm not sure if such tricks will scale forever, as more > > functions are seen on some USB devices. > > Well, no matter how it "scales" it needs to be done in userspace, like > usb_modeswitch does. We made that decision a while ago, and it is > working out very well. I see no reason why you can't do it in userspace > as well as that is the easiest place to control this type of thing. > > I thought we had a long discussion about this topic a while ago and came > to this very conclusion. Or am I mistaken? Circumstances change. We want to keep the stuff in user space as much and as long as we can. However user space has limitations: - it has by necessity a race between resumption and access by others - it cannot resume anything we run a (rw) filesystem over. Furthermore, today PM actions that lead to a loss of mode are initiated by user space. If we ever want to oportunistically suspend a system we also need to restore mode from inside the kernel. We could avoid all that trouble, if we persuaded vendors to use plain USB configurations for those purposes. Regards Oliver ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-05 6:57 ` Oliver Neukum @ 2011-08-05 17:38 ` Theodore Kilgore 0 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-05 17:38 UTC (permalink / raw) To: Oliver Neukum Cc: Greg KH, Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Hans de Goede, Adam Baker On Fri, 5 Aug 2011, Oliver Neukum wrote: > Am Freitag, 5. August 2011, 00:56:03 schrieb Greg KH: > > On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > > I know that this problem were somewhat solved for 3G modems, with the usage > > > of the userspace problem usb_modeswitch, and with some quirks for the USB > > > storage driver, but I'm not sure if such tricks will scale forever, as more > > > functions are seen on some USB devices. > > > > Well, no matter how it "scales" it needs to be done in userspace, like > > usb_modeswitch does. We made that decision a while ago, and it is > > working out very well. I see no reason why you can't do it in userspace > > as well as that is the easiest place to control this type of thing. > > > > I thought we had a long discussion about this topic a while ago and came > > to this very conclusion. Or am I mistaken? > > Circumstances change. We want to keep the stuff in user space as much and > as long as we can. However user space has limitations: > > - it has by necessity a race between resumption and access by others > - it cannot resume anything we run a (rw) filesystem over. > > Furthermore, today PM actions that lead to a loss of mode are initiated > by user space. If we ever want to oportunistically suspend a system > we also need to restore mode from inside the kernel. > > We could avoid all that trouble, if we persuaded vendors to use plain > USB configurations for those purposes. But that would happen, I suspect, in an alternate universe. Better to anticipate the trouble, I suspect. :-{ Moreover, the spark for the current discussion was the problem of dual-mode cameras, which can work both as webcams and stillcams, not the 3G modems that you mention. The problems are analogous but not identical. -- dual-mode cameras are, typically, Class Proprietary devices in all of their functions. None of them that I know of are Mass Storage devices. Therefore, usb_modeswitch would have to be rewritten completely in order to be used for such hardware. As things stand right now, it has nothing at all to do with the problem. Not to say, of course, that the experience gained with usb_modeswitch is totally irrelevant. -- I don't have one of those modems, but I have the impression that the need to access the "disk" partition on one of them is basically a one-shot deal. Namely, one needs to load some firmware or so on the "disk" before the gadget can be used. The problem with a dual-mode camera is that the user ought to be able to switch at will between a download-the-pictures stillcam application and a stream application. While, of course, not being able to start one of these activities while the other is going on, because that would cause obvious trouble. The similarity of the modem-and-mass-storage device and the web-and-still camera is, of course, that both partake of being devices that will do more than one kind of thing and they need to be supported in that respect. Other hardware exists with similar characteristics, but sometimes the functionaliy is not dual but even triple, and one can reasonably suspect that more of this kind of thing is going to come at us in the future. I think it is a good occasion to sit back and think things over a bit. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-04 22:56 ` Greg KH ` (2 preceding siblings ...) 2011-08-05 6:57 ` Oliver Neukum @ 2011-08-05 7:45 ` Hans de Goede 2011-08-05 7:59 ` USB mini-summit at LinuxCon Vancouveroliver Oliver Neukum ` (2 more replies) 3 siblings, 3 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-05 7:45 UTC (permalink / raw) To: Greg KH Cc: Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi, On 08/05/2011 12:56 AM, Greg KH wrote: > On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: >> I know that this problem were somewhat solved for 3G modems, with the usage >> of the userspace problem usb_modeswitch, and with some quirks for the USB >> storage driver, but I'm not sure if such tricks will scale forever, as more >> functions are seen on some USB devices. > > Well, no matter how it "scales" it needs to be done in userspace, like > usb_modeswitch does. We made that decision a while ago, and it is > working out very well. I see no reason why you can't do it in userspace > as well as that is the easiest place to control this type of thing. > > I thought we had a long discussion about this topic a while ago and came > to this very conclusion. Or am I mistaken? I think we've had multiple discussions about this, surrounding various topics / use cases. I would not call the do it in userspace a conclusion. I rather more have a feeling of getting stonewalled on this by various people surrounding the usb system. Me shutting up on this is basically a case of: "Discussion not getting anywhere and I don't have the time to do a kernel proof of concept myself right now". To be clear about the stonewalling I'm not talking about the dual cam issue here, nor about the morphing devices thing which usb_modeswitch fixes. I think it is important to separate oranges from apples here, there are at least 3 different problem classes which all seem to have gotten thrown onto a pile here: 1) The reason Mauro suggested having some discussion on this at the USB summit is because of a discussion about dual mode cameras on the linux media list. Dual-mode cameras are (usually very cheap) digital photo cameras which can take still pictures in stand alone mode and store them in onboard memory, like regular digital photo cameras, this is one mode. The other mode is they can operate as a regular webcam, the hardware actually is more regular webcam hardware with some memory and a battery added (usually no viewfinder screen). These devices typically use 1 usb interface (and thus 1 driver) for both modes. We currently have drivers for both modes, but they are separate drivers, which leads to fighting for device ownership, with the stillmode driver always winning as that is userspace and it simply kicks of the kernel webcam /streaming mode driver making it see a device unplug, even if an app was streaming from the device at the time. The solution here is simple, move both functions to 1 driver, which can then properly return -EBUSY when the device is active in one mode and an app tries to use it in the other mode. This 1 driver will likely end up being a kernel driver, since doing streaming drivers in userspace is problematic, as the v4l2 API assumes a kernel device node, and we don't want to invent a new API, we want something which works with existing applications. 2) So called morphing devices. For example a USB 3G modem which initially shows up as a cdrom drive with windows software, only to reveal its real identity (with completely different usb descriptors) after some magic command. This is solved by the usb_modeswitch command, nothing to see here move along. 3) Re-direction of usb devices to virtual machines. This works by using the userspace usbfs interface from qemu / vmware / virtualbox / whatever. The basics of this work fine, but it lacks proper locking / safeguards for when a vm takes over a usb device from the in kernel driver. Example of the problem: usb mass storage devices is mounted, filesystem is in use and dirty, with writebacks pending, user redirects to vm, usb mass storage driver sees the cable beeing yanked out of the device. Now we've lost the pending writebacks and have a dirty, if not journaling possibly corrupt, fs on the device. This is the issue on which I feel a bit stonewalled. Simple putting your fingers in your ears and singing la la la do it in userspace is not going to cut it here. There is no way to do this race free in userspace, unless all possible callers of mount get modified. Moreover 99% of the necessary accounting for this is already done in the kernel. We already have the notion of a block device being in use. We simply need to add some code to pass this notion to the usb mass storage driver, and add a new try_disconnect callback for usb drivers. I'm not saying this is going to be completely straight forward, but it ain't rocket science either. And it so very obviously is the *right* thing to do, that I'm getting very tired of the do it in userspace song I keep hearing. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouveroliver 2011-08-05 7:45 ` Hans de Goede @ 2011-08-05 7:59 ` Oliver Neukum 2011-08-05 8:18 ` Hans de Goede 2011-08-05 13:07 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab 2011-08-08 17:58 ` Sarah Sharp 2 siblings, 1 reply; 90+ messages in thread From: Oliver Neukum @ 2011-08-05 7:59 UTC (permalink / raw) To: Hans de Goede Cc: Greg KH, Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Am Freitag, 5. August 2011, 09:45:56 schrieb Hans de Goede: > This is the issue on which I feel a bit stonewalled. Simple putting your > fingers in your ears and singing la la la do it in userspace is not going > to cut it here. There is no way to do this race free in userspace, unless > all possible callers of mount get modified. Moreover 99% of the necessary > accounting for this is already done in the kernel. We already have the notion > of a block device being in use. We simply need to add some code to pass > this notion to the usb mass storage driver, and add a new try_disconnect > callback for usb drivers. I'm not saying this is going to be completely > straight forward, but it ain't rocket science either. And it so very > obviously is the right thing to do, that I'm getting very tired of > the do it in userspace song I keep hearing. Doing a try_disconnect() would also solve the dual camera issue. But it doesn't really interfere with the user space vs. kernel space issue. You simply have to expand the ioctl interface to have an ioctl that triggers this API call in the kernel. A V4L2 device would return an error if its device node is opened, otherwise disconnect. Regards Oliver ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouveroliver 2011-08-05 7:59 ` USB mini-summit at LinuxCon Vancouveroliver Oliver Neukum @ 2011-08-05 8:18 ` Hans de Goede 0 siblings, 0 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-05 8:18 UTC (permalink / raw) To: Oliver Neukum Cc: Greg KH, Mauro Carvalho Chehab, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi, On 08/05/2011 09:59 AM, Oliver Neukum wrote: > Am Freitag, 5. August 2011, 09:45:56 schrieb Hans de Goede: >> This is the issue on which I feel a bit stonewalled. Simple putting your >> fingers in your ears and singing la la la do it in userspace is not going >> to cut it here. There is no way to do this race free in userspace, unless >> all possible callers of mount get modified. Moreover 99% of the necessary >> accounting for this is already done in the kernel. We already have the notion >> of a block device being in use. We simply need to add some code to pass >> this notion to the usb mass storage driver, and add a new try_disconnect >> callback for usb drivers. I'm not saying this is going to be completely >> straight forward, but it ain't rocket science either. And it so very >> obviously is the right thing to do, that I'm getting very tired of >> the do it in userspace song I keep hearing. > > Doing a try_disconnect() would also solve the dual camera issue. > But it doesn't really interfere with the user space vs. kernel space > issue. You simply have to expand the ioctl interface to have > an ioctl that triggers this API call in the kernel. > > A V4L2 device would return an error if its device node is opened, > otherwise disconnect. Getting a bit offtopic here, but no a try_disconnect will fix the userspace stillcam mode driver being able to disconnect the device while the webcam function is active. If the webcam is not active userspace will still "win", and possibly never return the device back to the kernel driver (this already happens today with gvfs-gphoto creating a fuse mount and keeping the device open indefinitely, locking out the webcam function). Likewise a v4l2 control panel like app (think alsamixer for videodevs to set brightness / contrast, etc.) can keep the /dev/video node open indefinitely. Unless we rewrite most of userspace, we need to allow the device to be open in bode modes *at the same time* and only fail with -EBUSY when something really exclusive is requested (so not just having the device open, or setting contrast, but trying to stream and read/delete pictures from the stillcam memory at the same time). Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-05 7:45 ` Hans de Goede 2011-08-05 7:59 ` USB mini-summit at LinuxCon Vancouveroliver Oliver Neukum @ 2011-08-05 13:07 ` Mauro Carvalho Chehab 2011-08-08 17:58 ` Sarah Sharp 2 siblings, 0 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-05 13:07 UTC (permalink / raw) To: Hans de Goede Cc: Greg KH, Sarah Sharp, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Em 05-08-2011 04:45, Hans de Goede escreveu: > Hi, > > On 08/05/2011 12:56 AM, Greg KH wrote: >> On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: >>> I know that this problem were somewhat solved for 3G modems, with the usage >>> of the userspace problem usb_modeswitch, and with some quirks for the USB >>> storage driver, but I'm not sure if such tricks will scale forever, as more >>> functions are seen on some USB devices. >> >> Well, no matter how it "scales" it needs to be done in userspace, like >> usb_modeswitch does. We made that decision a while ago, and it is >> working out very well. I see no reason why you can't do it in userspace >> as well as that is the easiest place to control this type of thing. >> >> I thought we had a long discussion about this topic a while ago and came >> to this very conclusion. Or am I mistaken? > > I think we've had multiple discussions about this, surrounding various > topics / use cases. I would not call the do it in userspace a conclusion. > > I rather more have a feeling of getting stonewalled on this by various people > surrounding the usb system. Me shutting up on this is basically a case of: > "Discussion not getting anywhere and I don't have the time to do a kernel > proof of concept myself right now". > > To be clear about the stonewalling I'm not talking about the dual cam issue > here, nor about the morphing devices thing which usb_modeswitch fixes. > > I think it is important to separate oranges from apples here, there are > at least 3 different problem classes which all seem to have gotten thrown > onto a pile here: > > 1) The reason Mauro suggested having some discussion on this at the > USB summit is because of a discussion about dual mode cameras on the > linux media list. Dual-mode cameras are (usually very cheap) digital > photo cameras which can take still pictures in stand alone mode and > store them in onboard memory, like regular digital photo cameras, this > is one mode. The other mode is they can operate as a regular webcam, > the hardware actually is more regular webcam hardware with some memory > and a battery added (usually no viewfinder screen). Yes, that's my concern. > These devices typically use 1 usb interface (and thus 1 driver) for > both modes. I also want to cover the case where there are two drivers involved, as this is a typical trouble we need to do with all devices that have both analog and digital TV's. A typical media device exposes several different API's to userspace, each with its own character device, and sometimes implemented by a different driver. For example, Conexant cx23102 USB chipsets provide 2 USB interfaces, but require 3 different drivers, and exposes 4 different userspace API's to userspace: USB interface 0: Remote Controller event/input API - Handled by drivers/media/rc/mceusb.c USB interface 1: Audio/Video ALSA API - Handled by drivers/media/video/cx231xx/cx231xx-audio.c It is a separate driver, called cx231xx-alsa V4L2 API - Handled by drivers/media/video/cx231xx/cx231xx-video.c DVB API - Handled by drivers/media/video/cx231xx/cx231xx-dvb.c Both DVB and V4L2 are currently part of the same driver (cx231xx). The mceusb is completely independent of the other drivers. The cx231xx-alsa also is almost independent. It only shares a lock with the main cx231xx driver, and uses the I2C functions implemented by the main driver. Now, we're starting to see devices like ZTE MF845 that have TV, 3G and USB storage, exposing 2 more userspace API's to userspace (vfs and tty). I don't think that userspace usb_modeswitch-like type of locking "scales" such complex devices. Besides that, on a large amount of USB sticks, if you enable both V4L2 and DVB, the power consumption may exceed the 500mA power limit of USB, and can overheat the device. There are several reports of burnt devices due to that. Also, there's no way for both to work, as they share the same tuner, that can either be tuning an analog or a digital channel. To me, the right solution seems to add some sort of locking schema that will protect the common resources on complex hardware. Something like: int get_hw_resource(struct device, enum hw_resource_type); void put_hw_resource(struct device, enum hw_resource_type); enum hw_resource_type { HW_RES_USB, HW_RES_I2C, HW_RES_TUNER, HW_RES_USB, ... }; So that, when a device need, for example, an exclusive lock for the USB device, for example, it could be doing something like: rc = get_hw_resource(dev, HW_RES_USB); if (rc < 0) return -EBUSY; and, after finishing, doing a put_hw_resource(). > We currently have drivers for both modes, but they are > separate drivers, which leads to fighting for device ownership, > with the stillmode driver always winning as that is userspace and > it simply kicks of the kernel webcam /streaming mode driver making > it see a device unplug, even if an app was streaming from the device > at the time. The solution here is simple, move both functions to > 1 driver, which can then properly return -EBUSY when the device > is active in one mode and an app tries to use it in the other mode. > > This 1 driver will likely end up being a kernel driver, since doing > streaming drivers in userspace is problematic, as the v4l2 API assumes > a kernel device node, and we don't want to invent a new API, we want > something which works with existing applications. > > 2) So called morphing devices. For example a USB 3G modem which > initially shows up as a cdrom drive with windows software, only > to reveal its real identity (with completely different usb > descriptors) after some magic command. This is solved by the > usb_modeswitch command, nothing to see here move along. > > 3) Re-direction of usb devices to virtual machines. This works by using > the userspace usbfs interface from qemu / vmware / virtualbox / whatever. > The basics of this work fine, but it lacks proper locking / safeguards > for when a vm takes over a usb device from the in kernel driver. > > Example of the problem: usb mass storage devices is mounted, filesystem > is in use and dirty, with writebacks pending, user redirects to vm, > usb mass storage driver sees the cable beeing yanked out of the device. > > Now we've lost the pending writebacks and have a dirty, if not journaling > possibly corrupt, fs on the device. > > This is the issue on which I feel a bit stonewalled. Simple putting your > fingers in your ears and singing la la la do it in userspace is not going > to cut it here. There is no way to do this race free in userspace, unless > all possible callers of mount get modified. Moreover 99% of the necessary > accounting for this is already done in the kernel. We already have the notion > of a block device being in use. We simply need to add some code to pass > this notion to the usb mass storage driver, and add a new try_disconnect > callback for usb drivers. I'm not saying this is going to be completely > straight forward, but it ain't rocket science either. And it so very > obviously is the *right* thing to do, that I'm getting very tired of > the do it in userspace song I keep hearing. > > Regards, > > Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-05 7:45 ` Hans de Goede 2011-08-05 7:59 ` USB mini-summit at LinuxCon Vancouveroliver Oliver Neukum 2011-08-05 13:07 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab @ 2011-08-08 17:58 ` Sarah Sharp 2011-08-08 18:23 ` Theodore Kilgore ` (2 more replies) 2 siblings, 3 replies; 90+ messages in thread From: Sarah Sharp @ 2011-08-08 17:58 UTC (permalink / raw) To: Hans de Goede Cc: Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: > Hi, > > On 08/05/2011 12:56 AM, Greg KH wrote: > >On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > I think it is important to separate oranges from apples here, there are > at least 3 different problem classes which all seem to have gotten thrown > onto a pile here: > > 1) The reason Mauro suggested having some discussion on this at the > USB summit is because of a discussion about dual mode cameras on the > linux media list. ... > 3) Re-direction of usb devices to virtual machines. This works by using > the userspace usbfs interface from qemu / vmware / virtualbox / whatever. > The basics of this work fine, but it lacks proper locking / safeguards > for when a vm takes over a usb device from the in kernel driver. Hi Hans and Mauro, We have do room in the schedule for the USB mini-summit for this discussion, since the schedule is still pretty flexible. The preliminary schedule is up here: http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html I think it's best to discuss the VM redirection in the afternoon when some of the KVM folks join us after Hans' talk on USB redirection over the network. It sounds like we need a separate topic for the dual mode cameras and TV tuners. Mauro, do you want to lead that discussion in the early morning (in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 slot)? I want to be sure we have all the video/media developers who are interested in this topic present, and I don't know if they will be going to the KVM forum. Link PM was originally slated for the 10am slot, but since we're missing several people for that discussion (Alan Stern, Andiry Xu, and any of the Intel folks who did the Moorestown USB 2.0 LPM), I think it will be more useful to have that discussion on-list or at another conference. Sarah Sharp ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-08 17:58 ` Sarah Sharp @ 2011-08-08 18:23 ` Theodore Kilgore 2011-08-08 18:32 ` Sarah Sharp 2011-08-08 19:37 ` Mauro Carvalho Chehab 2011-08-09 7:52 ` Hans de Goede 2 siblings, 1 reply; 90+ messages in thread From: Theodore Kilgore @ 2011-08-08 18:23 UTC (permalink / raw) To: Sarah Sharp Cc: Hans de Goede, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Mon, 8 Aug 2011, Sarah Sharp wrote: > On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: > > Hi, > > > > On 08/05/2011 12:56 AM, Greg KH wrote: > > >On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > I think it is important to separate oranges from apples here, there are > > at least 3 different problem classes which all seem to have gotten thrown > > onto a pile here: > > > > 1) The reason Mauro suggested having some discussion on this at the > > USB summit is because of a discussion about dual mode cameras on the > > linux media list. > ... > > 3) Re-direction of usb devices to virtual machines. This works by using > > the userspace usbfs interface from qemu / vmware / virtualbox / whatever. > > The basics of this work fine, but it lacks proper locking / safeguards > > for when a vm takes over a usb device from the in kernel driver. > > Hi Hans and Mauro, > > We have do room in the schedule for the USB mini-summit for this > discussion, since the schedule is still pretty flexible. The > preliminary schedule is up here: > > http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html > > I think it's best to discuss the VM redirection in the afternoon when > some of the KVM folks join us after Hans' talk on USB redirection over > the network. > > It sounds like we need a separate topic for the dual mode cameras and TV > tuners. Mauro, do you want to lead that discussion in the early morning > (in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 > slot)? I want to be sure we have all the video/media developers who are > interested in this topic present, and I don't know if they will be going > to the KVM forum. Sarah, Alas. I would suspect that I am one of the people most interested in the topic of dual-mode cameras, since I have worked on supporting them both in libgphoto2 and in the kernel. But I teach in a university for a living, and the first classes of Fall Semester 2011 start on August 17 in Auburn, Alabama. Knowing this, I decided, months ago, that I simply could not attend a conference which starts on August 16 in Vancouver. So, after starting all of the current mailing-list discussion on the topic I will not be at the conference. I can only hope that those who do attend will keep me current about what gets discussed. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-08 18:23 ` Theodore Kilgore @ 2011-08-08 18:32 ` Sarah Sharp 0 siblings, 0 replies; 90+ messages in thread From: Sarah Sharp @ 2011-08-08 18:32 UTC (permalink / raw) To: Theodore Kilgore Cc: Hans de Goede, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Mon, Aug 08, 2011 at 01:23:56PM -0500, Theodore Kilgore wrote: > > > On Mon, 8 Aug 2011, Sarah Sharp wrote: > > > On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: > > > Hi, > > > > > > On 08/05/2011 12:56 AM, Greg KH wrote: > > > >On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > > > I think it is important to separate oranges from apples here, there are > > > at least 3 different problem classes which all seem to have gotten thrown > > > onto a pile here: > > > > > > 1) The reason Mauro suggested having some discussion on this at the > > > USB summit is because of a discussion about dual mode cameras on the > > > linux media list. > > ... > > > 3) Re-direction of usb devices to virtual machines. This works by using > > > the userspace usbfs interface from qemu / vmware / virtualbox / whatever. > > > The basics of this work fine, but it lacks proper locking / safeguards > > > for when a vm takes over a usb device from the in kernel driver. > > > > Hi Hans and Mauro, > > > > We have do room in the schedule for the USB mini-summit for this > > discussion, since the schedule is still pretty flexible. The > > preliminary schedule is up here: > > > > http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html > > > > I think it's best to discuss the VM redirection in the afternoon when > > some of the KVM folks join us after Hans' talk on USB redirection over > > the network. > > > > It sounds like we need a separate topic for the dual mode cameras and TV > > tuners. Mauro, do you want to lead that discussion in the early morning > > (in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 > > slot)? I want to be sure we have all the video/media developers who are > > interested in this topic present, and I don't know if they will be going > > to the KVM forum. > > Sarah, > > Alas. I would suspect that I am one of the people most interested in the > topic of dual-mode cameras, since I have worked on supporting them both in > libgphoto2 and in the kernel. But I teach in a university for a living, > and the first classes of Fall Semester 2011 start on August 17 in Auburn, > Alabama. Knowing this, I decided, months ago, that I simply could not > attend a conference which starts on August 16 in Vancouver. > > So, after starting all of the current mailing-list discussion on the topic > I will not be at the conference. I can only hope that those who do attend > will keep me current about what gets discussed. No worries, I'll be taking notes and post them to the Linux USB and Linux media lists. Sarah Sharp ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-08 17:58 ` Sarah Sharp 2011-08-08 18:23 ` Theodore Kilgore @ 2011-08-08 19:37 ` Mauro Carvalho Chehab 2011-08-09 7:52 ` Hans de Goede 2 siblings, 0 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-08 19:37 UTC (permalink / raw) To: Sarah Sharp Cc: Hans de Goede, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi Sarah, Em 08-08-2011 14:58, Sarah Sharp escreveu: > On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: >> Hi, >> >> On 08/05/2011 12:56 AM, Greg KH wrote: >>> On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: >> I think it is important to separate oranges from apples here, there are >> at least 3 different problem classes which all seem to have gotten thrown >> onto a pile here: >> >> 1) The reason Mauro suggested having some discussion on this at the >> USB summit is because of a discussion about dual mode cameras on the >> linux media list. > ... >> 3) Re-direction of usb devices to virtual machines. This works by using >> the userspace usbfs interface from qemu / vmware / virtualbox / whatever. >> The basics of this work fine, but it lacks proper locking / safeguards >> for when a vm takes over a usb device from the in kernel driver. > > Hi Hans and Mauro, > > We have do room in the schedule for the USB mini-summit for this > discussion, since the schedule is still pretty flexible. The > preliminary schedule is up here: > > http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html > > I think it's best to discuss the VM redirection in the afternoon when > some of the KVM folks join us after Hans' talk on USB redirection over > the network. > > It sounds like we need a separate topic for the dual mode cameras and TV > tuners. Yes, I suspect so. This subject is pending for quite some time, and it would be good to have people think together around a solution for that. > Mauro, do you want to lead that discussion in the early morning > (in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 > slot)? I want to be sure we have all the video/media developers who are > interested in this topic present, and I don't know if they will be going > to the KVM forum. >From my side, both slots work ok. Hans, what would fit better for you? > Link PM was originally slated for the 10am slot, but since we're missing > several people for that discussion (Alan Stern, Andiry Xu, and any of > the Intel folks who did the Moorestown USB 2.0 LPM), I think it will be > more useful to have that discussion on-list or at another conference. > > Sarah Sharp Thanks! Mauro ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-08 17:58 ` Sarah Sharp 2011-08-08 18:23 ` Theodore Kilgore 2011-08-08 19:37 ` Mauro Carvalho Chehab @ 2011-08-09 7:52 ` Hans de Goede 2011-08-09 14:19 ` Alan Stern 2011-08-09 17:10 ` Sarah Sharp 2 siblings, 2 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-09 7:52 UTC (permalink / raw) To: Sarah Sharp Cc: Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi, On 08/08/2011 07:58 PM, Sarah Sharp wrote: > On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: >> Hi, >> >> On 08/05/2011 12:56 AM, Greg KH wrote: >>> On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: >> I think it is important to separate oranges from apples here, there are >> at least 3 different problem classes which all seem to have gotten thrown >> onto a pile here: >> >> 1) The reason Mauro suggested having some discussion on this at the >> USB summit is because of a discussion about dual mode cameras on the >> linux media list. > ... >> 3) Re-direction of usb devices to virtual machines. This works by using >> the userspace usbfs interface from qemu / vmware / virtualbox / whatever. >> The basics of this work fine, but it lacks proper locking / safeguards >> for when a vm takes over a usb device from the in kernel driver. > > Hi Hans and Mauro, > > We have do room in the schedule for the USB mini-summit for this > discussion, since the schedule is still pretty flexible. The > preliminary schedule is up here: > > http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html > > I think it's best to discuss the VM redirection in the afternoon when > some of the KVM folks join us after Hans' talk on USB redirection over > the network. > That seems best to me too. I'm available at both proposed time slots, and I would like to join both discussions. > It sounds like we need a separate topic for the dual mode cameras and TV > tuners. Mauro, do you want to lead that discussion in the early morning > (in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 > slot)? I want to be sure we have all the video/media developers who are > interested in this topic present, and I don't know if they will be going > to the KVM forum. I would really like to see the dual mode camera and TV tuner discussion separated. They are 2 different issues AFAIK. 1) Dual mode cameras: In the case of the dual mode camera we have 1 single device (both at the hardware level and at the logical block level), which can do 2 things, but not at the same time. It can stream live video data from a sensor, or it can retrieve earlier taken pictures from some picture memory. Unfortunately even though these 2 functions live in a single logical block, historically we've developed 2 drivers for them. This leads to fighting over device ownership (surprise surprise), and to me the solution is very clear, 1 logical block == 1 driver. 2) Tv tuners: Here we have a bunch of logical blocks, each with their own driver (and not 2 drivers for one block), which together form not 1 but 2 video pipelines, typically one pipeline for analog TV, and one for DVB. The problem here is some blocks are shared between the 2 pipelines. The solution here, at least to me, is clear too, we need some way of claiming/locking the pipeline blocks. If pipe1 starts streaming video data it locks all the blocks it uses. If some app then tries to use pipe2 (and thus for example tune the tuner to a different frequency), it will fail with -EBUSY when pipe2 tries to take the tuner lock and gets told it is in use. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 7:52 ` Hans de Goede @ 2011-08-09 14:19 ` Alan Stern 2011-08-09 15:03 ` Marko Ristola 2011-08-09 19:57 ` Hans de Goede 2011-08-09 17:10 ` Sarah Sharp 1 sibling, 2 replies; 90+ messages in thread From: Alan Stern @ 2011-08-09 14:19 UTC (permalink / raw) To: Hans de Goede Cc: Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker On Tue, 9 Aug 2011, Hans de Goede wrote: > I would really like to see the dual mode camera and TV tuner discussion > separated. They are 2 different issues AFAIK. > > 1) Dual mode cameras: > > In the case of the dual mode camera we have 1 single device (both at > the hardware level and at the logical block level), which can do 2 things, > but not at the same time. It can stream live video data from a sensor, > or it can retrieve earlier taken pictures from some picture memory. > > Unfortunately even though these 2 functions live in a single logical block, > historically we've developed 2 drivers for them. This leads to fighting > over device ownership (surprise surprise), and to me the solution is > very clear, 1 logical block == 1 driver. According to Theodore, we have developed 5 drivers for them because the stillcam modes in different devices use four different vendor-specific drivers. Does it really make sense to combine 5 drivers into one? I think some sort of sharing protocol would work out better. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 14:19 ` Alan Stern @ 2011-08-09 15:03 ` Marko Ristola 2011-08-09 19:57 ` Hans de Goede 1 sibling, 0 replies; 90+ messages in thread From: Marko Ristola @ 2011-08-09 15:03 UTC (permalink / raw) To: Alan Stern Cc: Hans de Goede, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi I've been thinking about the Kernel driver side. Mauro and others emailed requirements on Jun or July. I'm sorry for this spam: maybe you have thought this already. A linked list of read/write locks as the solution for these protections could be a base for the general solution. Locks could be accessed either by name "string name" or by an integer identifier. The bridge driver would be the container of the lock list. Lock take / release handles would be changeable by Bridge driver at driver init. This way bridge could tune the lock taking actions. Sub devices would not have to know the details of the bridge device. When implemented as a library ".ko" module, this could be used by all related kernel drivers. The locking code would be very general. Maybe adding a lock list into each PCI bus device would solve the device export problem to KVM too. If some module wouldn't handle the proper locking yet, it would not deliver the protection, but it would work as before: no regressions. The library could also be called so that a driver would ask three locks at the same time. If the driver would get all three locks, it would return success. If the driver would not get all three locks, it would not lock any of them (with _trylock case). I don't have time to implement this feature. Happy meeting for all of you, Marko Ristola ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 14:19 ` Alan Stern 2011-08-09 15:03 ` Marko Ristola @ 2011-08-09 19:57 ` Hans de Goede 2011-08-09 20:31 ` Adam Baker ` (2 more replies) 1 sibling, 3 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-09 19:57 UTC (permalink / raw) To: Alan Stern Cc: Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker Hi, On 08/09/2011 04:19 PM, Alan Stern wrote: > On Tue, 9 Aug 2011, Hans de Goede wrote: > >> I would really like to see the dual mode camera and TV tuner discussion >> separated. They are 2 different issues AFAIK. >> >> 1) Dual mode cameras: >> >> In the case of the dual mode camera we have 1 single device (both at >> the hardware level and at the logical block level), which can do 2 things, >> but not at the same time. It can stream live video data from a sensor, >> or it can retrieve earlier taken pictures from some picture memory. >> >> Unfortunately even though these 2 functions live in a single logical block, >> historically we've developed 2 drivers for them. This leads to fighting >> over device ownership (surprise surprise), and to me the solution is >> very clear, 1 logical block == 1 driver. > > According to Theodore, we have developed 5 drivers for them because the > stillcam modes in different devices use four different vendor-specific > drivers. Yes, but so the the webcam modes of the different devices, so for the 5 (not sure if that is the right number) dual-cam mode chipsets we support there will be 5 drivers, each supporting both the webcam and the access to pictures stored in memory of the chipset they support. So 5 chipsets -> 5 drivers each supporting 1 chipset, and both functions of the single logical device that chipset represents. > Does it really make sense to combine 5 drivers into one? Right, that is not the plan. The plan is to simply stop having 2 drivers for 1 logical (and physical) block. So we go from 10 drivers, 5 stillcam + 5 webcam, to just 5 drivers. We will also likely be able to share code between the code for the 2 functionalities for things like generic set / get register functions, initialization, etc. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 19:57 ` Hans de Goede @ 2011-08-09 20:31 ` Adam Baker 2011-08-09 20:57 ` Hans de Goede 2011-08-09 23:05 ` Theodore Kilgore 2011-08-10 14:19 ` Alan Stern 2 siblings, 1 reply; 90+ messages in thread From: Adam Baker @ 2011-08-09 20:31 UTC (permalink / raw) To: Hans de Goede Cc: Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore On Tuesday 09 August 2011, Hans de Goede wrote: > Hi, > > On 08/09/2011 04:19 PM, Alan Stern wrote: > > Does it really make sense to combine 5 drivers into one? > > Right, that is not the plan. The plan is to simply stop having 2 drivers > for 1 logical (and physical) block. So we go from 10 drivers, 5 stillcam > + 5 webcam, to just 5 drivers. We will also likely be able to share > code between the code for the 2 functionalities for things like generic > set / get register functions, initialization, etc. > Unfortunately as Theodore recently pointed out you don't go from 10 to 5, you go from 10 to 10 where 5 of the new 10 are only used on Win32, FreeBSD and OSX (but they aren't any simpler because they still rely on libusb) and the other 5 that are only used on Linux become significantly more complicated than they currently are. It has also just occured to me that it might be possible to solve the issues we are facing just in the kernel. At the moment when the kernel performs a USBDEVFS_DISCONNECT it keeps the kernel driver locked out until userspace performs a USBDEVFS_CONNECT. If the kernel reattached the kernel driver when the device file was closed then, as gvfs doesn't keep the file open the biggest current issue would be solved instantly. If a mechanism could be found to prevent USBDEVFS_DISCONNECT from succeeding when the corresponding /dev/videox file was open then that would seem to be a reasonable solution. Hans had expressed the opinion that merely having the device open to control the camera not to stream shouldn't prevent stillcam operation - I disagree because if you are setting up the controls you are probably already streaming so you can see what you are doing and if not you are probably about to. Of course changing the behaviour of USBDEVFS_DISCONNECT is not something to be done lightly. I don't know how many other users there are for it and if the current behaviour is actually correct for any of them. Cleaning up on file close does have the useful side effect though that applications no longer need to worry about the fact that even if they clean up properly on a normal exit, if they crash they leave the kernel driver permanently disabled Regards Adam ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 20:31 ` Adam Baker @ 2011-08-09 20:57 ` Hans de Goede 2011-08-10 2:05 ` Xiaofan Chen 2011-08-10 23:04 ` Adam Baker 0 siblings, 2 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-09 20:57 UTC (permalink / raw) To: Adam Baker Cc: Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore Hi, On 08/09/2011 10:31 PM, Adam Baker wrote: > On Tuesday 09 August 2011, Hans de Goede wrote: <snip> > It has also just occured to me that it might be possible to solve the issues > we are facing just in the kernel. At the moment when the kernel performs a > USBDEVFS_DISCONNECT it keeps the kernel driver locked out until userspace > performs a USBDEVFS_CONNECT. If the kernel reattached the kernel driver when > the device file was closed then, as gvfs doesn't keep the file open the > biggest current issue would be solved instantly. If a mechanism could be found > to prevent USBDEVFS_DISCONNECT from succeeding when the corresponding > /dev/videox file was open then that would seem to be a reasonable solution. <sigh> This has been discussed over and over and over again, playing clever tricks with USBDEVFS_[DIS]CONNECT like adding a new USBDEVFS_TRYDISCONNECT which the v4l2 driver could intercept won't cut it. We need some central manager of the device doing multiplexing between the 2 functions, and you can *not* assume that either side will be nice wrt closing file descriptors. Examples: 1) You are wrong wrt gvfs, it does keep the libgphoto2 context open all the time, and through that the usbfs device nodes. 2) Lets say a user starts a photo managing app like f-spot, and that opens the device through libgphoto2 on startup, then the user switches to another virtual desktop and forgets all about having f-spot open. Notice that if the user now tries to stream he will not get a busy error, but the app trying to do the streaming will simply not see the camera at all (kernel driver unbound /dev/video# node is gone). 3) Notice that little speaker icon in your panel on your average Linux desktop, that keeps the mixer of the audio device open *all the time* it is quite easy to imagine a similar applet for v4l2 device controls (see for example gtk-v4l) doing the same. Or a user could simply start up a v4l2 control panel app like gtk-v4l, qv4l2 or v4l2ucp, and leave it running minimized ... 4) Some laptops have a Fn + F## key which enables / disables the builtin webcam by turning its power on / off. Effectively plugging it into / out of a usb port. We would like to have an on screen notification of this one day like we have now for brightness and volume controls, based on udev events. But the current dual mode cam stuff causes udev events for a *new* video device being added / an existing one being removed each time libgphoto2 releases / takes control of the camera. 5) More in general, more and more software is dynamically monitoring the addition / removal of (usb) devices using udev, our current solution suggests to this software the /dev/video device is being unplugged / re-plugged all the time, not pretty. All in all what we've today is a kludge, and if we want to provide a "seamless" user experience we need to fix it. Don't get me wrong usbfs is a really nice solution for driving usb devices from userspace, like scanners and all other sorts of devices. But what all these devices have in common is that they have no kernel driver. Having a userspace based driver and a kernel driver "fight it out" just does not work well. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 20:57 ` Hans de Goede @ 2011-08-10 2:05 ` Xiaofan Chen 2011-08-10 23:04 ` Adam Baker 1 sibling, 0 replies; 90+ messages in thread From: Xiaofan Chen @ 2011-08-10 2:05 UTC (permalink / raw) To: Hans de Goede Cc: Adam Baker, Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore On Wed, Aug 10, 2011 at 4:57 AM, Hans de Goede <hdegoede@redhat.com> wrote: > This has been discussed over and over and over again, playing clever > tricks with USBDEVFS_[DIS]CONNECT like adding a new > USBDEVFS_TRYDISCONNECT > which the v4l2 driver could intercept won't cut it. We need some central > manager of the device doing multiplexing between the 2 functions, and you > can *not* assume that either side will be nice wrt closing file descriptors. So it seems to be both good and bad for Linux's ability to do multiplexing between the two drivers (one purpose-built kernel driver and one generic usbfs driver to be used libusb). The good thing is that it is quite useful in many use cases. The ability to detach the existing kernel driver and use usbfs is the base for many libusb based program under Linux and it beats Windows and Mac OS X handsomely in this aspect. On the other hand, based on the discussions, it seems to adds quite some issues/complexities in some other use cases. Windows typically does not have such an issue. You can use only one driver. In order to use libusb-win32 or libusb-1.0 Windows backend, you have to replace the existing kernel driver with libusb-win32 kernel driver (libusb0.sys) or libusb-1.0 Windows backend driver (currently winusb.sys). The exception is the libusb0.sys filter driver. In that case, you use libusb0.sys as the upper filter driver on top of the existing kernel driver. On the other hand, libusb-win32 filter driver is not really widely used due to issues with older versions. > All in all what we've today is a kludge, and if we want to provide > a "seamless" user experience we need to fix it. > > Don't get me wrong usbfs is a really nice solution for driving > usb devices from userspace, like scanners and all other sorts of > devices. But what all these devices have in common is that they > have no kernel driver. Having a userspace based driver and a kernel > driver "fight it out" just does not work well. > All in all, I think this is a good summary. There are cases where a kernel driver and a usbfs are both used, an example is FTDI device where ftdi-sio driver and the user space libftdi (based on libusb) are used, but there are problems with that as well with regard to switching between the two drivers. There are a few discussions here. http://libusb.6.n5.nabble.com/Patch-libusb-os-linux-usbfs-c-Distingush-between-usbfs-and-other-kernel-mode-drivers-td3199947.html http://libusb.6.n5.nabble.com/open-device-exclusively-td4524397.html And there are no good solutions to a "simple" issue as the above mentioned. Not so sure if this is a good suggestion or not: just wondering how the other side (eg Windows) deal with these dual mode cameras? Does Windows use single driver or does Windows use two drivers for these dual mode cameras? Since they are customized device, then there must be a vendor driver for them. And since they will function as a video cam and a still cam device, I assume there will be standard Windows drivers on top of the vendor drivers. So there will be more than one drivers associated with them. -- Xiaofan ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 20:57 ` Hans de Goede 2011-08-10 2:05 ` Xiaofan Chen @ 2011-08-10 23:04 ` Adam Baker 2011-08-11 8:14 ` Hans de Goede 1 sibling, 1 reply; 90+ messages in thread From: Adam Baker @ 2011-08-10 23:04 UTC (permalink / raw) To: Hans de Goede Cc: Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore On Tuesday 09 August 2011, Hans de Goede wrote: > Hi, > > On 08/09/2011 10:31 PM, Adam Baker wrote: > > On Tuesday 09 August 2011, Hans de Goede wrote: > <snip> > > > It has also just occured to me that it might be possible to solve the > > issues we are facing just in the kernel. At the moment when the kernel > > performs a USBDEVFS_DISCONNECT it keeps the kernel driver locked out > > until userspace performs a USBDEVFS_CONNECT. If the kernel reattached > > the kernel driver when the device file was closed then, as gvfs doesn't > > keep the file open the biggest current issue would be solved instantly. > > If a mechanism could be found to prevent USBDEVFS_DISCONNECT from > > succeeding when the corresponding /dev/videox file was open then that > > would seem to be a reasonable solution. > > <sigh> > > This has been discussed over and over and over again, playing clever > tricks with USBDEVFS_[DIS]CONNECT like adding a new USBDEVFS_TRYDISCONNECT > which the v4l2 driver could intercept won't cut it. We need some central > manager of the device doing multiplexing between the 2 functions, and you > can *not* assume that either side will be nice wrt closing file > descriptors. > > Examples: > 1) You are wrong wrt gvfs, it does keep the libgphoto2 context open all the > time, and through that the usbfs device nodes. It seems that that depends, on my system gvfs isn't actually automounting the camera after it detects it and the file is only open (according to lsof) when the device is actually mounted. As soon as you unmount it the device gets closed again. Because it does do a brief open, USBDEVFS_DISCONNECT then close at connection time it does still disable the kernel driver. > > 2) Lets say a user starts a photo managing app like f-spot, and that opens > the device through libgphoto2 on startup, then the user switches to another > virtual desktop and forgets all about having f-spot open. Notice that if > the user now tries to stream he will not get a busy error, but the app > trying to do the streaming will simply not see the camera at all (kernel > driver unbound /dev/video# node is gone). This does seem like a situation where your approach could potentially give a better user experience. I'm wondering slightly how you define busy though. For webcams the streamon and streamoff ioctls tell you if you are using mmap or userptr transfers but you don't know if when the user has finished if they just use read. For stillcam mode it is again hard to determine a busy condition other than being in the middle of transfering an individual picture. > > 3) Notice that little speaker icon in your panel on your average Linux > desktop, that keeps the mixer of the audio device open *all the time* it > is quite easy to imagine a similar applet for v4l2 device controls (see > for example gtk-v4l) doing the same. Or a user could simply start up a > v4l2 control panel app like gtk-v4l, qv4l2 or v4l2ucp, and leave it running > minimized ... > This again needs a usable concept of busy > 4) Some laptops have a Fn + F## key which enables / disables the builtin > webcam by turning its power on / off. Effectively plugging it into / out > of a usb port. We would like to have an on screen notification of this one > day like we have now for brightness and volume controls, based on udev > events. But the current dual mode cam stuff causes udev events for > a *new* video device being added / an existing one being removed > each time libgphoto2 releases / takes control of the camera. > Would such a system know what camera is supposed to be the internal one so it doesn't show the camera as turned on just because you plug in an external camera. If so then it won't turn on and off as an external camera changes modes. If not then showing on when any camera is usable and off when it isn't seems like sensible behaviour. > 5) More in general, more and more software is dynamically monitoring the > addition / removal of (usb) devices using udev, our current solution > suggests to this software the /dev/video device is being unplugged / > re-plugged all the time, not pretty. > > > All in all what we've today is a kludge, and if we want to provide > a "seamless" user experience we need to fix it. I think in summary I'm concerned about the possibility of perfect being the enemy of good enough. At the moment we've got a significant usability problem (a web search for gvfs-gphoto2-volume-monitor turns up mostly instructions on how to disable it). If we come up with a solution that whilst it would be perfect there isn't enough effort available to implement then that is worse than a solution that fixes most of the problem. This is an even greater concern when the technically superior solution has a higher long term maintenance overhead (as we no longer get Win32 and OSX users helping to maintain the stillcam drivers). I'm not sure if there is anything in this discussion that is relevant to the cameras in phones or tablets. These appear to the user as if they are dual mode devices but they don't have any independent storage - taking a photo is more like capturing a single higher than usual res frame so I suspect they aren't going to be an issue. Regards Adam ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 23:04 ` Adam Baker @ 2011-08-11 8:14 ` Hans de Goede 0 siblings, 0 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-11 8:14 UTC (permalink / raw) To: Adam Baker Cc: Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore Hi, On 08/11/2011 01:04 AM, Adam Baker wrote: > On Tuesday 09 August 2011, Hans de Goede wrote: >> Hi, >> >> On 08/09/2011 10:31 PM, Adam Baker wrote: >>> On Tuesday 09 August 2011, Hans de Goede wrote: >> <snip> >> >>> It has also just occured to me that it might be possible to solve the >>> issues we are facing just in the kernel. At the moment when the kernel >>> performs a USBDEVFS_DISCONNECT it keeps the kernel driver locked out >>> until userspace performs a USBDEVFS_CONNECT. If the kernel reattached >>> the kernel driver when the device file was closed then, as gvfs doesn't >>> keep the file open the biggest current issue would be solved instantly. >>> If a mechanism could be found to prevent USBDEVFS_DISCONNECT from >>> succeeding when the corresponding /dev/videox file was open then that >>> would seem to be a reasonable solution. >> >> <sigh> >> >> This has been discussed over and over and over again, playing clever >> tricks with USBDEVFS_[DIS]CONNECT like adding a new USBDEVFS_TRYDISCONNECT >> which the v4l2 driver could intercept won't cut it. We need some central >> manager of the device doing multiplexing between the 2 functions, and you >> can *not* assume that either side will be nice wrt closing file >> descriptors. >> >> Examples: >> 1) You are wrong wrt gvfs, it does keep the libgphoto2 context open all the >> time, and through that the usbfs device nodes. > > It seems that that depends, on my system gvfs isn't actually automounting the > camera after it detects it and the file is only open (according to lsof) when > the device is actually mounted. Right, but on some systems it does auto-mount, and even if it does not auto-mount and the user then just clicks on its icon in the filemanager to look at the photo's it will mount, and it will be non obvious to the user to umount it. Once mounted, the /dev/video# node will be gone. So if for example skype gets started it will say the user has no camera. If it were to say the device is busy, the user just might make a link to the mounted gvfs share, but even then that is a big might. The whole disappear / reappear act of the /dev/video# node is one of my concerns with the current solution. > As soon as you unmount it the device gets > closed again. Because it does do a brief open, USBDEVFS_DISCONNECT then close > at connection time it does still disable the kernel driver. Then you've a bug in your libgphoto2, Fedora had a similar bug which I fixed (and send the fix upstream). libgphoto2 now a days does make the call to re-attach the kernel driver. > >> >> 2) Lets say a user starts a photo managing app like f-spot, and that opens >> the device through libgphoto2 on startup, then the user switches to another >> virtual desktop and forgets all about having f-spot open. Notice that if >> the user now tries to stream he will not get a busy error, but the app >> trying to do the streaming will simply not see the camera at all (kernel >> driver unbound /dev/video# node is gone). > > This does seem like a situation where your approach could potentially give a > better user experience. I'm wondering slightly how you define busy though. For > webcams the streamon and streamoff ioctls tell you if you are using mmap or > userptr transfers but you don't know if when the user has finished if they > just use read. For stillcam mode it is again hard to determine a busy > condition other than being in the middle of transfering an individual picture. Right, so my plan is to define busy for stillcam mode as not being in the middle of some operation (be it download or delete). >> 3) Notice that little speaker icon in your panel on your average Linux >> desktop, that keeps the mixer of the audio device open *all the time* it >> is quite easy to imagine a similar applet for v4l2 device controls (see >> for example gtk-v4l) doing the same. Or a user could simply start up a >> v4l2 control panel app like gtk-v4l, qv4l2 or v4l2ucp, and leave it running >> minimized ... >> > > This again needs a usable concept of busy Actually this one is easy. Most streaming drivers already handle controls while not streaming by remembering the set values, and these cached values usually then get set on stream start. There are multiple reasons for this: - usb transfers are expensive no need to send a value to the device if it may change before it gets actually used - in many cases parts of the device are powered down when not streaming, so sending values while not streaming just drops them in a black hole <snip> > If we come up with a solution that whilst it would be > perfect there isn't enough effort available to implement then that is worse > than a solution that fixes most of the problem. Agreed. > This is an even greater > concern when the technically superior solution has a higher long term > maintenance overhead (as we no longer get Win32 and OSX users helping to > maintain the stillcam drivers). I've been subscribed to the libgphoto2-devel list for quite some time now, I believe I still have to see the first win32 / OS-X user input. We do get occasional FreeBSD input, usually in the form of "it no longer compiles". I'm not saying that portability is not important, but contributions from users on other platforms should not be overrated either. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 19:57 ` Hans de Goede 2011-08-09 20:31 ` Adam Baker @ 2011-08-09 23:05 ` Theodore Kilgore 2011-08-10 14:19 ` Alan Stern 2 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-09 23:05 UTC (permalink / raw) To: Hans de Goede Cc: Alan Stern, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Tue, 9 Aug 2011, Hans de Goede wrote: > Hi, > > On 08/09/2011 04:19 PM, Alan Stern wrote: > > On Tue, 9 Aug 2011, Hans de Goede wrote: > > > > > I would really like to see the dual mode camera and TV tuner discussion > > > separated. They are 2 different issues AFAIK. > > > > > > 1) Dual mode cameras: > > > > > > In the case of the dual mode camera we have 1 single device (both at > > > the hardware level and at the logical block level), which can do 2 things, > > > but not at the same time. It can stream live video data from a sensor, > > > or it can retrieve earlier taken pictures from some picture memory. > > > > > > Unfortunately even though these 2 functions live in a single logical > > > block, > > > historically we've developed 2 drivers for them. This leads to fighting > > > over device ownership (surprise surprise), and to me the solution is > > > very clear, 1 logical block == 1 driver. > > > > According to Theodore, we have developed 5 drivers for them because the > > stillcam modes in different devices use four different vendor-specific > > drivers. > > Yes, but so the the webcam modes of the different devices, so for > the 5 (not sure if that is the right number) dual-cam mode chipsets > we support there will be 5 drivers, each supporting both the > webcam and the access to pictures stored in memory of the chipset > they support. So 5 chipsets -> 5 drivers each supporting 1 chipset, > and both functions of the single logical device that chipset > represents. > > > Does it really make sense to combine 5 drivers into one? > > Right, that is not the plan. The plan is to simply stop having 2 drivers > for 1 logical (and physical) block. So we go from 10 drivers, 5 stillcam > + 5 webcam, to just 5 drivers. We will also likely be able to share > code between the code for the 2 functionalities for things like generic > set / get register functions, initialization, etc. Probably this is a good time for some inventory to be done. sure of the exact number of drivers. The jeilinj driver does not count here. It is a standard mass storage device as a still camera, and if it is going to be hooked up as a webcam then it has to have some buttons pushed just so, prior to hookup -- after which it comes up with a different USB Product number as a proprietary webcam device, supported by the jeilinj driver. Thus, this is not one of the droids we are looking for. Specifically, the currently affected drivers are sq905, sq905c, mr97310a, and sn9c2028. Fuji Finepix seems to produce both still cameras (PTP protocol) and webcams. If there are currently any dual-mode cameras, it is not obvious. Perhaps any of these which can be dual mode are like the jeilinj cameras and come up under different Product numbers in the different modes, which is not a problem. There do seem to exist stv06xx dual-mode cameras: stv0680/stv0680.c: { "STM:USB Dual-mode camera", 0x0553, 0x0202, 1 }, is found in camlibs/stv0680, for example, along with a whole bunch of other cameras with the same Vendor:Product number. It seems, too, that the same Vendor:Product number is listed in gspca/stv0680.c So I guess we found another affected driver. The intersection seems to be void between the lists of the spca50x cameras and webcams, though one can not be certain about the future. I do know of another dualmode camera, powered by the JL2005A chip. I wrote a kernel module for it some months ago. Said module has not been sent upward because of mysterious issues with one of the three cameras used for testing it. We also find among the gspca driver files this * GSPCA sub driver for W996[78]CF JPEG USB Dual Mode Camera Chip. * * Copyright (C) 2009 Hans de Goede <hdegoede@redhat.com> but I am not sure about which cameras are using this. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 19:57 ` Hans de Goede 2011-08-09 20:31 ` Adam Baker 2011-08-09 23:05 ` Theodore Kilgore @ 2011-08-10 14:19 ` Alan Stern 2011-08-10 15:03 ` Theodore Kilgore 2 siblings, 1 reply; 90+ messages in thread From: Alan Stern @ 2011-08-10 14:19 UTC (permalink / raw) To: Hans de Goede Cc: Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker On Tue, 9 Aug 2011, Hans de Goede wrote: > Hi, > > On 08/09/2011 04:19 PM, Alan Stern wrote: > > On Tue, 9 Aug 2011, Hans de Goede wrote: > > According to Theodore, we have developed 5 drivers for them because the > > stillcam modes in different devices use four different vendor-specific > > drivers. > > Yes, but so the the webcam modes of the different devices, so for > the 5 (not sure if that is the right number) dual-cam mode chipsets > we support there will be 5 drivers, each supporting both the > webcam and the access to pictures stored in memory of the chipset > they support. So 5 chipsets -> 5 drivers each supporting 1 chipset, > and both functions of the single logical device that chipset > represents. > > > Does it really make sense to combine 5 drivers into one? > > Right, that is not the plan. The plan is to simply stop having 2 drivers > for 1 logical (and physical) block. So we go from 10 drivers, 5 stillcam > + 5 webcam, to just 5 drivers. We will also likely be able to share > code between the code for the 2 functionalities for things like generic > set / get register functions, initialization, etc. Okay, I didn't realize that the different cameras used different webcam drivers as well as different stillcam drivers. As far as I can see, there's nothing to stop anybody from adding the stillcam functionality into the webcam drivers right now. If some common code can be abstracted out into a shared source file, so much the better. That would solve the problem, right? Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 14:19 ` Alan Stern @ 2011-08-10 15:03 ` Theodore Kilgore 2011-08-10 16:09 ` Alan Stern 0 siblings, 1 reply; 90+ messages in thread From: Theodore Kilgore @ 2011-08-10 15:03 UTC (permalink / raw) To: Alan Stern Cc: Hans de Goede, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011, Alan Stern wrote: > On Tue, 9 Aug 2011, Hans de Goede wrote: > > > Hi, > > > > On 08/09/2011 04:19 PM, Alan Stern wrote: > > > On Tue, 9 Aug 2011, Hans de Goede wrote: > > > According to Theodore, we have developed 5 drivers for them because the > > > stillcam modes in different devices use four different vendor-specific > > > drivers. > > > > Yes, but so the the webcam modes of the different devices, so for > > the 5 (not sure if that is the right number) dual-cam mode chipsets > > we support there will be 5 drivers, each supporting both the > > webcam and the access to pictures stored in memory of the chipset > > they support. So 5 chipsets -> 5 drivers each supporting 1 chipset, > > and both functions of the single logical device that chipset > > represents. > > > > > Does it really make sense to combine 5 drivers into one? > > > > Right, that is not the plan. The plan is to simply stop having 2 drivers > > for 1 logical (and physical) block. So we go from 10 drivers, 5 stillcam > > + 5 webcam, to just 5 drivers. We will also likely be able to share > > code between the code for the 2 functionalities for things like generic > > set / get register functions, initialization, etc. > > Okay, I didn't realize that the different cameras used different webcam > drivers as well as different stillcam drivers. Oh, yes. They are Proprietary devices. And that means what it says. :-) And all different from each other, too. > As far as I can see, there's nothing to stop anybody from adding the > stillcam functionality into the webcam drivers right now. If some > common code can be abstracted out into a shared source file, so much > the better. > > That would solve the problem, right? I think everyone involved believes that it would solve the problem. The question has been all along whether or not there is any other way which would work. Also the question of what, exactly, "belongs" in the kernel and what does not. For, if something has been historically supported in userspace (stillcam support, in this case) and has worked well there, I would think it is kind of too bad to have to move said support into the kernel just because the same hardware requires kernel support for another functionality and the two sides clash. I mean, the kernel is already big enough, no? But the logic that Hans has set forth seems rather compelling. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 15:03 ` Theodore Kilgore @ 2011-08-10 16:09 ` Alan Stern 2011-08-10 18:33 ` Theodore Kilgore 2011-08-11 8:14 ` Hans de Goede 0 siblings, 2 replies; 90+ messages in thread From: Alan Stern @ 2011-08-10 16:09 UTC (permalink / raw) To: Theodore Kilgore Cc: Hans de Goede, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011, Theodore Kilgore wrote: > > Okay, I didn't realize that the different cameras used different webcam > > drivers as well as different stillcam drivers. > > Oh, yes. They are Proprietary devices. And that means what it says. :-) > And all different from each other, too. > > > As far as I can see, there's nothing to stop anybody from adding the > > stillcam functionality into the webcam drivers right now. If some > > common code can be abstracted out into a shared source file, so much > > the better. > > > > That would solve the problem, right? > > I think everyone involved believes that it would solve the problem. > > The question has been all along whether or not there is any other way > which would work. Also the question of what, exactly, "belongs" in the > kernel and what does not. For, if something has been historically > supported in userspace (stillcam support, in this case) and has worked > well there, I would think it is kind of too bad to have to move said > support into the kernel just because the same hardware requires kernel > support for another functionality and the two sides clash. I mean, the > kernel is already big enough, no? But the logic that Hans has set forth > seems rather compelling. The alternative seems to be to define a device-sharing protocol for USB drivers. Kernel drivers would implement a new callback (asking them to give up control of the device), and usbfs would implement new ioctls by which a program could ask for and relinquish control of a device. The amount of rewriting needed would be relatively small. A few loose ends would remain, such as how to handle suspends, resumes, resets, and disconnects. Assuming usbfs is the only driver that will want to share a device in this way, we could handle them. Hans, what do you think? Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 16:09 ` Alan Stern @ 2011-08-10 18:33 ` Theodore Kilgore 2011-08-10 19:39 ` Hans Verkuil ` (2 more replies) 2011-08-11 8:14 ` Hans de Goede 1 sibling, 3 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-10 18:33 UTC (permalink / raw) To: Alan Stern Cc: Hans de Goede, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011, Alan Stern wrote: > On Wed, 10 Aug 2011, Theodore Kilgore wrote: > > > > Okay, I didn't realize that the different cameras used different webcam > > > drivers as well as different stillcam drivers. > > > > Oh, yes. They are Proprietary devices. And that means what it says. :-) > > And all different from each other, too. > > > > > As far as I can see, there's nothing to stop anybody from adding the > > > stillcam functionality into the webcam drivers right now. If some > > > common code can be abstracted out into a shared source file, so much > > > the better. > > > > > > That would solve the problem, right? > > > > I think everyone involved believes that it would solve the problem. > > > > The question has been all along whether or not there is any other way > > which would work. Also the question of what, exactly, "belongs" in the > > kernel and what does not. For, if something has been historically > > supported in userspace (stillcam support, in this case) and has worked > > well there, I would think it is kind of too bad to have to move said > > support into the kernel just because the same hardware requires kernel > > support for another functionality and the two sides clash. I mean, the > > kernel is already big enough, no? But the logic that Hans has set forth > > seems rather compelling. > > The alternative seems to be to define a device-sharing protocol for USB > drivers. Kernel drivers would implement a new callback (asking them to > give up control of the device), and usbfs would implement new ioctls by > which a program could ask for and relinquish control of a device. The > amount of rewriting needed would be relatively small. > > A few loose ends would remain, such as how to handle suspends, resumes, > resets, and disconnects. Assuming usbfs is the only driver that will > want to share a device in this way, we could handle them. > > Hans, what do you think? > > Alan Stern Alan, Hans seems to have argued cogently for doing all of this in the kernel and for abandoning the usbfs-based drivers for these particular drivers for dual-mode cameras and, I would conjecture, for drivers for dual-mode hardware in general. Therefore, I anticipate that he won't like that very much. My position: I do not have preconceptions about how the problem gets handled, and at this point I remain agnostic and believe that all approaches ought to be carefully analysed. I can imagine, abstractly, that things like this could be handled by -- moving all basic functionality to the kernel, and fixing the relevant libgphoto2 drivers to look to the kernel instead of to libusb. (What Hans argues for, and I am not opposed if his arguments convince other concerned parties) -- doing some kind of patch job to make current arrangement somehow to work better (this seems to be the position of Adam Baker; I do share the skepticism Hans has expressed about how well this could all be pasted together) -- doing something like the previous, but also figuring out how to bring udev rules into play, which would make it all work better (just tossing this one in, for laughs, but who knows someone might like it) -- moving the kernel webcam drivers out of the kernel and doing with these cameras _everything_ including webcam function through libusb. I myself do not have the imagination to be able to figure out how this could be done without a rather humongous amount of work (for example, which streaming apps that are currently available would be able to live with this?) but unless I misunderstand what he was saying, Greg K-H seems to think that this would be the best thing to do. Which one of these possibile approaches gets adopted is a policy issue which I would consider is ultimately way above my pay grade. My main motivation for bringing up the issue was to get it to the front burner so that _something_ gets done. It is a matter which has been left alone for too long. Therefore, I am very glad that the matter is being addressed. Let me add to this that I have gotten permission for time off and for a expense money which might possibly cover my air fare. I hope to arrive in Vancouver by sometime on Monday and intend to attend the mini-summit. I suggest that we get all intersted parties together and figure out what is the best way to go. I hope everyone who is actively concerned can meet in Vancouver, and if all goes well then on Monday as well as Tuesday. I can hang around for another day or two after Tuesday, but I do not expect to register for LinuxCon or be involved in it. When I leave Vancouver I will probably go to Seattle and spend a couple of days with my oldest son, the musician, before coming home on the next weekend. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 18:33 ` Theodore Kilgore @ 2011-08-10 19:39 ` Hans Verkuil 2011-08-10 19:43 ` Greg KH 2011-08-10 20:14 ` Mauro Carvalho Chehab 2 siblings, 0 replies; 90+ messages in thread From: Hans Verkuil @ 2011-08-10 19:39 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wednesday, August 10, 2011 20:33:25 Theodore Kilgore wrote: > > On Wed, 10 Aug 2011, Alan Stern wrote: > > > On Wed, 10 Aug 2011, Theodore Kilgore wrote: > > > > > > Okay, I didn't realize that the different cameras used different webcam > > > > drivers as well as different stillcam drivers. > > > > > > Oh, yes. They are Proprietary devices. And that means what it says. :-) > > > And all different from each other, too. > > > > > > > As far as I can see, there's nothing to stop anybody from adding the > > > > stillcam functionality into the webcam drivers right now. If some > > > > common code can be abstracted out into a shared source file, so much > > > > the better. > > > > > > > > That would solve the problem, right? > > > > > > I think everyone involved believes that it would solve the problem. > > > > > > The question has been all along whether or not there is any other way > > > which would work. Also the question of what, exactly, "belongs" in the > > > kernel and what does not. For, if something has been historically > > > supported in userspace (stillcam support, in this case) and has worked > > > well there, I would think it is kind of too bad to have to move said > > > support into the kernel just because the same hardware requires kernel > > > support for another functionality and the two sides clash. I mean, the > > > kernel is already big enough, no? But the logic that Hans has set forth > > > seems rather compelling. > > > > The alternative seems to be to define a device-sharing protocol for USB > > drivers. Kernel drivers would implement a new callback (asking them to > > give up control of the device), and usbfs would implement new ioctls by > > which a program could ask for and relinquish control of a device. The > > amount of rewriting needed would be relatively small. > > > > A few loose ends would remain, such as how to handle suspends, resumes, > > resets, and disconnects. Assuming usbfs is the only driver that will > > want to share a device in this way, we could handle them. > > > > Hans, what do you think? > > > > Alan Stern > > Alan, > > Hans seems to have argued cogently for doing all of this in the kernel and > for abandoning the usbfs-based drivers for these particular drivers for > dual-mode cameras and, I would conjecture, for drivers for dual-mode > hardware in general. Therefore, I anticipate that he won't like that very > much. > > My position: > > I do not have preconceptions about how the problem gets handled, and at > this point I remain agnostic and believe that all approaches ought to be > carefully analysed. I can imagine, abstractly, that things like this > could be handled by > > -- moving all basic functionality to the kernel, and fixing the > relevant libgphoto2 drivers to look to the kernel instead of to libusb. > (What Hans argues for, and I am not opposed if his arguments convince > other concerned parties) > > -- doing some kind of patch job to make current arrangement somehow to > work better (this seems to be the position of Adam Baker; I do share > the skepticism Hans has expressed about how well this could all be > pasted together) > > -- doing something like the previous, but also figuring out how to bring > udev rules into play, which would make it all work better (just tossing > this one in, for laughs, but who knows someone might like it) > > -- moving the kernel webcam drivers out of the kernel and doing with these > cameras _everything_ including webcam function through libusb. I myself do > not have the imagination to be able to figure out how this could be done > without a rather humongous amount of work (for example, which streaming > apps that are currently available would be able to live with this?) but > unless I misunderstand what he was saying, Greg K-H seems to think that > this would be the best thing to do. Webcam drivers can rely on i2c sensor drivers to handle the sensor. These sensor drivers can be used in other non-USB devices as well (particularly embedded SoC devices). Moving webcam drivers out of the kernel would likely mean that you end up with duplicate kernel and userspace sensor drivers. Not a good idea. In all fairness I have to say that most webcam drivers program the sensor directly instead of relying on an i2c driver. Often because they were reverse engineered. But I know of several that should really be split into a webcam driver and a separate i2c sensor driver. Video hardware tends to rely heavily on i2c devices hooked up to them. The big advantage of keeping everything in the kernel is that that makes i2c driver sharing between USB, PCI, SoC, etc. much easier. Video hardware also often has to toggle between different modes or functionalities (e.g. analog TV vs DVB, MPEG encoded video vs raw). Having all components in the kernel makes this a much easier job. I think the approach favored by Hans de Goede is the right one based on what I've read. Whatever solution is chosen, in no way should that ever force us to duplicate i2c sensor control in userspace. This would completely defeat all our efforts to make sensor drivers reusable between webcams and SoCs. Regards, Hans > > Which one of these possibile approaches gets adopted is a policy issue > which I would consider is ultimately way above my pay grade. > > My main motivation for bringing up the issue was to get it to the front > burner so that _something_ gets done. It is a matter which has been left > alone for too long. Therefore, I am very glad that the matter is being > addressed. > > Let me add to this that I have gotten permission for time off and for a > expense money which might possibly cover my air fare. I hope to arrive in > Vancouver by sometime on Monday and intend to attend the mini-summit. I > suggest that we get all intersted parties together and figure out what is > the best way to go. > > I hope everyone who is actively concerned can meet in Vancouver, and if > all goes well then on Monday as well as Tuesday. I can hang around for > another day or two after Tuesday, but I do not expect to register for > LinuxCon or be involved in it. When I leave Vancouver I will probably go > to Seattle and spend a couple of days with my oldest son, the musician, > before coming home on the next weekend. > > Theodore Kilgore > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 18:33 ` Theodore Kilgore 2011-08-10 19:39 ` Hans Verkuil @ 2011-08-10 19:43 ` Greg KH 2011-08-10 20:34 ` Theodore Kilgore 2011-08-10 20:14 ` Mauro Carvalho Chehab 2 siblings, 1 reply; 90+ messages in thread From: Greg KH @ 2011-08-10 19:43 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Stern, Hans de Goede, Sarah Sharp, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, Aug 10, 2011 at 01:33:25PM -0500, Theodore Kilgore wrote: > -- moving the kernel webcam drivers out of the kernel and doing with these > cameras _everything_ including webcam function through libusb. I myself do > not have the imagination to be able to figure out how this could be done > without a rather humongous amount of work (for example, which streaming > apps that are currently available would be able to live with this?) but > unless I misunderstand what he was saying, Greg K-H seems to think that > this would be the best thing to do. No, I never said that. Or if I accidentally did, I do not think that is the best thing to do at all, sorry for any confusion. > Which one of these possibile approaches gets adopted is a policy issue > which I would consider is ultimately way above my pay grade. As no one is being paid here, there are no "pay grades", so don't worry about that :) greg k-h ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 19:43 ` Greg KH @ 2011-08-10 20:34 ` Theodore Kilgore 0 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-10 20:34 UTC (permalink / raw) To: Greg KH Cc: Alan Stern, Hans de Goede, Sarah Sharp, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011, Greg KH wrote: > On Wed, Aug 10, 2011 at 01:33:25PM -0500, Theodore Kilgore wrote: > > -- moving the kernel webcam drivers out of the kernel and doing with these > > cameras _everything_ including webcam function through libusb. I myself do > > not have the imagination to be able to figure out how this could be done > > without a rather humongous amount of work (for example, which streaming > > apps that are currently available would be able to live with this?) but > > unless I misunderstand what he was saying, Greg K-H seems to think that > > this would be the best thing to do. > > No, I never said that. Or if I accidentally did, I do not think that is > the best thing to do at all, sorry for any confusion. OK. > > Which one of these possibile approaches gets adopted is a policy issue > > which I would consider is ultimately way above my pay grade. > > As no one is being paid here, there are no "pay grades", so don't worry > about that :) I meant this, of course, in a metaphorical sense. Cheers, Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 18:33 ` Theodore Kilgore 2011-08-10 19:39 ` Hans Verkuil 2011-08-10 19:43 ` Greg KH @ 2011-08-10 20:14 ` Mauro Carvalho Chehab 2011-08-10 20:39 ` Theodore Kilgore 2011-08-11 8:57 ` Jean-Francois Moine 2 siblings, 2 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-10 20:14 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Em 10-08-2011 15:33, Theodore Kilgore escreveu: > Hans seems to have argued cogently for doing all of this in the kernel and > for abandoning the usbfs-based drivers for these particular drivers for > dual-mode cameras and, I would conjecture, for drivers for dual-mode > hardware in general. Therefore, I anticipate that he won't like that very > much. > > My position: > > I do not have preconceptions about how the problem gets handled, and at > this point I remain agnostic and believe that all approaches ought to be > carefully analysed. I can imagine, abstractly, that things like this > could be handled by > > -- moving all basic functionality to the kernel, and fixing the > relevant libgphoto2 drivers to look to the kernel instead of to libusb. > (What Hans argues for, and I am not opposed if his arguments convince > other concerned parties) Not looking on the amount of work to be done, I think that this would give better results, IMO. > -- doing some kind of patch job to make current arrangement somehow to > work better (this seems to be the position of Adam Baker; I do share > the skepticism Hans has expressed about how well this could all be > pasted together) Adam Baker's proposal of a locking between usbfs and the kernel driver seems to be interesting, but, as he pointed, there are some side effects to consider, like suspend/resume, PM, etc. > -- doing something like the previous, but also figuring out how to bring > udev rules into play, which would make it all work better (just tossing > this one in, for laughs, but who knows someone might like it) I don't think this is a good alternative. > -- moving the kernel webcam drivers out of the kernel and doing with these > cameras _everything_ including webcam function through libusb. I myself do > not have the imagination to be able to figure out how this could be done > without a rather humongous amount of work (for example, which streaming > apps that are currently available would be able to live with this?) but > unless I misunderstand what he was saying, Greg K-H seems to think that > this would be the best thing to do. I also don't think that this a good alternative. As Hans V. pointed, one of our long term targets is to create per-sensor I2C drivers that are independent from the bridges. Also, moving it to userspace would require lots of work with the duplication of V4L and gspca core into userspace for the devices that would be moved, and may have some performance impacts. > Which one of these possibile approaches gets adopted is a policy issue > which I would consider is ultimately way above my pay grade. > > My main motivation for bringing up the issue was to get it to the front > burner so that _something_ gets done. It is a matter which has been left > alone for too long. Therefore, I am very glad that the matter is being > addressed. > > Let me add to this that I have gotten permission for time off and for a > expense money which might possibly cover my air fare. I hope to arrive in > Vancouver by sometime on Monday and intend to attend the mini-summit. I > suggest that we get all intersted parties together and figure out what is > the best way to go. > > I hope everyone who is actively concerned can meet in Vancouver, and if > all goes well then on Monday as well as Tuesday. I can hang around for > another day or two after Tuesday, but I do not expect to register for > LinuxCon or be involved in it. It will be great to have you there for those discussions. > When I leave Vancouver I will probably go > to Seattle and spend a couple of days with my oldest son, the musician, > before coming home on the next weekend. > > Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 20:14 ` Mauro Carvalho Chehab @ 2011-08-10 20:39 ` Theodore Kilgore 2011-08-11 8:57 ` Jean-Francois Moine 1 sibling, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-10 20:39 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011, Mauro Carvalho Chehab wrote: > Em 10-08-2011 15:33, Theodore Kilgore escreveu: > > > Hans seems to have argued cogently for doing all of this in the kernel and > > for abandoning the usbfs-based drivers for these particular drivers for > > dual-mode cameras and, I would conjecture, for drivers for dual-mode > > hardware in general. Therefore, I anticipate that he won't like that very > > much. > > > > My position: > > > > I do not have preconceptions about how the problem gets handled, and at > > this point I remain agnostic and believe that all approaches ought to be > > carefully analysed. I can imagine, abstractly, that things like this > > could be handled by > > > > -- moving all basic functionality to the kernel, and fixing the > > relevant libgphoto2 drivers to look to the kernel instead of to libusb. > > (What Hans argues for, and I am not opposed if his arguments convince > > other concerned parties) > > Not looking on the amount of work to be done, I think that this would > give better results, IMO. Okay. I would guess that I am one of the guys who gets to do the work, though. > > > -- doing some kind of patch job to make current arrangement somehow to > > work better (this seems to be the position of Adam Baker; I do share > > the skepticism Hans has expressed about how well this could all be > > pasted together) > > Adam Baker's proposal of a locking between usbfs and the kernel driver seems > to be interesting, but, as he pointed, there are some side effects to consider, > like suspend/resume, PM, etc. > > > -- doing something like the previous, but also figuring out how to bring > > udev rules into play, which would make it all work better (just tossing > > this one in, for laughs, but who knows someone might like it) > > I don't think this is a good alternative. I was trying to mention all alternatives. I should have also mentioned "leave things the way they are" but that is certainly out the window. > > > -- moving the kernel webcam drivers out of the kernel and doing with these > > cameras _everything_ including webcam function through libusb. I myself do > > not have the imagination to be able to figure out how this could be done > > without a rather humongous amount of work (for example, which streaming > > apps that are currently available would be able to live with this?) but > > unless I misunderstand what he was saying, Greg K-H seems to think that > > this would be the best thing to do. > > I also don't think that this a good alternative. As Hans V. pointed, one of > our long term targets is to create per-sensor I2C drivers that are independent > from the bridges. Also, moving it to userspace would require lots of work > with the duplication of V4L and gspca core into userspace for the devices > that would be moved, and may have some performance impacts. A good argument, though it probably does not affect the devices on my list one way or the other. > > > Which one of these possibile approaches gets adopted is a policy issue > > which I would consider is ultimately way above my pay grade. > > > > My main motivation for bringing up the issue was to get it to the front > > burner so that _something_ gets done. It is a matter which has been left > > alone for too long. Therefore, I am very glad that the matter is being > > addressed. > > > > Let me add to this that I have gotten permission for time off and for a > > expense money which might possibly cover my air fare. I hope to arrive in > > Vancouver by sometime on Monday and intend to attend the mini-summit. I > > suggest that we get all intersted parties together and figure out what is > > the best way to go. > > > > I hope everyone who is actively concerned can meet in Vancouver, and if > > all goes well then on Monday as well as Tuesday. I can hang around for > > another day or two after Tuesday, but I do not expect to register for > > LinuxCon or be involved in it. > > It will be great to have you there for those discussions. My take on this was that it seems to have become important for me to attend, which, frankly, I was not expecting. So thanks for the nice words. > > > When I leave Vancouver I will probably go > > to Seattle and spend a couple of days with my oldest son, the musician, > > before coming home on the next weekend. > > > > Theodore Kilgore > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 20:14 ` Mauro Carvalho Chehab 2011-08-10 20:39 ` Theodore Kilgore @ 2011-08-11 8:57 ` Jean-Francois Moine 2011-08-11 11:19 ` Mauro Carvalho Chehab 1 sibling, 1 reply; 90+ messages in thread From: Jean-Francois Moine @ 2011-08-11 8:57 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Theodore Kilgore, Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Wed, 10 Aug 2011 17:14:05 -0300 Mauro Carvalho Chehab <mchehab@infradead.org> wrote: > > -- moving the kernel webcam drivers out of the kernel and doing with these > > cameras _everything_ including webcam function through libusb. I myself do > > not have the imagination to be able to figure out how this could be done > > without a rather humongous amount of work (for example, which streaming > > apps that are currently available would be able to live with this?) but > > unless I misunderstand what he was saying, Greg K-H seems to think that > > this would be the best thing to do. > > I also don't think that this a good alternative. As Hans V. pointed, one of > our long term targets is to create per-sensor I2C drivers that are independent > from the bridges. Also, moving it to userspace would require lots of work > with the duplication of V4L and gspca core into userspace for the devices > that would be moved, and may have some performance impacts. If only gspca subdrivers are concerned, moving them to usespace should be easy: only the gspca main has to be rewritten, and its best place is in the v4l library. -- Ken ar c'hentañ | ** Breizh ha Linux atav! ** Jef | http://moinejf.free.fr/ ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 8:57 ` Jean-Francois Moine @ 2011-08-11 11:19 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-11 11:19 UTC (permalink / raw) To: Jean-Francois Moine Cc: Theodore Kilgore, Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Em 11-08-2011 05:57, Jean-Francois Moine escreveu: > On Wed, 10 Aug 2011 17:14:05 -0300 > Mauro Carvalho Chehab <mchehab@infradead.org> wrote: > >>> -- moving the kernel webcam drivers out of the kernel and doing with these >>> cameras _everything_ including webcam function through libusb. I myself do >>> not have the imagination to be able to figure out how this could be done >>> without a rather humongous amount of work (for example, which streaming >>> apps that are currently available would be able to live with this?) but >>> unless I misunderstand what he was saying, Greg K-H seems to think that >>> this would be the best thing to do. >> >> I also don't think that this a good alternative. As Hans V. pointed, one of >> our long term targets is to create per-sensor I2C drivers that are independent >> from the bridges. Also, moving it to userspace would require lots of work >> with the duplication of V4L and gspca core into userspace for the devices >> that would be moved, and may have some performance impacts. > > If only gspca subdrivers are concerned, moving them to usespace should > be easy: only the gspca main has to be rewritten, and its best place is > in the v4l library. It would end by having two userspace drivers, as the data access driver is at libgphoto2. So, the problem will just be moved to userspace. You'll additionally need to deal with conflicts between kernelspace ALSA driver and userspace video driver. Regards, Mauro. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-10 16:09 ` Alan Stern 2011-08-10 18:33 ` Theodore Kilgore @ 2011-08-11 8:14 ` Hans de Goede 2011-08-11 14:56 ` Alan Stern 1 sibling, 1 reply; 90+ messages in thread From: Hans de Goede @ 2011-08-11 8:14 UTC (permalink / raw) To: Alan Stern Cc: Theodore Kilgore, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Hi, On 08/10/2011 06:09 PM, Alan Stern wrote: > On Wed, 10 Aug 2011, Theodore Kilgore wrote: > >>> Okay, I didn't realize that the different cameras used different webcam >>> drivers as well as different stillcam drivers. >> >> Oh, yes. They are Proprietary devices. And that means what it says. :-) >> And all different from each other, too. >> >>> As far as I can see, there's nothing to stop anybody from adding the >>> stillcam functionality into the webcam drivers right now. If some >>> common code can be abstracted out into a shared source file, so much >>> the better. >>> >>> That would solve the problem, right? >> >> I think everyone involved believes that it would solve the problem. >> >> The question has been all along whether or not there is any other way >> which would work. Also the question of what, exactly, "belongs" in the >> kernel and what does not. For, if something has been historically >> supported in userspace (stillcam support, in this case) and has worked >> well there, I would think it is kind of too bad to have to move said >> support into the kernel just because the same hardware requires kernel >> support for another functionality and the two sides clash. I mean, the >> kernel is already big enough, no? But the logic that Hans has set forth >> seems rather compelling. > > The alternative seems to be to define a device-sharing protocol for USB > drivers. Kernel drivers would implement a new callback (asking them to > give up control of the device), and usbfs would implement new ioctls by > which a program could ask for and relinquish control of a device. The > amount of rewriting needed would be relatively small. > > A few loose ends would remain, such as how to handle suspends, resumes, > resets, and disconnects. Assuming usbfs is the only driver that will > want to share a device in this way, we could handle them. > > Hans, what do you think? > First of all thanks for the constructive input! When you say: "device-sharing protocol", do you mean 2 drivers been attached, but only 1 being active. Or just some additional glue to make hand-over between them work better? I've 2 concerns with this approach: 1) Assuming we are going for the just make hand over work better solution we will still have the whole disappear / reappear act of the /dev/video# node, which I don't like at all. If for example skype gets started it will say the user has no camera. If it were to say the device is busy, the user just might make a link to some application using the device in stillcam mode still being open. 2) The whole notion of the device being in use is rather vague when it comes to the userspace parts of this. Simply leaving say F-Spot running, or having a gvfs libgphoto share mounted, should not lead to not being able to use the device in webcam mode. But currently it will. Fixing all users of libgphoto2 wrt this is unlikely to happen, and even if we do that now, more broken ones will likely come along later. I estimate 98% of all cameras are not dual mode cameras, so the average stillcam application developer will not test for this. That leaves us with fixing the busy notion inside libgphoto2, iow, release the device as soon as an operation has completed. This will be quite slow, since both drivers don't know anything about each other, they will just know there is some $random_other_driver. So they need to assume the device state is unclean and re-init the device from scratch each time. Where as if we have both functions in one driver, that can remember the actual device state and only make changes if needed, so downloading + deleting 10 photos will lead to setting it to stillcam mode once, rather then 20 times. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 8:14 ` Hans de Goede @ 2011-08-11 14:56 ` Alan Stern 2011-08-11 15:13 ` Mauro Carvalho Chehab 2011-08-12 7:26 ` Hans de Goede 0 siblings, 2 replies; 90+ messages in thread From: Alan Stern @ 2011-08-11 14:56 UTC (permalink / raw) To: Hans de Goede Cc: Theodore Kilgore, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Hans de Goede wrote: > > The alternative seems to be to define a device-sharing protocol for USB > > drivers. Kernel drivers would implement a new callback (asking them to > > give up control of the device), and usbfs would implement new ioctls by > > which a program could ask for and relinquish control of a device. The > > amount of rewriting needed would be relatively small. > > > > A few loose ends would remain, such as how to handle suspends, resumes, > > resets, and disconnects. Assuming usbfs is the only driver that will > > want to share a device in this way, we could handle them. > > > > Hans, what do you think? > > > > First of all thanks for the constructive input! > > When you say: "device-sharing protocol", do you mean 2 drivers been > attached, but only 1 being active. Or just some additional glue to make > hand-over between them work better? I was thinking that the webcam driver would always be attached, but from time to time usbfs would ask to use the device. When the webcam driver gives away control, it remains bound to the device but does not send any URBs. If it needs to send an URB, first it has to ask usbfs to give control back. > I've 2 concerns with this approach: > 1) Assuming we are going for the just make hand over work better solution > we will still have the whole disappear / reappear act of the /dev/video# > node, which I don't like at all. That will not happen any more, because the webcam driver will always be bound to the device. > If for example skype gets started it will say the user has no camera. If it > were to say the device is busy, the user just might make a link to some > application using the device in stillcam mode still being open. > > 2) The whole notion of the device being in use is rather vague when it comes > to the userspace parts of this. Simply leaving say F-Spot running, or having > a gvfs libgphoto share mounted, should not lead to not being able to use the > device in webcam mode. But currently it will. That's true -- but it's true no matter what solution we adopt. The various drivers (whether in the kernel or in userspace) will have to decide for themselves when they can give up control. > Fixing all users of libgphoto2 wrt this is unlikely to happen, and even if > we do that now, more broken ones will likely come along later. I estimate > 98% of all cameras are not dual mode cameras, so the average stillcam > application developer will not test for this. Not all users of libgphoto2 have to be changed; only those which manage dual-mode cameras. Adding a few ioctls to ask for and give up control at the appropriate times must be a lot easier than porting the entire driver into the kernel. > That leaves us with fixing the busy notion inside libgphoto2, iow, release > the device as soon as an operation has completed. This will be quite slow, > since both drivers don't know anything about each other, they will just > know there is some $random_other_driver. So they need to assume the > device state is unclean and re-init the device from scratch each time. Well, a user program can assume that the kernel driver left the device in a clean state. The reverse isn't always true, however -- it's one of the drawbacks of using a userspace driver. Besides, even though drivers don't always have to re-init the device from scratch every time, they do send all the current settings each time they use the device. So maybe the extra overhead is tolerable. > Where as if we have both functions in one driver, that can remember the > actual device state and only make changes if needed, so downloading + > deleting 10 photos will lead to setting it to stillcam mode once, rather > then 20 times. Depends how the ask-for-control ioctl is implemented. It might return a value indicating whether or not the webcam driver took control during the interval when the user program wasn't using the device. If usbfs retained control the entire time, the program should be able to assume the device's state hasn't changed. I'm not claiming that this is a better solution than putting everything in the kernel. Just that it is a workable alternative which would involve a lot less coding. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 14:56 ` Alan Stern @ 2011-08-11 15:13 ` Mauro Carvalho Chehab 2011-08-11 15:25 ` Alan Cox 2011-08-11 15:44 ` Alan Stern 2011-08-12 7:26 ` Hans de Goede 1 sibling, 2 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-11 15:13 UTC (permalink / raw) To: Alan Stern Cc: Hans de Goede, Theodore Kilgore, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Em 11-08-2011 11:56, Alan Stern escreveu: > On Thu, 11 Aug 2011, Hans de Goede wrote: > >>> The alternative seems to be to define a device-sharing protocol for USB >>> drivers. Kernel drivers would implement a new callback (asking them to >>> give up control of the device), and usbfs would implement new ioctls by >>> which a program could ask for and relinquish control of a device. The >>> amount of rewriting needed would be relatively small. >>> >>> A few loose ends would remain, such as how to handle suspends, resumes, >>> resets, and disconnects. Assuming usbfs is the only driver that will >>> want to share a device in this way, we could handle them. >>> >>> Hans, what do you think? >>> >> >> First of all thanks for the constructive input! >> >> When you say: "device-sharing protocol", do you mean 2 drivers been >> attached, but only 1 being active. Or just some additional glue to make >> hand-over between them work better? > > I was thinking that the webcam driver would always be attached, but > from time to time usbfs would ask to use the device. When the webcam > driver gives away control, it remains bound to the device but does not > send any URBs. If it needs to send an URB, first it has to ask usbfs > to give control back. > >> I've 2 concerns with this approach: >> 1) Assuming we are going for the just make hand over work better solution >> we will still have the whole disappear / reappear act of the /dev/video# >> node, which I don't like at all. > > That will not happen any more, because the webcam driver will always be > bound to the device. > >> If for example skype gets started it will say the user has no camera. If it >> were to say the device is busy, the user just might make a link to some >> application using the device in stillcam mode still being open. >> >> 2) The whole notion of the device being in use is rather vague when it comes >> to the userspace parts of this. Simply leaving say F-Spot running, or having >> a gvfs libgphoto share mounted, should not lead to not being able to use the >> device in webcam mode. But currently it will. > > That's true -- but it's true no matter what solution we adopt. The > various drivers (whether in the kernel or in userspace) will have to > decide for themselves when they can give up control. Between two or more kernel drivers, a resource locking mechanism like the one you've proposed works fine, but, when the driver is on userspace, there's one additional issue that needs to be addressed: What happens if, for example, if a camera application using libgphoto2 crashes? The lock will be enabled, and the V4L driver will be locked forever. Also, if the lock is made generic enough to protect between two different userspace applications, re-starting the camera application won't get the control back. To avoid such risk, kernel might need to implement some ugly hacks to detect when the application was killed, and put the device into a sane state, if this happens. >> Fixing all users of libgphoto2 wrt this is unlikely to happen, and even if >> we do that now, more broken ones will likely come along later. I estimate >> 98% of all cameras are not dual mode cameras, so the average stillcam >> application developer will not test for this. > > Not all users of libgphoto2 have to be changed; only those which manage > dual-mode cameras. Adding a few ioctls to ask for and give up control > at the appropriate times must be a lot easier than porting the entire > driver into the kernel. Again, applications that won't implement this control will take the lock forever. >> That leaves us with fixing the busy notion inside libgphoto2, iow, release >> the device as soon as an operation has completed. This will be quite slow, >> since both drivers don't know anything about each other, they will just >> know there is some $random_other_driver. So they need to assume the >> device state is unclean and re-init the device from scratch each time. > > Well, a user program can assume that the kernel driver left the device > in a clean state. The reverse isn't always true, however -- it's one > of the drawbacks of using a userspace driver. > > Besides, even though drivers don't always have to re-init the device > from scratch every time, they do send all the current settings each > time they use the device. So maybe the extra overhead is tolerable. > >> Where as if we have both functions in one driver, that can remember the >> actual device state and only make changes if needed, so downloading + >> deleting 10 photos will lead to setting it to stillcam mode once, rather >> then 20 times. > > Depends how the ask-for-control ioctl is implemented. It might return > a value indicating whether or not the webcam driver took control during > the interval when the user program wasn't using the device. If usbfs > retained control the entire time, the program should be able to assume > the device's state hasn't changed. > > I'm not claiming that this is a better solution than putting everything > in the kernel. Just that it is a workable alternative which would > involve a lot less coding. > > Alan Stern > ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 15:13 ` Mauro Carvalho Chehab @ 2011-08-11 15:25 ` Alan Cox 2011-08-11 15:49 ` Alan Stern 2011-08-11 15:44 ` Alan Stern 1 sibling, 1 reply; 90+ messages in thread From: Alan Cox @ 2011-08-11 15:25 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Alan Stern, Hans de Goede, Theodore Kilgore, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker > Between two or more kernel drivers, a resource locking mechanism like the one > you've proposed works fine, but, when the driver is on userspace, there's one > additional issue that needs to be addressed: What happens if, for example, > if a camera application using libgphoto2 crashes? The lock will be enabled, and > the V4L driver will be locked forever. Also, if the lock is made generic enough > to protect between two different userspace applications, re-starting the > camera application won't get the control back. Actually there are more issues than that - you've also got to worry about a security/permission model, and that is hard to get right, especially if you are not very careful that anything that can be retrieved which might violate the security model (eg the last frame on the capture) has been blanked before handover etc. > To avoid such risk, kernel might need to implement some ugly hacks to detect > when the application was killed, and put the device into a sane state, if this > happens. At which point history says it's easier to do the job right, once, in the kernel. > Again, applications that won't implement this control will take the lock forever. And applications that are touching both video (even indirectly) and still camera may get surprise deadlocks if they accidentally reference both the still and video device even via some library or service. > > Well, a user program can assume that the kernel driver left the device > > in a clean state. The reverse isn't always true, however -- it's one Not it cannot - the user program doesn't know a) if the kernel driver has ever been loaded b) the values the kernel driver leaves set (and those will change no doubt at times) c) if the camera has been plugged and unplugged and not yet had the kernel driver loaded To me it sounds like a recipe for disaster. For those tiny number of devices involved just use V4L and if need be some small V4L tweaks to handle still mode. In most cases the interface is basically identical and I'd bet much of the code is identical too. Alan ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 15:25 ` Alan Cox @ 2011-08-11 15:49 ` Alan Stern 2011-08-11 20:01 ` Theodore Kilgore 0 siblings, 1 reply; 90+ messages in thread From: Alan Stern @ 2011-08-11 15:49 UTC (permalink / raw) To: Alan Cox Cc: Mauro Carvalho Chehab, Hans de Goede, Theodore Kilgore, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Alan Cox wrote: > Actually there are more issues than that - you've also got to worry about > a security/permission model, and that is hard to get right, especially if > you are not very careful that anything that can be retrieved which might > violate the security model (eg the last frame on the capture) has been > blanked before handover etc. As far as I can tell, these same security issues exist today. I don't see them getting any worse than they are now. > And applications that are touching both video (even indirectly) and still > camera may get surprise deadlocks if they accidentally reference both the > still and video device even via some library or service. No, not deadlocks. Just -EBUSY errors. > > > Well, a user program can assume that the kernel driver left the device > > > in a clean state. The reverse isn't always true, however -- it's one > > Not it cannot - the user program doesn't know > > a) if the kernel driver has ever been loaded > b) the values the kernel driver leaves set (and those will change no > doubt at times) > c) if the camera has been plugged and unplugged and not yet had the > kernel driver loaded That's true. The program can't assume that a kernel driver was ever bound to the device; all it can assume is that _if_ a kernel driver was bound then it left the device in a sane state -- whatever "sane" might mean in this context. > To me it sounds like a recipe for disaster. For those tiny number of > devices involved just use V4L and if need be some small V4L tweaks to > handle still mode. In most cases the interface is basically identical and > I'd bet much of the code is identical too. I'm not against moving the whole thing into the kernel. I'm just pointing out that an easier-to-code-up solution will accomplish much the same result. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 15:49 ` Alan Stern @ 2011-08-11 20:01 ` Theodore Kilgore 2011-08-11 20:32 ` Mauro Carvalho Chehab 2011-08-12 1:07 ` Alan Stern 0 siblings, 2 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-11 20:01 UTC (permalink / raw) To: Alan Stern Cc: Alan Cox, Mauro Carvalho Chehab, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Alan Stern wrote: > On Thu, 11 Aug 2011, Alan Cox wrote: > > > Actually there are more issues than that - you've also got to worry about > > a security/permission model, and that is hard to get right, especially if > > you are not very careful that anything that can be retrieved which might > > violate the security model (eg the last frame on the capture) has been > > blanked before handover etc. > > As far as I can tell, these same security issues exist today. I don't > see them getting any worse than they are now. > > > And applications that are touching both video (even indirectly) and still > > camera may get surprise deadlocks if they accidentally reference both the > > still and video device even via some library or service. > > No, not deadlocks. Just -EBUSY errors. > > > > > Well, a user program can assume that the kernel driver left the device > > > > in a clean state. The reverse isn't always true, however -- it's one > > > > Not it cannot - the user program doesn't know > > > > a) if the kernel driver has ever been loaded > > b) the values the kernel driver leaves set (and those will change no > > doubt at times) > > c) if the camera has been plugged and unplugged and not yet had the > > kernel driver loaded > > That's true. The program can't assume that a kernel driver was ever > bound to the device; all it can assume is that _if_ a kernel driver was > bound then it left the device in a sane state -- whatever "sane" might > mean in this context. > > > To me it sounds like a recipe for disaster. For those tiny number of > > devices involved just use V4L and if need be some small V4L tweaks to > > handle still mode. In most cases the interface is basically identical and > > I'd bet much of the code is identical too. > > I'm not against moving the whole thing into the kernel. I'm just > pointing out that an easier-to-code-up solution will accomplish much > the same result. > > Alan Stern > Alan, As I said, I am agnostic, though leaning in the direction that Hans de Goede is pointing. What he says about a single control mechanism seems to make a lot of sense. If you can come up with an outline of the "easier to code" solution, that would be interesting, though. I assume you are also going to be in Vancouver? If you will be there on Monday, then Hans and I are already planning to meet and discuss. BTW, as to using V4L with "tweaks" to handle still mode, it would probably be more difficult than is imagined. For, though the operations required to process still images and webcam frames are in principle similar, the priorities and constraints are too different. Therefore, my understanding is that the libgphoto2 image processing routines, not the libv4l image processing routines, would still be used for still images. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 20:01 ` Theodore Kilgore @ 2011-08-11 20:32 ` Mauro Carvalho Chehab 2011-08-11 23:13 ` Theodore Kilgore ` (2 more replies) 2011-08-12 1:07 ` Alan Stern 1 sibling, 3 replies; 90+ messages in thread From: Mauro Carvalho Chehab @ 2011-08-11 20:32 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Stern, Alan Cox, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Em 11-08-2011 17:01, Theodore Kilgore escreveu: > As I said, I am agnostic, though leaning in the direction that Hans de > Goede is pointing. What he says about a single control mechanism seems to > make a lot of sense. If you can come up with an outline of the "easier to > code" solution, that would be interesting, though. > > I assume you are also going to be in Vancouver? If you will be there on > Monday, then Hans and I are already planning to meet and discuss. > > BTW, as to using V4L with "tweaks" to handle still mode, it would probably > be more difficult than is imagined. For, though the operations required to > process still images and webcam frames are in principle similar, the > priorities and constraints are too different. Therefore, my understanding > is that the libgphoto2 image processing routines, not the libv4l image > processing routines, would still be used for still images. I agree with Alan Cox: most of the code that the driver needs is already there: register read/write routines, bulk transfer support, etc. The amount of extra code for adding still cam functionality is probably not big. >From the kernel driver's perspective, it doesn't matter if the access will come via libv4l, libgphoto2 or whatever. The driver should be able to allow simultaneous open, while protecting the data access when userspace requests data stream or still image retrieve. instead of using the V4L2 device node to access the stored images, it probably makes more sense to use a separate device for that, that will handle a separate set of ioctl's, and just use read() to retrieve the image data, after selecting the desired image number, via ioctl(). It probably makes sense to add a new set of callbacks at the gspca core in order to handle the new device node, and letting it to avoid start streaming while the store access is happening, and vice-versa. Alternatively, we may create a separate "still cam" core library to handle the new device node,. If all agree around such solution, I suggest to take the most complex case and try to map it into the driver and core, and see how it behaves, testing with some simple command line applications, only changing the libgphoto2 code after those initial tests. Writing a simple code for reading still images should be easy, and we have already some testing tools for V4L2. After coding the core changes that are common to all drives, I suspect that adding the remaining 4 drivers will be quick. With regards to libgphoto2, all it needs to do is to test if the new device nodes exist. If they exist, then the new code will be used. Otherwise, it will fallback to the libusb. This way, we can incrementally add the Dual mode drivers into the kernel. There is one advantage on using this strategy: if, in the future, new Dual Cams arise, one can write first a still cam kernel driver, adding V4L support later. Cheers, Mauro ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 20:32 ` Mauro Carvalho Chehab @ 2011-08-11 23:13 ` Theodore Kilgore 2011-08-12 7:16 ` Hans de Goede 2011-08-12 10:11 ` Alan Cox 2 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-11 23:13 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Alan Stern, Alan Cox, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Mauro Carvalho Chehab wrote: > Em 11-08-2011 17:01, Theodore Kilgore escreveu: > > > As I said, I am agnostic, though leaning in the direction that Hans de > > Goede is pointing. What he says about a single control mechanism seems to > > make a lot of sense. If you can come up with an outline of the "easier to > > code" solution, that would be interesting, though. > > > > I assume you are also going to be in Vancouver? If you will be there on > > Monday, then Hans and I are already planning to meet and discuss. > > > > BTW, as to using V4L with "tweaks" to handle still mode, it would probably > > be more difficult than is imagined. For, though the operations required to > > process still images and webcam frames are in principle similar, the > > priorities and constraints are too different. Therefore, my understanding > > is that the libgphoto2 image processing routines, not the libv4l image > > processing routines, would still be used for still images. > > I agree with Alan Cox: most of the code that the driver needs is already there: > register read/write routines, bulk transfer support, etc. The amount of extra > code for adding still cam functionality is probably not big. > > >From the kernel driver's perspective, it doesn't matter if the access will come > via libv4l, libgphoto2 or whatever. The driver should be able to allow simultaneous > open, while protecting the data access when userspace requests data stream or > still image retrieve. > > instead of using the V4L2 device node to access the stored images, it probably makes > more sense to use a separate device for that, that will handle a separate set of > ioctl's, and just use read() to retrieve the image data, after selecting the desired > image number, via ioctl(). > > It probably makes sense to add a new set of callbacks at the gspca core in order > to handle the new device node, and letting it to avoid start streaming while the > store access is happening, and vice-versa. Alternatively, we may create a separate > "still cam" core library to handle the new device node,. > > If all agree around such solution, I suggest to take the most complex case and try > to map it into the driver and core, and see how it behaves, testing with some simple > command line applications, only changing the libgphoto2 code after those initial > tests. Writing a simple code for reading still images should be easy, and we have > already some testing tools for V4L2. > > After coding the core changes that are common to all drives, I suspect that adding > the remaining 4 drivers will be quick. > > With regards to libgphoto2, all it needs to do is to test if the new device nodes > exist. If they exist, then the new code will be used. Otherwise, it will fallback > to the libusb. This way, we can incrementally add the Dual mode drivers into the > kernel. > > There is one advantage on using this strategy: if, in the future, new Dual Cams > arise, one can write first a still cam kernel driver, adding V4L support later. > > Cheers, > Mauro > Try it out with one camera seems to me to be a reasonable way forward. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 20:32 ` Mauro Carvalho Chehab 2011-08-11 23:13 ` Theodore Kilgore @ 2011-08-12 7:16 ` Hans de Goede 2011-08-12 10:11 ` Alan Cox 2 siblings, 0 replies; 90+ messages in thread From: Hans de Goede @ 2011-08-12 7:16 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Theodore Kilgore, Alan Stern, Alan Cox, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Hi, On 08/11/2011 10:32 PM, Mauro Carvalho Chehab wrote: <snip stuff I agree with> > instead of using the V4L2 device node to access the stored images, it probably makes > more sense to use a separate device for that, that will handle a separate set of > ioctl's, and just use read() to retrieve the image data, after selecting the desired > image number, via ioctl(). I don't see a lot of added value in doing things this way. We can simply have a set of new ioctls on the /dev/video# node for this and a new V4L2_CAP_STILL_IMAGE to indicate the availability of these ioctls. This will keep the driver a lot simpler then doing 2 separate device nodes for 1 device. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 20:32 ` Mauro Carvalho Chehab 2011-08-11 23:13 ` Theodore Kilgore 2011-08-12 7:16 ` Hans de Goede @ 2011-08-12 10:11 ` Alan Cox 2 siblings, 0 replies; 90+ messages in thread From: Alan Cox @ 2011-08-12 10:11 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Theodore Kilgore, Alan Stern, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker > instead of using the V4L2 device node to access the stored images, it probably makes > more sense to use a separate device for that, that will handle a separate set of > ioctl's, and just use read() to retrieve the image data, after selecting the desired > image number, via ioctl(). How will you handle the permission model, what about devices where you need the V4L2 controls to take the picture in the first place ? It seems it should really be the same device and just a minor tweak to the API. V4L2 already provides frame reading and mapping functionality and the media layer provides post processing services which I could conceive some setups needing to use. gphoto would just open the v4l device ask for still image stuff and discover it wasn't available. ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 20:01 ` Theodore Kilgore 2011-08-11 20:32 ` Mauro Carvalho Chehab @ 2011-08-12 1:07 ` Alan Stern 2011-08-12 2:38 ` Theodore Kilgore 1 sibling, 1 reply; 90+ messages in thread From: Alan Stern @ 2011-08-12 1:07 UTC (permalink / raw) To: Theodore Kilgore Cc: Alan Cox, Mauro Carvalho Chehab, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Theodore Kilgore wrote: > Alan, > > As I said, I am agnostic, though leaning in the direction that Hans de > Goede is pointing. What he says about a single control mechanism seems to > make a lot of sense. If you can come up with an outline of the "easier to > code" solution, that would be interesting, though. That approach has been outlined in the other emails I have posted recently. > I assume you are also going to be in Vancouver? If you will be there on > Monday, then Hans and I are already planning to meet and discuss. No, I'm not going to Vancouver. However I will attend the Linux Plumbers conference in Santa Rosa. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-12 1:07 ` Alan Stern @ 2011-08-12 2:38 ` Theodore Kilgore 0 siblings, 0 replies; 90+ messages in thread From: Theodore Kilgore @ 2011-08-12 2:38 UTC (permalink / raw) To: Alan Stern Cc: Alan Cox, Mauro Carvalho Chehab, Hans de Goede, Sarah Sharp, Greg KH, linux-usb, linux-media, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Thu, 11 Aug 2011, Alan Stern wrote: <snip> > No, I'm not going to Vancouver. Well, someday, somewhere, we will be together in the same place at the same time and able to enjoy a good seafood restaurant. However I will attend the Linux > Plumbers conference in Santa Rosa. Which I expect I will not. Until next time we meet, cheers. Theodore Kilgore ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 15:13 ` Mauro Carvalho Chehab 2011-08-11 15:25 ` Alan Cox @ 2011-08-11 15:44 ` Alan Stern 1 sibling, 0 replies; 90+ messages in thread From: Alan Stern @ 2011-08-11 15:44 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Hans de Goede, Theodore Kilgore, Sarah Sharp, Greg KH, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Alan Cox raised a bunch of good points. I'm not going to respond to them; they pretty much speak for themselves. On Thu, 11 Aug 2011, Mauro Carvalho Chehab wrote: > Between two or more kernel drivers, a resource locking mechanism like the one > you've proposed works fine, It's not a locking mechanism. More like cooperative multi-access. > but, when the driver is on userspace, there's one > additional issue that needs to be addressed: What happens if, for example, > if a camera application using libgphoto2 crashes? The lock will be enabled, and > the V4L driver will be locked forever. Also, if the lock is made generic enough > to protect between two different userspace applications, re-starting the > camera application won't get the control back. You're wrong, because what I proposed isn't a lock. If the user program dies, the usbfs device file will automatically be closed and then usbfs will freely give control back to the webcam driver. If the program hangs at the wrong time, then the webcam driver won't be able to regain control. At least the user will have the option of killing the program when this happens; a similar hang in a kernel driver tends to be much uglier. Two different user programs trying to drive the device at the same time doesn't necessarily have to cause a problem. At any particular moment, only one of them would be in control of the device. > To avoid such risk, kernel might need to implement some ugly hacks to detect > when the application was killed, and put the device into a sane state, if this > happens. No ugly hack is needed. usbfs can directly inform the webcam driver whether or not the device file was closed while the user program retained control. If that didn't happen, it's safe to assume that the program gave up control voluntarily. > > Not all users of libgphoto2 have to be changed; only those which manage > > dual-mode cameras. Adding a few ioctls to ask for and give up control > > at the appropriate times must be a lot easier than porting the entire > > driver into the kernel. > > Again, applications that won't implement this control will take the lock forever. No, because there is no lock. Programs that haven't been changed will continue to behave as they do today -- they will unbind the webcam driver and assume full control of the device. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-11 14:56 ` Alan Stern 2011-08-11 15:13 ` Mauro Carvalho Chehab @ 2011-08-12 7:26 ` Hans de Goede 2011-08-12 15:36 ` Alan Stern 1 sibling, 1 reply; 90+ messages in thread From: Hans de Goede @ 2011-08-12 7:26 UTC (permalink / raw) To: Alan Stern Cc: Theodore Kilgore, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker Hi, On 08/11/2011 04:56 PM, Alan Stern wrote: > On Thu, 11 Aug 2011, Hans de Goede wrote: > >>> The alternative seems to be to define a device-sharing protocol for USB >>> drivers. Kernel drivers would implement a new callback (asking them to >>> give up control of the device), and usbfs would implement new ioctls by >>> which a program could ask for and relinquish control of a device. The >>> amount of rewriting needed would be relatively small. >>> >>> A few loose ends would remain, such as how to handle suspends, resumes, >>> resets, and disconnects. Assuming usbfs is the only driver that will >>> want to share a device in this way, we could handle them. >>> >>> Hans, what do you think? >>> >> >> First of all thanks for the constructive input! >> >> When you say: "device-sharing protocol", do you mean 2 drivers been >> attached, but only 1 being active. Or just some additional glue to make >> hand-over between them work better? > > I was thinking that the webcam driver would always be attached, but > from time to time usbfs would ask to use the device. When the webcam > driver gives away control, it remains bound to the device but does not > send any URBs. If it needs to send an URB, first it has to ask usbfs > to give control back. > Oh, interesting... <snip lots of good stuff> > I'm not claiming that this is a better solution than putting everything > in the kernel. Just that it is a workable alternative which would > involve a lot less coding. This is definitely an interesting proposal, something to think about ... I have 2 concerns wrt this approach: 1) It feels less clean then just having a single driver; and 2) I agree it will be less coding, but I doubt it will really be that much less work. It will likely need less new code (but a lot can be more or less copy pasted), but it will need changes across a wider array of subsystems / userspace components, requiring a lot of coordinating, getting patches merged in different projects, etc. So in the end I think it too will be quite a bit of work. I guess that what I'm trying to say here is, that if we are going to spend a significant amount of time on this, we might just as well go for the best solution we can come up with even if that is some more work. Regards, Hans ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-12 7:26 ` Hans de Goede @ 2011-08-12 15:36 ` Alan Stern 0 siblings, 0 replies; 90+ messages in thread From: Alan Stern @ 2011-08-12 15:36 UTC (permalink / raw) To: Hans de Goede Cc: Theodore Kilgore, Sarah Sharp, Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Adam Baker On Fri, 12 Aug 2011, Hans de Goede wrote: > > I'm not claiming that this is a better solution than putting everything > > in the kernel. Just that it is a workable alternative which would > > involve a lot less coding. > > This is definitely an interesting proposal, something to think about ... > > I have 2 concerns wrt this approach: > > 1) It feels less clean then just having a single driver; and Agreed. > 2) I agree it will be less coding, but I doubt it will really be that much > less work. It will likely need less new code (but a lot can be more or > less copy pasted), but it will need changes across a wider array of > subsystems / userspace components, requiring a lot of coordinating, > getting patches merged in different projects, etc. So in the end I > think it too will be quite a bit of work. > > I guess that what I'm trying to say here is, that if we are going to > spend a significant amount of time on this, we might just as well > go for the best solution we can come up with even if that is some > more work. Okay, go ahead. I have no objection. Alan Stern ^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: USB mini-summit at LinuxCon Vancouver 2011-08-09 7:52 ` Hans de Goede 2011-08-09 14:19 ` Alan Stern @ 2011-08-09 17:10 ` Sarah Sharp 1 sibling, 0 replies; 90+ messages in thread From: Sarah Sharp @ 2011-08-09 17:10 UTC (permalink / raw) To: Hans de Goede Cc: Greg KH, Mauro Carvalho Chehab, linux-usb, linux-media, linux-kernel, libusb-devel, Alexander Graf, Gerd Hoffmann, hector, Jan Kiszka, Stefan Hajnoczi, pbonzini, Anthony Liguori, Jes Sorensen, Alan Stern, Oliver Neukum, Felipe Balbi, Clemens Ladisch, Jaroslav Kysela, Takashi Iwai, Laurent Pinchart, Theodore Kilgore, Adam Baker On Tue, Aug 09, 2011 at 09:52:44AM +0200, Hans de Goede wrote: > Hi, > > On 08/08/2011 07:58 PM, Sarah Sharp wrote: > >On Fri, Aug 05, 2011 at 09:45:56AM +0200, Hans de Goede wrote: > >>Hi, > >> > >>On 08/05/2011 12:56 AM, Greg KH wrote: > >>>On Thu, Aug 04, 2011 at 07:21:47PM -0300, Mauro Carvalho Chehab wrote: > >>I think it is important to separate oranges from apples here, there are > >>at least 3 different problem classes which all seem to have gotten thrown > >>onto a pile here: > >> > >>1) The reason Mauro suggested having some discussion on this at the > >>USB summit is because of a discussion about dual mode cameras on the > >>linux media list. > >... > >>3) Re-direction of usb devices to virtual machines. This works by using > >>the userspace usbfs interface from qemu / vmware / virtualbox / whatever. > >>The basics of this work fine, but it lacks proper locking / safeguards > >>for when a vm takes over a usb device from the in kernel driver. > > > >Hi Hans and Mauro, > > > >We have do room in the schedule for the USB mini-summit for this > >discussion, since the schedule is still pretty flexible. The > >preliminary schedule is up here: > > > >http://userweb.kernel.org/~sarah/linuxcon-usb-minisummit.html > > > >I think it's best to discuss the VM redirection in the afternoon when > >some of the KVM folks join us after Hans' talk on USB redirection over > >the network. > > > > That seems best to me too. I'm available at both proposed time slots, > and I would like to join both discussions. > > >It sounds like we need a separate topic for the dual mode cameras and TV > >tuners. Mauro, do you want to lead that discussion in the early morning > >(in a 9:30 to 10:30 slot) or in the late afternoon (in a 15:30 to 16:30 > >slot)? I want to be sure we have all the video/media developers who are > >interested in this topic present, and I don't know if they will be going > >to the KVM forum. > > I would really like to see the dual mode camera and TV tuner discussion > separated. They are 2 different issues AFAIK. Ok, great, I've put both topics in the morning, in separate slots. Sarah Sharp ^ permalink raw reply [flat|nested] 90+ messages in thread
end of thread, other threads:[~2011-08-12 15:36 UTC | newest] Thread overview: 90+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-06-10 0:21 USB mini-summit at LinuxCon Vancouver Sarah Sharp 2011-06-10 3:18 ` Greg KH 2011-06-10 6:59 ` Gerd Hoffmann 2011-06-10 19:48 ` Sarah Sharp 2011-06-10 20:50 ` Greg KH 2011-06-13 10:44 ` Alexander Graf 2011-06-13 10:44 ` Alexander Graf 2011-06-13 16:29 ` Greg KH 2011-06-13 16:29 ` Greg KH 2011-06-13 17:11 ` Alexander Graf 2011-06-13 17:11 ` Alexander Graf 2011-06-10 7:19 ` Hans de Goede 2011-06-10 7:55 ` Improving kernel -> userspace (usbfs) usb device hand off Hans de Goede 2011-06-10 8:22 ` Felipe Balbi 2011-06-10 8:36 ` Hans de Goede 2011-06-10 8:42 ` Felipe Balbi 2011-06-10 12:19 ` Hans de Goede 2011-06-10 12:28 ` Felipe Balbi 2011-06-10 14:48 ` Alan Stern 2011-06-10 15:07 ` Mauro Carvalho Chehab 2011-06-10 15:21 ` Alan Stern 2011-06-11 9:15 ` Hans de Goede 2011-06-11 16:19 ` Theodore Kilgore 2011-06-12 11:43 ` Hans de Goede 2011-06-12 21:20 ` Theodore Kilgore 2011-06-13 2:03 ` Xiaofan Chen 2011-06-13 2:27 ` [Libusb-devel] " Michael Bender 2011-06-11 16:57 ` Alan Stern 2011-06-10 18:16 ` Theodore Kilgore 2011-06-10 18:34 ` Felipe Balbi 2011-06-10 21:18 ` Alan Stern 2011-06-10 21:46 ` Felipe Balbi 2011-06-10 22:46 ` Theodore Kilgore 2011-06-10 22:43 ` Theodore Kilgore 2011-06-11 1:30 ` Xiaofan Chen 2011-06-11 4:17 ` Theodore Kilgore 2011-06-13 9:05 ` Felipe Balbi 2011-06-13 13:06 ` Mauro Carvalho Chehab 2011-06-13 13:12 ` Felipe Balbi 2011-08-04 22:21 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab 2011-08-04 22:56 ` Greg KH [not found] ` <CAA6KcBBZv7bvVxvEWOYL83igpNZHyzh=bcGxh6Dr5aKsvJK5Cg@mail.gmail.com> 2011-08-05 0:33 ` Mauro Carvalho Chehab 2011-08-05 2:56 ` Theodore Kilgore 2011-08-05 6:57 ` Oliver Neukum 2011-08-05 17:38 ` Theodore Kilgore 2011-08-05 7:45 ` Hans de Goede 2011-08-05 7:59 ` USB mini-summit at LinuxCon Vancouveroliver Oliver Neukum 2011-08-05 8:18 ` Hans de Goede 2011-08-05 13:07 ` USB mini-summit at LinuxCon Vancouver Mauro Carvalho Chehab 2011-08-08 17:58 ` Sarah Sharp 2011-08-08 18:23 ` Theodore Kilgore 2011-08-08 18:32 ` Sarah Sharp 2011-08-08 19:37 ` Mauro Carvalho Chehab 2011-08-09 7:52 ` Hans de Goede 2011-08-09 14:19 ` Alan Stern 2011-08-09 15:03 ` Marko Ristola 2011-08-09 19:57 ` Hans de Goede 2011-08-09 20:31 ` Adam Baker 2011-08-09 20:57 ` Hans de Goede 2011-08-10 2:05 ` Xiaofan Chen 2011-08-10 23:04 ` Adam Baker 2011-08-11 8:14 ` Hans de Goede 2011-08-09 23:05 ` Theodore Kilgore 2011-08-10 14:19 ` Alan Stern 2011-08-10 15:03 ` Theodore Kilgore 2011-08-10 16:09 ` Alan Stern 2011-08-10 18:33 ` Theodore Kilgore 2011-08-10 19:39 ` Hans Verkuil 2011-08-10 19:43 ` Greg KH 2011-08-10 20:34 ` Theodore Kilgore 2011-08-10 20:14 ` Mauro Carvalho Chehab 2011-08-10 20:39 ` Theodore Kilgore 2011-08-11 8:57 ` Jean-Francois Moine 2011-08-11 11:19 ` Mauro Carvalho Chehab 2011-08-11 8:14 ` Hans de Goede 2011-08-11 14:56 ` Alan Stern 2011-08-11 15:13 ` Mauro Carvalho Chehab 2011-08-11 15:25 ` Alan Cox 2011-08-11 15:49 ` Alan Stern 2011-08-11 20:01 ` Theodore Kilgore 2011-08-11 20:32 ` Mauro Carvalho Chehab 2011-08-11 23:13 ` Theodore Kilgore 2011-08-12 7:16 ` Hans de Goede 2011-08-12 10:11 ` Alan Cox 2011-08-12 1:07 ` Alan Stern 2011-08-12 2:38 ` Theodore Kilgore 2011-08-11 15:44 ` Alan Stern 2011-08-12 7:26 ` Hans de Goede 2011-08-12 15:36 ` Alan Stern 2011-08-09 17:10 ` Sarah Sharp
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.