Mac OS real USB device support issue

Mac OS real USB device support issue

[Programmingkid]

Hi Gerd,

I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:

- Windows 7
- Mac OS 9.2
- Windows 2000

I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?

Re: Mac OS real USB device support issue

[Howard Spoelstra]

On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>
> Hi Gerd,
>
> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>
> - Windows 7
> - Mac OS 9.2
> - Windows 2000
>
> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?

Hi John,

As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
not support USB 2.0. I was successful only in passing through a USB
flash drive that was forced into USB 1.1 mode by connecting it to a
real USB 1.1 hub and unloading the kext it used.

Best,
Howard

Re: Mac OS real USB device support issue

[BALATON Zoltan]

On Tue, 6 Apr 2021, Howard Spoelstra wrote:
> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>
>> Hi Gerd,
>>
>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>
>> - Windows 7
>> - Mac OS 9.2
>> - Windows 2000
>>
>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>
> Hi John,
>
> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
> not support USB 2.0. I was successful only in passing through a USB
> flash drive that was forced into USB 1.1 mode by connecting it to a
> real USB 1.1 hub and unloading the kext it used.

I think USB passthorugh works in VirtualBox on macOS so maybe you could 
check how that works there and what might need to be done for it. Although 
I'm not sure if it's in the open source part or in the non-free extensions 
so may not be that easy to find out what's needed if the sources for it 
are not available.

Regards.
BALATON Zoltan

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> 
> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>> 
>> Hi Gerd,
>> 
>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>> 
>> - Windows 7
>> - Mac OS 9.2
>> - Windows 2000
>> 
>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
> 
> Hi John,
> 
> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
> not support USB 2.0. I was successful only in passing through a USB
> flash drive that was forced into USB 1.1 mode by connecting it to a
> real USB 1.1 hub and unloading the kext it used.
> 
> Best,
> Howard

Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it. 

Thanks.

Currently there doesn't appear to be a single page with info on how to use USB devices in QEMU with a Mac OS host. Hopefully when I have collected enough information I can make one.

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 6, 2021, at 10:20 AM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> 
> On Tue, 6 Apr 2021, Howard Spoelstra wrote:
>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>> 
>>> Hi Gerd,
>>> 
>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>> 
>>> - Windows 7
>>> - Mac OS 9.2
>>> - Windows 2000
>>> 
>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>> 
>> Hi John,
>> 
>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>> not support USB 2.0. I was successful only in passing through a USB
>> flash drive that was forced into USB 1.1 mode by connecting it to a
>> real USB 1.1 hub and unloading the kext it used.
> 
> I think USB passthorugh works in VirtualBox on macOS so maybe you could check how that works there and what might need to be done for it. Although I'm not sure if it's in the open source part or in the non-free extensions so may not be that easy to find out what's needed if the sources for it are not available.
> 
> Regards.
> BALATON Zoltan

Excellent idea.

Re: Mac OS real USB device support issue

[BALATON Zoltan]

On Tue, 6 Apr 2021, Programmingkid wrote:
>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>>
>>> Hi Gerd,
>>>
>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>>
>>> - Windows 7
>>> - Mac OS 9.2
>>> - Windows 2000
>>>
>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>>
>> Hi John,
>>
>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>> not support USB 2.0. I was successful only in passing through a USB
>> flash drive that was forced into USB 1.1 mode by connecting it to a
>> real USB 1.1 hub and unloading the kext it used.
>>
>> Best,
>> Howard
>
> Hi Howard, I was actually thinking about CC'ing you for this email. Glad 
> you found it. Unloading kext files does not sound pleasant. Maybe there 
> is some better way of doing it.

In any case, until you make sure nothing tries to drive the device on the 
host, passing it to a guest likely will fail because then two drivers from 
two OSes would try to access it simultaneously which likely creates a mess 
as the device and drivers don't expect this. So you can't just pass a 
device through that the host has recognised and is driving without somehow 
getting the host to leave it alone first before you can pass it through. 
Unloading the driver is one way to do that (although it probably breaks 
all other similar devices too). Maybe there's another way to unbind a 
device from the host such as ejecting it first but then I'm not sure if 
the low level USB needed for accessing the device still works after that 
or it's completely forgotten. There's probably a doc somewhere that 
describes how it works and how can you plug a device without also getting 
higher level drivers to load or if there's no official ways for that then 
you'll need to do some configuration on the host to avoid it grabbing 
devices that you want to pass through. On Linux you can add an udev rule 
to ignore the device (maybe also adding TAG+="uaccess" to allow console 
users to use it without needing root access) but not sure how USB works on 
macOS.

Regards,
BALATON Zoltan

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> 
> On Tue, 6 Apr 2021, Programmingkid wrote:
>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>>> 
>>>> Hi Gerd,
>>>> 
>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>>> 
>>>> - Windows 7
>>>> - Mac OS 9.2
>>>> - Windows 2000
>>>> 
>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>>> 
>>> Hi John,
>>> 
>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>>> not support USB 2.0. I was successful only in passing through a USB
>>> flash drive that was forced into USB 1.1 mode by connecting it to a
>>> real USB 1.1 hub and unloading the kext it used.
>>> 
>>> Best,
>>> Howard
>> 
>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
> 
> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host to avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
> 
> Regards,
> BALATON Zoltan

Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.

I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device. 
So this:

USB device <-> Host <-> QEMU USB middleman <-> Guest

This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory). 

Re: Mac OS real USB device support issue

[BALATON Zoltan]

[-- Attachment #1: Type: text/plain, Size: 6142 bytes --]

On Tue, 6 Apr 2021, Programmingkid wrote:
>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>>>>
>>>>> Hi Gerd,
>>>>>
>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>>>>
>>>>> - Windows 7
>>>>> - Mac OS 9.2
>>>>> - Windows 2000
>>>>>
>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>>>>
>>>> Hi John,
>>>>
>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>>>> not support USB 2.0. I was successful only in passing through a USB
>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
>>>> real USB 1.1 hub and unloading the kext it used.
>>>>
>>>> Best,
>>>> Howard
>>>
>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
>>
>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
 o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
>>
>> Regards,
>> BALATON Zoltan
>
> Being able to dissociate a real USB device from its Mac OS driver would 
> be very useful in this situation. IOKit might be one place to look for 
> such a feature. The Mach kernel documentation is another place that 
> might have what we want.

Those might be a good place to start. IOKit provides the drivers and also 
the io registry which is probably where you can get if a driver is bound 
to a device and which one is it. How to dissociate the driver from the 
device though I don't know.

> I have one theory. What if we introduce a middleman. A pseudo-USB device 
> that the guest operating system could apply its configuration data to 
> and will also talk directly with to the real USB device.
> So this:
>
> USB device <-> Host <-> QEMU USB middleman <-> Guest

Isn't this middleman the QEMU usb-host device that we already have?

> This could make USB 2.0 and 3.0 flash drives compatible with an older 
> operating system like Mac OS 9. The USB middleman could fully accept Mac 
> OS 9's configuration and make it think it is talking to a USB 1.1 
> device. Parameters like data packet payload size would no longer be a 
> problem. Host driver unloading would no longer be needed (in theory).

However I think you're mixing up a few things here. The idea of passing 
through USB devices is to let the guest handle it with its own drivers 
like it was connected directly to the virtual machine and not to emulate a 
USB device using host resources. If you want the latter then use 
usb-storage, usb-audio or similar. All that usb-host does is just 
forwarding the packets from guest to the physical device and let it talk 
to it and drive it without help from the host. (I may be wrong about the 
details, I haven't checked actual code but at least conceptually this 
should be the case.) If you have this scenario then you can easily see 
that both the host and guest driving the same USB device will not work. 
You should not try to mount a USB drive in guest that's already mounted by 
the host or you can't send audio from both the guest and the host at the 
same time without totally confusing the device and both drivers that don't 
expect this to ever happen. So for passing through the device you have to 
make sure the host does not try to access it while it's used by the guest.

If your guest does not have drivers for the device that you want to pass 
through that's a different problem. With pass through the guest is 
exclusively given the task of driving the device so it should have a 
driver for it. If the device does not work with the guest if you plug it 
in a physical machine then it won't work with pass through either. But the 
problem here is probably not that but the disagreement between USB speed 
between host and guest. If the guest does not have USB 2.0 then you can't 
pass through USB 2.0 devices unless downgrading them on the host as well 
in some way. So you either connect them to an USB 1 hub to match the 
emulated USB hardware in the guest or you need to emulate an USB 2 card in 
the guest and connect passed through devices to that. Did USB 2 cards 
exist for older G3/G4 Macs? Some PowerBooks had USB 2 ports, what hardware 
did those use? Could those be emulated in QEMU? These are separate 
problems though from getting the device freed from host drivers to avoid 
the problems with both guest and host accessing the device.

Regards,
BALATON Zoltan

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> 
> On Tue, 6 Apr 2021, Programmingkid wrote:
>>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>>>>> 
>>>>>> Hi Gerd,
>>>>>> 
>>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>>>>> 
>>>>>> - Windows 7
>>>>>> - Mac OS 9.2
>>>>>> - Windows 2000
>>>>>> 
>>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>>>>> 
>>>>> Hi John,
>>>>> 
>>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>>>>> not support USB 2.0. I was successful only in passing through a USB
>>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
>>>>> real USB 1.1 hub and unloading the kext it used.
>>>>> 
>>>>> Best,
>>>>> Howard
>>>> 
>>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
>>> 
>>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
> o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
>>> 
>>> Regards,
>>> BALATON Zoltan
>> 
>> Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.
> 
> Those might be a good place to start. IOKit provides the drivers and also the io registry which is probably where you can get if a driver is bound to a device and which one is it. How to dissociate the driver from the device though I don't know.

https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
According to this article a driver has a stop() and detach() method that is called by the IOKit to remove a device. I'm thinking QEMU can be the one that calls these methods for a certain device.

> 
>> I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device.
>> So this:
>> 
>> USB device <-> Host <-> QEMU USB middleman <-> Guest
> 
> Isn't this middleman the QEMU usb-host device that we already have?

It could be. I need to research this issue some more.

> 
>> This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory).
> 
> However I think you're mixing up a few things here. The idea of passing through USB devices is to let the guest handle it with its own drivers like it was connected directly to the virtual machine and not to emulate a USB device using host resources. If you want the latter then use usb-storage, usb-audio or similar. All that usb-host does is just forwarding the packets from guest to the physical device and let it talk to it and drive it without help from the host. (I may be wrong about the details, I haven't checked actual code but at least conceptually this should be the case.) If you have this scenario then you can easily see that both the host and guest driving the same USB device will not work. You should not try to mount a USB drive in guest that's already mounted by the host or you can't send audio from both the guest and the host at the same time without totally confusing the device and both drivers that don't expect this to ever happen. So for passing through the device you have to make sure the host does not try to access it while it's used by the guest.

This clarifies things on my part.

> 
> If your guest does not have drivers for the device that you want to pass through that's a different problem. With pass through the guest is exclusively given the task of driving the device so it should have a driver for it. If the device does not work with the guest if you plug it in a physical machine then it won't work with pass through either. But the problem here is probably not that but the disagreement between USB speed between host and guest. If the guest does not have USB 2.0 then you can't pass through USB 2.0 devices unless downgrading them on the host as well in some way. So you either connect them to an USB 1 hub to match the emulated USB hardware in the guest or you need to emulate an USB 2 card in the guest and connect passed through devices to that. Did USB 2 cards exist for older G3/G4 Macs?

Yes. I upgraded a friend's PowerMac G4 with such a card.

> Some PowerBooks had USB 2 ports, what hardware did those use?

I'm assuming it was the G4 PowerBooks.

> Could those be emulated in QEMU?

The PowerMac already is.  

> These are separate problems though from getting the device freed from host drivers to avoid the problems with both guest and host accessing the device.

Thank you again for the help.

I think a simple algorithm would be 
1) find out if a host driver is already using a real USB device.
2) If it is call that driver's stop() and detach() methods for only that USB device (other devices should not be effected).
3) Let the guest start using the USB device.

Re: Mac OS real USB device support issue

[Howard Spoelstra]

On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> wrote:
>
>
>
> > On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> >
> > On Tue, 6 Apr 2021, Programmingkid wrote:
> >>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> >>> On Tue, 6 Apr 2021, Programmingkid wrote:
> >>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> >>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
> >>>>>>
> >>>>>> Hi Gerd,
> >>>>>>
> >>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
> >>>>>>
> >>>>>> - Windows 7
> >>>>>> - Mac OS 9.2
> >>>>>> - Windows 2000
> >>>>>>
> >>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
> >>>>>
> >>>>> Hi John,
> >>>>>
> >>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
> >>>>> not support USB 2.0. I was successful only in passing through a USB
> >>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
> >>>>> real USB 1.1 hub and unloading the kext it used.
> >>>>>
> >>>>> Best,
> >>>>> Howard
> >>>>
> >>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
> >>>
> >>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
> > o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
> >>>
> >>> Regards,
> >>> BALATON Zoltan
> >>
> >> Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.
> >
> > Those might be a good place to start. IOKit provides the drivers and also the io registry which is probably where you can get if a driver is bound to a device and which one is it. How to dissociate the driver from the device though I don't know.
>
> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
> According to this article a driver has a stop() and detach() method that is called by the IOKit to remove a device. I'm thinking QEMU can be the one that calls these methods for a certain device.
>
> >
> >> I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device.
> >> So this:
> >>
> >> USB device <-> Host <-> QEMU USB middleman <-> Guest
> >
> > Isn't this middleman the QEMU usb-host device that we already have?
>
> It could be. I need to research this issue some more.
>
> >
> >> This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory).
> >
> > However I think you're mixing up a few things here. The idea of passing through USB devices is to let the guest handle it with its own drivers like it was connected directly to the virtual machine and not to emulate a USB device using host resources. If you want the latter then use usb-storage, usb-audio or similar. All that usb-host does is just forwarding the packets from guest to the physical device and let it talk to it and drive it without help from the host. (I may be wrong about the details, I haven't checked actual code but at least conceptually this should be the case.) If you have this scenario then you can easily see that both the host and guest driving the same USB device will not work. You should not try to mount a USB drive in guest that's already mounted by the host or you can't send audio from both the guest and the host at the same time without totally confusing the device and both drivers that don't expect this to ever happen. So for passing through the device you have to make sure the host does not try to access it while it's used by the guest.
>
> This clarifies things on my part.
>
> >
> > If your guest does not have drivers for the device that you want to pass through that's a different problem. With pass through the guest is exclusively given the task of driving the device so it should have a driver for it. If the device does not work with the guest if you plug it in a physical machine then it won't work with pass through either. But the problem here is probably not that but the disagreement between USB speed between host and guest. If the guest does not have USB 2.0 then you can't pass through USB 2.0 devices unless downgrading them on the host as well in some way. So you either connect them to an USB 1 hub to match the emulated USB hardware in the guest or you need to emulate an USB 2 card in the guest and connect passed through devices to that. Did USB 2 cards exist for older G3/G4 Macs?
>
> Yes. I upgraded a friend's PowerMac G4 with such a card.
>
> > Some PowerBooks had USB 2 ports, what hardware did those use?
>
> I'm assuming it was the G4 PowerBooks.
>
> > Could those be emulated in QEMU?
>
> The PowerMac already is.
>
> > These are separate problems though from getting the device freed from host drivers to avoid the problems with both guest and host accessing the device.
>
> Thank you again for the help.
>
> I think a simple algorithm would be
> 1) find out if a host driver is already using a real USB device.
> 2) If it is call that driver's stop() and detach() methods for only that USB device (other devices should not be effected).
> 3) Let the guest start using the USB device.
>
>

This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:

"Problem is when the device is plugged into a usb2 port you can't query
the usb1 descriptors.  So qemu presents the wrong descriptors to the
guest in case host and guest use different usb speeds.  That may or may
not work ...

The other way around is less problematic, when plugging a usb2 device
into a usb3-capable (xhci) port I can tell the guest "this is a usb2
device".  But reporting "this is a usb2 device" via ohci isn't going to
fly for obvious reasons ..."

So then I forced my device it into USB 1.1 mode by attaching it to a
real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
upwards you can use the ehci hub. This works for a simple usb storage
device that uses the usbstorage kext in macOS. Indeed, as Zoltan
wrote, unloading a kext for a device that has its class driver loaded,
one looses access to other devices that match too. Reboot time ;-)

There used to be a solution by creating an empty kext that loaded with
higher priority compared to the standard kexts. I don't know whether
that solution could still work with Apple putting new requirements on
kext loading. I believe unloading a kext requires disabling SIP. Sudo
won't do anymore from Catalina upwards.

I gather Qemu is currenty relying on libusb to do the heavy lifting.
For Linux that seems OK together with perhaps first unbinding a
driver, but for macOS and Windows hosts the unbinding does not work.

Here is some discussion on how to create an empty kext:
https://github.com/libusb/libusb/issues/158#issuecomment-190582178

Best,
Howard

Re: Mac OS real USB device support issue

[Howard Spoelstra]

On Wed, Apr 7, 2021 at 7:26 AM Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>
> On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> wrote:
> >
> >
> >
> > > On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> > >
> > > On Tue, 6 Apr 2021, Programmingkid wrote:
> > >>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> > >>> On Tue, 6 Apr 2021, Programmingkid wrote:
> > >>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> > >>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
> > >>>>>>
> > >>>>>> Hi Gerd,
> > >>>>>>
> > >>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
> > >>>>>>
> > >>>>>> - Windows 7
> > >>>>>> - Mac OS 9.2
> > >>>>>> - Windows 2000
> > >>>>>>
> > >>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
> > >>>>>
> > >>>>> Hi John,
> > >>>>>
> > >>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
> > >>>>> not support USB 2.0. I was successful only in passing through a USB
> > >>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
> > >>>>> real USB 1.1 hub and unloading the kext it used.
> > >>>>>
> > >>>>> Best,
> > >>>>> Howard
> > >>>>
> > >>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
> > >>>
> > >>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
> > > o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
> > >>>
> > >>> Regards,
> > >>> BALATON Zoltan
> > >>
> > >> Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.
> > >
> > > Those might be a good place to start. IOKit provides the drivers and also the io registry which is probably where you can get if a driver is bound to a device and which one is it. How to dissociate the driver from the device though I don't know.
> >
> > https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
> > According to this article a driver has a stop() and detach() method that is called by the IOKit to remove a device. I'm thinking QEMU can be the one that calls these methods for a certain device.
> >
> > >
> > >> I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device.
> > >> So this:
> > >>
> > >> USB device <-> Host <-> QEMU USB middleman <-> Guest
> > >
> > > Isn't this middleman the QEMU usb-host device that we already have?
> >
> > It could be. I need to research this issue some more.
> >
> > >
> > >> This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory).
> > >
> > > However I think you're mixing up a few things here. The idea of passing through USB devices is to let the guest handle it with its own drivers like it was connected directly to the virtual machine and not to emulate a USB device using host resources. If you want the latter then use usb-storage, usb-audio or similar. All that usb-host does is just forwarding the packets from guest to the physical device and let it talk to it and drive it without help from the host. (I may be wrong about the details, I haven't checked actual code but at least conceptually this should be the case.) If you have this scenario then you can easily see that both the host and guest driving the same USB device will not work. You should not try to mount a USB drive in guest that's already mounted by the host or you can't send audio from both the guest and the host at the same time without totally confusing the device and both drivers that don't expect this to ever happen. So for passing through the device you have to make sure the host does not try to access it while it's used by the guest.
> >
> > This clarifies things on my part.
> >
> > >
> > > If your guest does not have drivers for the device that you want to pass through that's a different problem. With pass through the guest is exclusively given the task of driving the device so it should have a driver for it. If the device does not work with the guest if you plug it in a physical machine then it won't work with pass through either. But the problem here is probably not that but the disagreement between USB speed between host and guest. If the guest does not have USB 2.0 then you can't pass through USB 2.0 devices unless downgrading them on the host as well in some way. So you either connect them to an USB 1 hub to match the emulated USB hardware in the guest or you need to emulate an USB 2 card in the guest and connect passed through devices to that. Did USB 2 cards exist for older G3/G4 Macs?
> >
> > Yes. I upgraded a friend's PowerMac G4 with such a card.
> >
> > > Some PowerBooks had USB 2 ports, what hardware did those use?
> >
> > I'm assuming it was the G4 PowerBooks.
> >
> > > Could those be emulated in QEMU?
> >
> > The PowerMac already is.
> >
> > > These are separate problems though from getting the device freed from host drivers to avoid the problems with both guest and host accessing the device.
> >
> > Thank you again for the help.
> >
> > I think a simple algorithm would be
> > 1) find out if a host driver is already using a real USB device.
> > 2) If it is call that driver's stop() and detach() methods for only that USB device (other devices should not be effected).
> > 3) Let the guest start using the USB device.
> >
> >
>
> This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:
>
> "Problem is when the device is plugged into a usb2 port you can't query
> the usb1 descriptors.  So qemu presents the wrong descriptors to the
> guest in case host and guest use different usb speeds.  That may or may
> not work ...
>
> The other way around is less problematic, when plugging a usb2 device
> into a usb3-capable (xhci) port I can tell the guest "this is a usb2
> device".  But reporting "this is a usb2 device" via ohci isn't going to
> fly for obvious reasons ..."
>
> So then I forced my device it into USB 1.1 mode by attaching it to a
> real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
> X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
> upwards you can use the ehci hub. This works for a simple usb storage
> device that uses the usbstorage kext in macOS. Indeed, as Zoltan
> wrote, unloading a kext for a device that has its class driver loaded,
> one looses access to other devices that match too. Reboot time ;-)
>
> There used to be a solution by creating an empty kext that loaded with
> higher priority compared to the standard kexts. I don't know whether
> that solution could still work with Apple putting new requirements on
> kext loading. I believe unloading a kext requires disabling SIP. Sudo
> won't do anymore from Catalina upwards.
>
> I gather Qemu is currenty relying on libusb to do the heavy lifting.
> For Linux that seems OK together with perhaps first unbinding a
> driver, but for macOS and Windows hosts the unbinding does not work.
>
> Here is some discussion on how to create an empty kext:
> https://github.com/libusb/libusb/issues/158#issuecomment-190582178
>
> Best,
> Howard

And this:
https://github.com/libusb/libusb/issues/906

Re: Mac OS real USB device support issue

[BALATON Zoltan]

On Wed, 7 Apr 2021, Howard Spoelstra wrote:
> On Wed, Apr 7, 2021 at 7:26 AM Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>> On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> wrote:
>>>> On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> 
>>>> wrote: Those might be a good place to start. IOKit provides the 
>>>> drivers and also the io registry which is probably where you can get 
>>>> if a driver is bound to a device and which one is it. How to 
>>>> dissociate the driver from the device though I don't know.
>>>
>>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
>>> According to this article a driver has a stop() and detach() method 
>>> that is called by the IOKit to remove a device. I'm thinking QEMU can 
>>> be the one that calls these methods for a certain device.

I think drivers are managed by IOKit so QEMU should and probably could not 
call driver methods directly. There should be some upper management layer 
to control this that you should talk to instead.

>>>> guest and connect passed through devices to that. Did USB 2 cards 
>>>> exist for older G3/G4 Macs?
>>>
>>> Yes. I upgraded a friend's PowerMac G4 with such a card.
>>>
>>>> Some PowerBooks had USB 2 ports, what hardware did those use?
>>>
>>> I'm assuming it was the G4 PowerBooks.
>>>
>>>> Could those be emulated in QEMU?
>>>
>>> The PowerMac already is.

The PowerMacs don't have USB2, only USB1 I think and as Howard wrote MacOS 
does not have drivers either so you'd need to emulate the USB2 card and 
probably also need its driver from the FCode ROM on it or a new enough Mac 
OS X with USB2 drivers. Or pass through devices as USB1 but then you may 
need to somehow downgrade them to that level on the host for which using a 
USB1 only hub seems to work and as Howard found the hub in older Apple 
keyboards work for that.

>> This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:
>>
>> "Problem is when the device is plugged into a usb2 port you can't query
>> the usb1 descriptors.  So qemu presents the wrong descriptors to the
>> guest in case host and guest use different usb speeds.  That may or may
>> not work ...
>>
>> The other way around is less problematic, when plugging a usb2 device
>> into a usb3-capable (xhci) port I can tell the guest "this is a usb2
>> device".  But reporting "this is a usb2 device" via ohci isn't going to
>> fly for obvious reasons ..."
>>
>> So then I forced my device it into USB 1.1 mode by attaching it to a
>> real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
>> X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
>> upwards you can use the ehci hub. This works for a simple usb storage
>> device that uses the usbstorage kext in macOS. Indeed, as Zoltan
>> wrote, unloading a kext for a device that has its class driver loaded,
>> one looses access to other devices that match too. Reboot time ;-)
>>
>> There used to be a solution by creating an empty kext that loaded with
>> higher priority compared to the standard kexts. I don't know whether
>> that solution could still work with Apple putting new requirements on
>> kext loading. I believe unloading a kext requires disabling SIP. Sudo
>> won't do anymore from Catalina upwards.
>>
>> I gather Qemu is currenty relying on libusb to do the heavy lifting.
>> For Linux that seems OK together with perhaps first unbinding a
>> driver, but for macOS and Windows hosts the unbinding does not work.
>>
>> Here is some discussion on how to create an empty kext:
>> https://github.com/libusb/libusb/issues/158#issuecomment-190582178
>>
>> Best,
>> Howard
>
> And this:
> https://github.com/libusb/libusb/issues/906

That sounds like it could be what's needed, I haven't read it in detail 
but somebody could try to make a patch based on this info and test it if 
it still works with latest macOS.

Regards,
BALATON Zoltan

Re: Mac OS real USB device support issue

[Gerd Hoffmann]

  Hi,

> > Those might be a good place to start. IOKit provides the drivers and
> > also the io registry which is probably where you can get if a driver
> > is bound to a device and which one is it. How to dissociate the
> > driver from the device though I don't know.

> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html

> According to this article a driver has a stop() and detach() method
> that is called by the IOKit to remove a device. I'm thinking QEMU can
> be the one that calls these methods for a certain device.

libusb should do that.  Interfaces exist already (see
libusb_detach_kernel_driver & friends) because we have the very same
problem on linux.

take care,
  Gerd

Re: Mac OS real USB device support issue

[Howard Spoelstra]

On Thu, Apr 8, 2021 at 1:05 PM Gerd Hoffmann <gerd@kraxel.org> wrote:
>
>   Hi,
>
> > > Those might be a good place to start. IOKit provides the drivers and
> > > also the io registry which is probably where you can get if a driver
> > > is bound to a device and which one is it. How to dissociate the
> > > driver from the device though I don't know.
>
> > https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
>
> > According to this article a driver has a stop() and detach() method
> > that is called by the IOKit to remove a device. I'm thinking QEMU can
> > be the one that calls these methods for a certain device.
>
> libusb should do that.  Interfaces exist already (see
> libusb_detach_kernel_driver & friends) because we have the very same
> problem on linux.
>
> take care,
>   Gerd
>

As far as I understand the patches here
https://github.com/libusb/libusb/issues/906 they are internal to
libusb, so we would need to build a libusb for use with e.g., brew to
build a macOS executable. Or wait for them to be finalised to get
included in libusb and then included in brew and then ....

Best,
Howard

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 8, 2021, at 7:05 AM, Gerd Hoffmann <gerd@kraxel.org> wrote:
> 
>  Hi,
> 
>>> Those might be a good place to start. IOKit provides the drivers and
>>> also the io registry which is probably where you can get if a driver
>>> is bound to a device and which one is it. How to dissociate the
>>> driver from the device though I don't know.
> 
>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
> 
>> According to this article a driver has a stop() and detach() method
>> that is called by the IOKit to remove a device. I'm thinking QEMU can
>> be the one that calls these methods for a certain device.
> 
> libusb should do that.  Interfaces exist already (see
> libusb_detach_kernel_driver & friends) because we have the very same
> problem on linux.
> 
> take care,
>  Gerd
> 

The questions that come to mind are:
- Does libusb_detach_kernel_driver() work on Mac OS?
- Is libusb_detach_kernel_driver() called on Mac OS in QEMU?

The only mention of this function in QEMU comes from host-libusb.c. 

After some tests I found out the function host-libusb.c:usb_host_detach_kernel() is being called on Mac OS 11.1. It never reaches the libusb_detach_kernel_driver() function. It stops at the continue statement. Here is the full function:

static void usb_host_detach_kernel(USBHostDevice *s)
{
    printf("usb_host_detach_kernel() called\n");
    struct libusb_config_descriptor *conf;
    int rc, i;

    rc = libusb_get_active_config_descriptor(s->dev, &conf);
    if (rc != 0) {
        printf("rc != 0 => %d\n", rc);
        return;
    }
    for (i = 0; i < USB_MAX_INTERFACES; i++) {
        rc = libusb_kernel_driver_active(s->dh, i);
        usb_host_libusb_error("libusb_kernel_driver_active", rc);
        if (rc != 1) {
            if (rc == 0) {
                s->ifs[i].detached = true;
            }
            printf("rc != 1 => %d\n", rc);
            continue;
        }
        trace_usb_host_detach_kernel(s->bus_num, s->addr, i);
        rc = libusb_detach_kernel_driver(s->dh, i);
        printf("libusb_detach_kernel_driver() called\n");
        usb_host_libusb_error("libusb_detach_kernel_driver", rc);
        s->ifs[i].detached = true;
    }
    libusb_free_config_descriptor(conf);
}


Next to the continue statement in the loop is where my printf function says that rc is equal to 0. So it looks like libusb_kernel_driver_active() may have an issue since it sets the rc variable. Later on I will try to figure out what is happening here.

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 8, 2021, at 12:40 PM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> 
> On Thu, Apr 8, 2021 at 1:05 PM Gerd Hoffmann <gerd@kraxel.org> wrote:
>> 
>>  Hi,
>> 
>>>> Those might be a good place to start. IOKit provides the drivers and
>>>> also the io registry which is probably where you can get if a driver
>>>> is bound to a device and which one is it. How to dissociate the
>>>> driver from the device though I don't know.
>> 
>>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
>> 
>>> According to this article a driver has a stop() and detach() method
>>> that is called by the IOKit to remove a device. I'm thinking QEMU can
>>> be the one that calls these methods for a certain device.
>> 
>> libusb should do that.  Interfaces exist already (see
>> libusb_detach_kernel_driver & friends) because we have the very same
>> problem on linux.
>> 
>> take care,
>>  Gerd
>> 
> 
> As far as I understand the patches here
> https://github.com/libusb/libusb/issues/906 they are internal to
> libusb, so we would need to build a libusb for use with e.g., brew to
> build a macOS executable. Or wait for them to be finalised to get
> included in libusb and then included in brew and then ....
> 
> Best,
> Howard

We could also consider our own git submodule incase the libusb people fail to fix their issue.

Re: Mac OS real USB device support issue

[Gerd Hoffmann]

  Hi,

> The questions that come to mind are:
> - Does libusb_detach_kernel_driver() work on Mac OS?
> - Is libusb_detach_kernel_driver() called on Mac OS in QEMU?
> 
> The only mention of this function in QEMU comes from host-libusb.c. 
> 
> After some tests I found out the function
> host-libusb.c:usb_host_detach_kernel() is being called on Mac OS 11.1.
> It never reaches the libusb_detach_kernel_driver() function. It stops
> at the continue statement.

Yep.  Most likely libusb_kernel_driver_active() returns 0 because the
kernel driver management functions are not implemented for macos.

take care,
  Gerd

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 7, 2021, at 1:28 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> 
> On Wed, Apr 7, 2021 at 7:26 AM Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>> 
>> On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> wrote:
>>> 
>>> 
>>> 
>>>> On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>>>> 
>>>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>>>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>>>>>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>>>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>>>>>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Gerd,
>>>>>>>>> 
>>>>>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
>>>>>>>>> 
>>>>>>>>> - Windows 7
>>>>>>>>> - Mac OS 9.2
>>>>>>>>> - Windows 2000
>>>>>>>>> 
>>>>>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
>>>>>>>> 
>>>>>>>> Hi John,
>>>>>>>> 
>>>>>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>>>>>>>> not support USB 2.0. I was successful only in passing through a USB
>>>>>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
>>>>>>>> real USB 1.1 hub and unloading the kext it used.
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Howard
>>>>>>> 
>>>>>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
>>>>>> 
>>>>>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
>>>> o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
>>>>>> 
>>>>>> Regards,
>>>>>> BALATON Zoltan
>>>>> 
>>>>> Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.
>>>> 
>>>> Those might be a good place to start. IOKit provides the drivers and also the io registry which is probably where you can get if a driver is bound to a device and which one is it. How to dissociate the driver from the device though I don't know.
>>> 
>>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
>>> According to this article a driver has a stop() and detach() method that is called by the IOKit to remove a device. I'm thinking QEMU can be the one that calls these methods for a certain device.
>>> 
>>>> 
>>>>> I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device.
>>>>> So this:
>>>>> 
>>>>> USB device <-> Host <-> QEMU USB middleman <-> Guest
>>>> 
>>>> Isn't this middleman the QEMU usb-host device that we already have?
>>> 
>>> It could be. I need to research this issue some more.
>>> 
>>>> 
>>>>> This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory).
>>>> 
>>>> However I think you're mixing up a few things here. The idea of passing through USB devices is to let the guest handle it with its own drivers like it was connected directly to the virtual machine and not to emulate a USB device using host resources. If you want the latter then use usb-storage, usb-audio or similar. All that usb-host does is just forwarding the packets from guest to the physical device and let it talk to it and drive it without help from the host. (I may be wrong about the details, I haven't checked actual code but at least conceptually this should be the case.) If you have this scenario then you can easily see that both the host and guest driving the same USB device will not work. You should not try to mount a USB drive in guest that's already mounted by the host or you can't send audio from both the guest and the host at the same time without totally confusing the device and both drivers that don't expect this to ever happen. So for passing through the device you have to make sure the host does not try to access it while it's used by the guest.
>>> 
>>> This clarifies things on my part.
>>> 
>>>> 
>>>> If your guest does not have drivers for the device that you want to pass through that's a different problem. With pass through the guest is exclusively given the task of driving the device so it should have a driver for it. If the device does not work with the guest if you plug it in a physical machine then it won't work with pass through either. But the problem here is probably not that but the disagreement between USB speed between host and guest. If the guest does not have USB 2.0 then you can't pass through USB 2.0 devices unless downgrading them on the host as well in some way. So you either connect them to an USB 1 hub to match the emulated USB hardware in the guest or you need to emulate an USB 2 card in the guest and connect passed through devices to that. Did USB 2 cards exist for older G3/G4 Macs?
>>> 
>>> Yes. I upgraded a friend's PowerMac G4 with such a card.
>>> 
>>>> Some PowerBooks had USB 2 ports, what hardware did those use?
>>> 
>>> I'm assuming it was the G4 PowerBooks.
>>> 
>>>> Could those be emulated in QEMU?
>>> 
>>> The PowerMac already is.
>>> 
>>>> These are separate problems though from getting the device freed from host drivers to avoid the problems with both guest and host accessing the device.
>>> 
>>> Thank you again for the help.
>>> 
>>> I think a simple algorithm would be
>>> 1) find out if a host driver is already using a real USB device.
>>> 2) If it is call that driver's stop() and detach() methods for only that USB device (other devices should not be effected).
>>> 3) Let the guest start using the USB device.
>>> 
>>> 
>> 
>> This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:
>> 
>> "Problem is when the device is plugged into a usb2 port you can't query
>> the usb1 descriptors.  So qemu presents the wrong descriptors to the
>> guest in case host and guest use different usb speeds.  That may or may
>> not work ...
>> 
>> The other way around is less problematic, when plugging a usb2 device
>> into a usb3-capable (xhci) port I can tell the guest "this is a usb2
>> device".  But reporting "this is a usb2 device" via ohci isn't going to
>> fly for obvious reasons ..."
>> 
>> So then I forced my device it into USB 1.1 mode by attaching it to a
>> real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
>> X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
>> upwards you can use the ehci hub. This works for a simple usb storage
>> device that uses the usbstorage kext in macOS. Indeed, as Zoltan
>> wrote, unloading a kext for a device that has its class driver loaded,
>> one looses access to other devices that match too. Reboot time ;-)
>> 
>> There used to be a solution by creating an empty kext that loaded with
>> higher priority compared to the standard kexts. I don't know whether
>> that solution could still work with Apple putting new requirements on
>> kext loading. I believe unloading a kext requires disabling SIP. Sudo
>> won't do anymore from Catalina upwards.
>> 
>> I gather Qemu is currenty relying on libusb to do the heavy lifting.
>> For Linux that seems OK together with perhaps first unbinding a
>> driver, but for macOS and Windows hosts the unbinding does not work.
>> 
>> Here is some discussion on how to create an empty kext:
>> https://github.com/libusb/libusb/issues/158#issuecomment-190582178
>> 
>> Best,
>> Howard
> 
> And this:
> https://github.com/libusb/libusb/issues/906

Have you tried the proposed changes yet for libusb? 

Re: Mac OS real USB device support issue

[Howard Spoelstra]

On Fri, Apr 9, 2021 at 9:37 PM Programmingkid <programmingkidx@gmail.com> wrote:
>
>
>
> > On Apr 7, 2021, at 1:28 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> >
> > On Wed, Apr 7, 2021 at 7:26 AM Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> >>
> >> On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>>> On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> >>>>
> >>>> On Tue, 6 Apr 2021, Programmingkid wrote:
> >>>>>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
> >>>>>> On Tue, 6 Apr 2021, Programmingkid wrote:
> >>>>>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> >>>>>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid <programmingkidx@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Gerd,
> >>>>>>>>>
> >>>>>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine to test USB support. I am on Mac OS 11.1 and cannot make USB devices work with any of my guests. So far these are the guests I have tested with:
> >>>>>>>>>
> >>>>>>>>> - Windows 7
> >>>>>>>>> - Mac OS 9.2
> >>>>>>>>> - Windows 2000
> >>>>>>>>>
> >>>>>>>>> I have tried using USB flash drives, USB sound cards, and an USB headset. They all show up under 'info usb', but cannot be used in the guest. My setup does use a USB-C hub so I'm not sure if this is a bug with QEMU or an issue with the hub. Would you have any information on this issue?
> >>>>>>>>
> >>>>>>>> Hi John,
> >>>>>>>>
> >>>>>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
> >>>>>>>> not support USB 2.0. I was successful only in passing through a USB
> >>>>>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
> >>>>>>>> real USB 1.1 hub and unloading the kext it used.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Howard
> >>>>>>>
> >>>>>>> Hi Howard, I was actually thinking about CC'ing you for this email. Glad you found it. Unloading kext files does not sound pleasant. Maybe there is some better way of doing it.
> >>>>>>
> >>>>>> In any case, until you make sure nothing tries to drive the device on the host, passing it to a guest likely will fail because then two drivers from two OSes would try to access it simultaneously which likely creates a mess as the device and drivers don't expect this. So you can't just pass a device through that the host has recognised and is driving without somehow getting the host to leave it alone first before you can pass it through. Unloading the driver is one way to do that (although it probably breaks all other similar devices too). Maybe there's another way to unbind a device from the host such as ejecting it first but then I'm not sure if the low level USB needed for accessing the device still works after that or it's completely forgotten. There's probably a doc somewhere that describes how it works and how can you plug a device without also getting higher level drivers to load or if there's no official ways for that then you'll need to do some configuration on the host t
> >>>> o avoid it grabbing devices that you want to pass through. On Linux you can add an udev rule to ignore the device (maybe also adding TAG+="uaccess" to allow console users to use it without needing root access) but not sure how USB works on macOS.
> >>>>>>
> >>>>>> Regards,
> >>>>>> BALATON Zoltan
> >>>>>
> >>>>> Being able to dissociate a real USB device from its Mac OS driver would be very useful in this situation. IOKit might be one place to look for such a feature. The Mach kernel documentation is another place that might have what we want.
> >>>>
> >>>> Those might be a good place to start. IOKit provides the drivers and also the io registry which is probably where you can get if a driver is bound to a device and which one is it. How to dissociate the driver from the device though I don't know.
> >>>
> >>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
> >>> According to this article a driver has a stop() and detach() method that is called by the IOKit to remove a device. I'm thinking QEMU can be the one that calls these methods for a certain device.
> >>>
> >>>>
> >>>>> I have one theory. What if we introduce a middleman. A pseudo-USB device that the guest operating system could apply its configuration data to and will also talk directly with to the real USB device.
> >>>>> So this:
> >>>>>
> >>>>> USB device <-> Host <-> QEMU USB middleman <-> Guest
> >>>>
> >>>> Isn't this middleman the QEMU usb-host device that we already have?
> >>>
> >>> It could be. I need to research this issue some more.
> >>>
> >>>>
> >>>>> This could make USB 2.0 and 3.0 flash drives compatible with an older operating system like Mac OS 9. The USB middleman could fully accept Mac OS 9's configuration and make it think it is talking to a USB 1.1 device. Parameters like data packet payload size would no longer be a problem. Host driver unloading would no longer be needed (in theory).
> >>>>
> >>>> However I think you're mixing up a few things here. The idea of passing through USB devices is to let the guest handle it with its own drivers like it was connected directly to the virtual machine and not to emulate a USB device using host resources. If you want the latter then use usb-storage, usb-audio or similar. All that usb-host does is just forwarding the packets from guest to the physical device and let it talk to it and drive it without help from the host. (I may be wrong about the details, I haven't checked actual code but at least conceptually this should be the case.) If you have this scenario then you can easily see that both the host and guest driving the same USB device will not work. You should not try to mount a USB drive in guest that's already mounted by the host or you can't send audio from both the guest and the host at the same time without totally confusing the device and both drivers that don't expect this to ever happen. So for passing through the device you have to make sure the host does not try to access it while it's used by the guest.
> >>>
> >>> This clarifies things on my part.
> >>>
> >>>>
> >>>> If your guest does not have drivers for the device that you want to pass through that's a different problem. With pass through the guest is exclusively given the task of driving the device so it should have a driver for it. If the device does not work with the guest if you plug it in a physical machine then it won't work with pass through either. But the problem here is probably not that but the disagreement between USB speed between host and guest. If the guest does not have USB 2.0 then you can't pass through USB 2.0 devices unless downgrading them on the host as well in some way. So you either connect them to an USB 1 hub to match the emulated USB hardware in the guest or you need to emulate an USB 2 card in the guest and connect passed through devices to that. Did USB 2 cards exist for older G3/G4 Macs?
> >>>
> >>> Yes. I upgraded a friend's PowerMac G4 with such a card.
> >>>
> >>>> Some PowerBooks had USB 2 ports, what hardware did those use?
> >>>
> >>> I'm assuming it was the G4 PowerBooks.
> >>>
> >>>> Could those be emulated in QEMU?
> >>>
> >>> The PowerMac already is.
> >>>
> >>>> These are separate problems though from getting the device freed from host drivers to avoid the problems with both guest and host accessing the device.
> >>>
> >>> Thank you again for the help.
> >>>
> >>> I think a simple algorithm would be
> >>> 1) find out if a host driver is already using a real USB device.
> >>> 2) If it is call that driver's stop() and detach() methods for only that USB device (other devices should not be effected).
> >>> 3) Let the guest start using the USB device.
> >>>
> >>>
> >>
> >> This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:
> >>
> >> "Problem is when the device is plugged into a usb2 port you can't query
> >> the usb1 descriptors.  So qemu presents the wrong descriptors to the
> >> guest in case host and guest use different usb speeds.  That may or may
> >> not work ...
> >>
> >> The other way around is less problematic, when plugging a usb2 device
> >> into a usb3-capable (xhci) port I can tell the guest "this is a usb2
> >> device".  But reporting "this is a usb2 device" via ohci isn't going to
> >> fly for obvious reasons ..."
> >>
> >> So then I forced my device it into USB 1.1 mode by attaching it to a
> >> real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
> >> X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
> >> upwards you can use the ehci hub. This works for a simple usb storage
> >> device that uses the usbstorage kext in macOS. Indeed, as Zoltan
> >> wrote, unloading a kext for a device that has its class driver loaded,
> >> one looses access to other devices that match too. Reboot time ;-)
> >>
> >> There used to be a solution by creating an empty kext that loaded with
> >> higher priority compared to the standard kexts. I don't know whether
> >> that solution could still work with Apple putting new requirements on
> >> kext loading. I believe unloading a kext requires disabling SIP. Sudo
> >> won't do anymore from Catalina upwards.
> >>
> >> I gather Qemu is currenty relying on libusb to do the heavy lifting.
> >> For Linux that seems OK together with perhaps first unbinding a
> >> driver, but for macOS and Windows hosts the unbinding does not work.
> >>
> >> Here is some discussion on how to create an empty kext:
> >> https://github.com/libusb/libusb/issues/158#issuecomment-190582178
> >>
> >> Best,
> >> Howard
> >
> > And this:
> > https://github.com/libusb/libusb/issues/906
>
> Have you tried the proposed changes yet for libusb?
>

Hi,

Yes, I experimented with the current libusb from brew, the latest
libusb code from github and a patched version based on that. I
couldn't get a flash drive passed through with any of them. Running as
root made no difference. My Mojave host doesn't allow unloading the
kext loaded for the flash drive where Sierra allowed that, but then
that should be handled by the patches.

I'll link to the latest libusb and the patched version plus the
patches. I guess it will not work on your host, but you might be able
to persuade qemu to use them by using
install_name_tool -change /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
@executable_path/libusb-1.0.0-latest.dylib qemu-system-ppc

I'll also include the patches, libusb is easily built.
https://surfdrive.surf.nl/files/index.php/s/Qs0rtTVe2qIudw4/download

Best,
Howard

Re: Mac OS real USB device support issue

[BALATON Zoltan]

On Sat, 10 Apr 2021, Howard Spoelstra wrote:
> On Fri, Apr 9, 2021 at 9:37 PM Programmingkid <programmingkidx@gmail.com> wrote:
>> Have you tried the proposed changes yet for libusb?
>
> Hi,
>
> Yes, I experimented with the current libusb from brew, the latest
> libusb code from github and a patched version based on that. I
> couldn't get a flash drive passed through with any of them. Running as
> root made no difference. My Mojave host doesn't allow unloading the
> kext loaded for the flash drive where Sierra allowed that, but then
> that should be handled by the patches.
>
> I'll link to the latest libusb and the patched version plus the
> patches. I guess it will not work on your host, but you might be able
> to persuade qemu to use them by using
> install_name_tool -change /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
> @executable_path/libusb-1.0.0-latest.dylib qemu-system-ppc
>
> I'll also include the patches, libusb is easily built.
> https://surfdrive.surf.nl/files/index.php/s/Qs0rtTVe2qIudw4/download

I think you (John and Gerd) found that detecting if a kernel driver is 
attached does not seem to work so it does not even get to unloading what 
these patches are about I think. So you first need to debug and fix 
libusb_kernel_driver_active() so the unloading function is called at all. 
I don't know how that's done on macOS but maybe querying the IO registry 
somehow that should have all info about devices and IOKit drivers.

Regards,
BALATON Zoltan

Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 9, 2021, at 4:08 AM, Gerd Hoffmann <gerd@kraxel.org> wrote:
> 
>  Hi,
> 
>> The questions that come to mind are:
>> - Does libusb_detach_kernel_driver() work on Mac OS?
>> - Is libusb_detach_kernel_driver() called on Mac OS in QEMU?
>> 
>> The only mention of this function in QEMU comes from host-libusb.c. 
>> 
>> After some tests I found out the function
>> host-libusb.c:usb_host_detach_kernel() is being called on Mac OS 11.1.
>> It never reaches the libusb_detach_kernel_driver() function. It stops
>> at the continue statement.
> 
> Yep.  Most likely libusb_kernel_driver_active() returns 0 because the
> kernel driver management functions are not implemented for macos.
> 
> take care,
>  Gerd
> 

I was wondering about the usb_host_detach_kernel() code. It iterates thru interface 0 to 15. Why does it stop at 15? Isn't possible to have up to 255 interfaces?

Re: Mac OS real USB device support issue

[Programmingkid]

I just wanted to give you guys an update on this issue. The libusb people have fixed the issue with USB device driver detaching on Mac OS. Here is where the code for the fix is located: https://github.com/libusb/libusb/pull/911/commits.

I was able to build the library and connect it to QEMU on Mac OS 11.1. As I write this email I am listening to mp3 files on my Mac OS 10.4 VM that is using a real USB sound card. It isn't perfect sounding but definitely a leap forward from where we were a month ago. 

To download each commit in patch form, just append ".patch" to the patch's URL. 
Apply the patches one by one using "patch -p1 < patch.txt".
Then open the Xcode project file and build the libusb target.
To locate the file I had to use this command in the terminal: cd  ~/Library/Developer/Xcode/DerivedData
Then find out the path of the old libusb being used in QEMU: otool -L <QEMU's path> | grep "libusb" 
Then to have QEMU use the new library file, use this command:
install_name_tool -change <old path> <new libusb's path> <QEMU's path>

Please let me know if you need any help with building your own version of libusb.