* 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series @ 2021-11-03 9:02 Hans de Goede 2021-11-03 9:07 ` Greg Kroah-Hartman 0 siblings, 1 reply; 9+ messages in thread From: Hans de Goede @ 2021-11-03 9:02 UTC (permalink / raw) To: Greg Kroah-Hartman, stable, Kishon Vijay Abraham I Hi Greg, We (Fedora) have been receiving multiple reports about USB devices stopping working starting with 5.14.14 . An Arch Linux user has found that reverting the first 2 patches from this series: https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). See here for the Arch-linux discussion surrounding this: https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 And here are 2 Fedora bug reports of Fedora users being unable to use their machines due their mouse + kbd not working: https://bugzilla.redhat.com/show_bug.cgi?id=2019542 https://bugzilla.redhat.com/show_bug.cgi?id=2019576 Can we get this patch-series reverted from the 5.14.y releases please ? Regards, Hans ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 9:02 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series Hans de Goede @ 2021-11-03 9:07 ` Greg Kroah-Hartman 2021-11-03 12:48 ` Hans de Goede 2021-11-05 7:57 ` Greg Kroah-Hartman 0 siblings, 2 replies; 9+ messages in thread From: Greg Kroah-Hartman @ 2021-11-03 9:07 UTC (permalink / raw) To: Hans de Goede; +Cc: stable, Kishon Vijay Abraham I On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: > Hi Greg, > > We (Fedora) have been receiving multiple reports about USB devices stopping > working starting with 5.14.14 . > > An Arch Linux user has found that reverting the first 2 patches from this series: > https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ > > Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). > > See here for the Arch-linux discussion surrounding this: > https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 > > And here are 2 Fedora bug reports of Fedora users being unable to use their > machines due their mouse + kbd not working: > > https://bugzilla.redhat.com/show_bug.cgi?id=2019542 > https://bugzilla.redhat.com/show_bug.cgi?id=2019576 > > Can we get this patch-series reverted from the 5.14.y releases please ? Sure, but can you also submit patches to get into 5.15.y and 5.16-rc1 that revert these changes as they should still be an issue there, right? thanks, greg k-h ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 9:07 ` Greg Kroah-Hartman @ 2021-11-03 12:48 ` Hans de Goede 2021-11-03 14:44 ` Kishon Vijay Abraham I 2021-11-03 17:45 ` Greg Kroah-Hartman 2021-11-05 7:57 ` Greg Kroah-Hartman 1 sibling, 2 replies; 9+ messages in thread From: Hans de Goede @ 2021-11-03 12:48 UTC (permalink / raw) To: Greg Kroah-Hartman; +Cc: stable, Kishon Vijay Abraham I Hi, On 11/3/21 10:07, Greg Kroah-Hartman wrote: > On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: >> Hi Greg, >> >> We (Fedora) have been receiving multiple reports about USB devices stopping >> working starting with 5.14.14 . >> >> An Arch Linux user has found that reverting the first 2 patches from this series: >> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ >> >> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). >> >> See here for the Arch-linux discussion surrounding this: >> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 >> >> And here are 2 Fedora bug reports of Fedora users being unable to use their >> machines due their mouse + kbd not working: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 >> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 >> >> Can we get this patch-series reverted from the 5.14.y releases please ? > > Sure, Thanks. > but can you also submit patches to get into 5.15.y and 5.16-rc1 > that revert these changes as they should still be an issue there, right? Yes I assume this is still an issue there too, but I was hoping that Kishon can take a look and maybe actually fix things, since just reverting presumably regresses whatever these patches were addressing. We've aprox 1-3 weeks before distros like Arch and Linux will switch to 5.15.y kernels. So we have some time to come up with a fix there, where as for 5.14.y this is hitting users now. Regards, Hans ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 12:48 ` Hans de Goede @ 2021-11-03 14:44 ` Kishon Vijay Abraham I 2021-11-03 14:59 ` Alan Stern 2021-11-03 17:45 ` Greg Kroah-Hartman 1 sibling, 1 reply; 9+ messages in thread From: Kishon Vijay Abraham I @ 2021-11-03 14:44 UTC (permalink / raw) To: Hans de Goede, Greg Kroah-Hartman, Alan Stern Cc: stable, linux-usb, chris.chiu, Mathias Nyman + Alan, Chris, Mathias, linux-usb Hi Hans, On 03/11/21 6:18 pm, Hans de Goede wrote: > Hi, > > On 11/3/21 10:07, Greg Kroah-Hartman wrote: >> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: >>> Hi Greg, >>> >>> We (Fedora) have been receiving multiple reports about USB devices stopping >>> working starting with 5.14.14 . >>> >>> An Arch Linux user has found that reverting the first 2 patches from this series: >>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ >>> >>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). >>> >>> See here for the Arch-linux discussion surrounding this: >>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 >>> >>> And here are 2 Fedora bug reports of Fedora users being unable to use their >>> machines due their mouse + kbd not working: >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 >>> >>> Can we get this patch-series reverted from the 5.14.y releases please ? >> >> Sure, > > Thanks. > >> but can you also submit patches to get into 5.15.y and 5.16-rc1 >> that revert these changes as they should still be an issue there, right? > > Yes I assume this is still an issue there too, but I was hoping that > Kishon can take a look and maybe actually fix things, since just > reverting presumably regresses whatever these patches were addressing. > > We've aprox 1-3 weeks before distros like Arch and Linux will switch > to 5.15.y kernels. So we have some time to come up with a fix > there, where as for 5.14.y this is hitting users now. Is the issue with PCIe USB devices or platform USB device? Is it specific to super speed devices or high speed device? Thanks, Kishon ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 14:44 ` Kishon Vijay Abraham I @ 2021-11-03 14:59 ` Alan Stern 2021-11-03 23:54 ` Mathias Nyman 0 siblings, 1 reply; 9+ messages in thread From: Alan Stern @ 2021-11-03 14:59 UTC (permalink / raw) To: Kishon Vijay Abraham I Cc: Hans de Goede, Greg Kroah-Hartman, stable, linux-usb, chris.chiu, Mathias Nyman On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote: > + Alan, Chris, Mathias, linux-usb > > Hi Hans, > > On 03/11/21 6:18 pm, Hans de Goede wrote: > > Hi, > > > > On 11/3/21 10:07, Greg Kroah-Hartman wrote: > >> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: > >>> Hi Greg, > >>> > >>> We (Fedora) have been receiving multiple reports about USB devices stopping > >>> working starting with 5.14.14 . > >>> > >>> An Arch Linux user has found that reverting the first 2 patches from this series: > >>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ > >>> > >>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). > >>> > >>> See here for the Arch-linux discussion surrounding this: > >>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 > >>> > >>> And here are 2 Fedora bug reports of Fedora users being unable to use their > >>> machines due their mouse + kbd not working: > >>> > >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 > >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 > >>> > >>> Can we get this patch-series reverted from the 5.14.y releases please ? > >> > >> Sure, > > > > Thanks. > > > >> but can you also submit patches to get into 5.15.y and 5.16-rc1 > >> that revert these changes as they should still be an issue there, right? > > > > Yes I assume this is still an issue there too, but I was hoping that > > Kishon can take a look and maybe actually fix things, since just > > reverting presumably regresses whatever these patches were addressing. > > > > We've aprox 1-3 weeks before distros like Arch and Linux will switch > > to 5.15.y kernels. So we have some time to come up with a fix > > there, where as for 5.14.y this is hitting users now. > > Is the issue with PCIe USB devices or platform USB device? Is it specific to > super speed devices or high speed device? Look at the bug reports. They are on standard PCs (so PCIe controllers) and some of them involve full speed (mouse and keyboard) devices. Although it looks like the problem has little to do with the device and a lot to do with the controller. Is there a good way to get more information about what is going wrong? For example, by enabling tracepoints in the xhci-hcd driver? Alan Stern ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 14:59 ` Alan Stern @ 2021-11-03 23:54 ` Mathias Nyman 2021-11-15 22:05 ` Mathias Nyman 0 siblings, 1 reply; 9+ messages in thread From: Mathias Nyman @ 2021-11-03 23:54 UTC (permalink / raw) To: Alan Stern, Kishon Vijay Abraham I Cc: Hans de Goede, Greg Kroah-Hartman, stable, linux-usb, chris.chiu, Mathias Nyman On 3.11.2021 16.59, Alan Stern wrote: > On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote: >> + Alan, Chris, Mathias, linux-usb >> >> Hi Hans, >> >> On 03/11/21 6:18 pm, Hans de Goede wrote: >>> Hi, >>> >>> On 11/3/21 10:07, Greg Kroah-Hartman wrote: >>>> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: >>>>> Hi Greg, >>>>> >>>>> We (Fedora) have been receiving multiple reports about USB devices stopping >>>>> working starting with 5.14.14 . >>>>> >>>>> An Arch Linux user has found that reverting the first 2 patches from this series: >>>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ >>>>> >>>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). >>>>> >>>>> See here for the Arch-linux discussion surrounding this: >>>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 >>>>> >>>>> And here are 2 Fedora bug reports of Fedora users being unable to use their >>>>> machines due their mouse + kbd not working: >>>>> >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 >>>>> >>>>> Can we get this patch-series reverted from the 5.14.y releases please ? >>>> >>>> Sure, >>> >>> Thanks. >>> >>>> but can you also submit patches to get into 5.15.y and 5.16-rc1 >>>> that revert these changes as they should still be an issue there, right? >>> >>> Yes I assume this is still an issue there too, but I was hoping that >>> Kishon can take a look and maybe actually fix things, since just >>> reverting presumably regresses whatever these patches were addressing. >>> >>> We've aprox 1-3 weeks before distros like Arch and Linux will switch >>> to 5.15.y kernels. So we have some time to come up with a fix >>> there, where as for 5.14.y this is hitting users now. >> >> Is the issue with PCIe USB devices or platform USB device? Is it specific to >> super speed devices or high speed device? > > Look at the bug reports. They are on standard PCs (so PCIe controllers) > and some of them involve full speed (mouse and keyboard) devices. > Although it looks like the problem has little to do with the device and > a lot to do with the controller. > > Is there a good way to get more information about what is going wrong? > For example, by enabling tracepoints in the xhci-hcd driver? > > Alan Stern > To enable xhci traces and dynamic debug at boot please add: "usbcore.dyndbg=+p xhci_hcd.dyndbg=+p trace_event=xhci-hcd trace_buf_size=80M" to the kernel cmdline before booting. (info added to https://bugzilla.redhat.com/show_bug.cgi?id=2019788 as well) Symptoms look similar to an old race issue where two usb devices were reset at the same time. xHC HW can't handle this, and to prevent it the hcd->address0_mutex was added. for details see: https://lkml.org/lkml/2016/2/8/312 -Mathias ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 23:54 ` Mathias Nyman @ 2021-11-15 22:05 ` Mathias Nyman 0 siblings, 0 replies; 9+ messages in thread From: Mathias Nyman @ 2021-11-15 22:05 UTC (permalink / raw) To: Alan Stern, Kishon Vijay Abraham I Cc: Hans de Goede, Greg Kroah-Hartman, stable, linux-usb, chris.chiu, Mathias Nyman On 4.11.2021 1.54, Mathias Nyman wrote: > On 3.11.2021 16.59, Alan Stern wrote: >> On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote: >>> + Alan, Chris, Mathias, linux-usb >>> >>> Hi Hans, >>> >>> On 03/11/21 6:18 pm, Hans de Goede wrote: >>>> Hi, >>>> >>>> On 11/3/21 10:07, Greg Kroah-Hartman wrote: >>>>> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: >>>>>> Hi Greg, >>>>>> >>>>>> We (Fedora) have been receiving multiple reports about USB devices stopping >>>>>> working starting with 5.14.14 . >>>>>> >>>>>> An Arch Linux user has found that reverting the first 2 patches from this series: >>>>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ >>>>>> >>>>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). >>>>>> >>>>>> See here for the Arch-linux discussion surrounding this: >>>>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 >>>>>> >>>>>> And here are 2 Fedora bug reports of Fedora users being unable to use their >>>>>> machines due their mouse + kbd not working: >>>>>> >>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 >>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 >>>>>> >>>>>> Can we get this patch-series reverted from the 5.14.y releases please ? >>>>> >>>>> Sure, >>>> >>>> Thanks. >>>> >>>>> but can you also submit patches to get into 5.15.y and 5.16-rc1 >>>>> that revert these changes as they should still be an issue there, right? >>>> >>>> Yes I assume this is still an issue there too, but I was hoping that >>>> Kishon can take a look and maybe actually fix things, since just >>>> reverting presumably regresses whatever these patches were addressing. >>>> >>>> We've aprox 1-3 weeks before distros like Arch and Linux will switch >>>> to 5.15.y kernels. So we have some time to come up with a fix >>>> there, where as for 5.14.y this is hitting users now. >>> >>> Is the issue with PCIe USB devices or platform USB device? Is it specific to >>> super speed devices or high speed device? >> >> Look at the bug reports. They are on standard PCs (so PCIe controllers) >> and some of them involve full speed (mouse and keyboard) devices. >> Although it looks like the problem has little to do with the device and >> a lot to do with the controller. >> >> Is there a good way to get more information about what is going wrong? >> For example, by enabling tracepoints in the xhci-hcd driver? >> >> Alan Stern >> > > To enable xhci traces and dynamic debug at boot please add: > "usbcore.dyndbg=+p xhci_hcd.dyndbg=+p trace_event=xhci-hcd trace_buf_size=80M" > to the kernel cmdline before booting. > (info added to https://bugzilla.redhat.com/show_bug.cgi?id=2019788 as well) > > Symptoms look similar to an old race issue where two usb devices were reset at the same time. > xHC HW can't handle this, and to prevent it the hcd->address0_mutex was added. > > for details see: > https://lkml.org/lkml/2016/2/8/312 > > -Mathias > Looks like it really is an issue with address0_mutex in hub.c not preventing two xhci slots from entering default state at the same time during retry. Seems extending the mutex across retries resolved this, details: https://bugzilla.redhat.com/show_bug.cgi?id=2019788 I'll post that patch here on linux-usb for review shortly -Mathias ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 12:48 ` Hans de Goede 2021-11-03 14:44 ` Kishon Vijay Abraham I @ 2021-11-03 17:45 ` Greg Kroah-Hartman 1 sibling, 0 replies; 9+ messages in thread From: Greg Kroah-Hartman @ 2021-11-03 17:45 UTC (permalink / raw) To: Hans de Goede; +Cc: stable, Kishon Vijay Abraham I On Wed, Nov 03, 2021 at 01:48:11PM +0100, Hans de Goede wrote: > Hi, > > On 11/3/21 10:07, Greg Kroah-Hartman wrote: > > On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: > >> Hi Greg, > >> > >> We (Fedora) have been receiving multiple reports about USB devices stopping > >> working starting with 5.14.14 . > >> > >> An Arch Linux user has found that reverting the first 2 patches from this series: > >> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ > >> > >> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). > >> > >> See here for the Arch-linux discussion surrounding this: > >> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 > >> > >> And here are 2 Fedora bug reports of Fedora users being unable to use their > >> machines due their mouse + kbd not working: > >> > >> https://bugzilla.redhat.com/show_bug.cgi?id=2019542 > >> https://bugzilla.redhat.com/show_bug.cgi?id=2019576 > >> > >> Can we get this patch-series reverted from the 5.14.y releases please ? > > > > Sure, > > Thanks. > > > but can you also submit patches to get into 5.15.y and 5.16-rc1 > > that revert these changes as they should still be an issue there, right? > > Yes I assume this is still an issue there too, but I was hoping that > Kishon can take a look and maybe actually fix things, since just > reverting presumably regresses whatever these patches were addressing. > > We've aprox 1-3 weeks before distros like Arch and Linux will switch > to 5.15.y kernels. So we have some time to come up with a fix > there, where as for 5.14.y this is hitting users now. I've reverted them from all stable kernels for now. Unless this gets figured out by 5.16-rc1, I will revert it from there then as well. thanks, greg k-h ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series 2021-11-03 9:07 ` Greg Kroah-Hartman 2021-11-03 12:48 ` Hans de Goede @ 2021-11-05 7:57 ` Greg Kroah-Hartman 1 sibling, 0 replies; 9+ messages in thread From: Greg Kroah-Hartman @ 2021-11-05 7:57 UTC (permalink / raw) To: Hans de Goede; +Cc: stable, Kishon Vijay Abraham I On Wed, Nov 03, 2021 at 10:07:18AM +0100, Greg Kroah-Hartman wrote: > On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote: > > Hi Greg, > > > > We (Fedora) have been receiving multiple reports about USB devices stopping > > working starting with 5.14.14 . > > > > An Arch Linux user has found that reverting the first 2 patches from this series: > > https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/ > > > > Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup). > > > > See here for the Arch-linux discussion surrounding this: > > https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956 > > > > And here are 2 Fedora bug reports of Fedora users being unable to use their > > machines due their mouse + kbd not working: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=2019542 > > https://bugzilla.redhat.com/show_bug.cgi?id=2019576 > > > > Can we get this patch-series reverted from the 5.14.y releases please ? > > Sure, but can you also submit patches to get into 5.15.y and 5.16-rc1 > that revert these changes as they should still be an issue there, right? I've reverted these in my tree now so that the revert will get into 5.16-rc1. thanks, greg k-h ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-11-15 22:05 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-03 9:02 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series Hans de Goede 2021-11-03 9:07 ` Greg Kroah-Hartman 2021-11-03 12:48 ` Hans de Goede 2021-11-03 14:44 ` Kishon Vijay Abraham I 2021-11-03 14:59 ` Alan Stern 2021-11-03 23:54 ` Mathias Nyman 2021-11-15 22:05 ` Mathias Nyman 2021-11-03 17:45 ` Greg Kroah-Hartman 2021-11-05 7:57 ` Greg Kroah-Hartman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).