5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Hans de Goede]

Hi Greg,

We (Fedora) have been receiving multiple reports about USB devices stopping
working starting with 5.14.14 .

An Arch Linux user has found that reverting the first 2 patches from this series:
https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/

Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).

See here for the Arch-linux discussion surrounding this:
https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956

And here are 2 Fedora bug reports of Fedora users being unable to use their
machines due their mouse + kbd not working:

https://bugzilla.redhat.com/show_bug.cgi?id=2019542
https://bugzilla.redhat.com/show_bug.cgi?id=2019576

Can we get this patch-series reverted from the 5.14.y releases please ?

Regards,

Hans

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Greg Kroah-Hartman]

On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
> Hi Greg,
> 
> We (Fedora) have been receiving multiple reports about USB devices stopping
> working starting with 5.14.14 .
> 
> An Arch Linux user has found that reverting the first 2 patches from this series:
> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> 
> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> 
> See here for the Arch-linux discussion surrounding this:
> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> 
> And here are 2 Fedora bug reports of Fedora users being unable to use their
> machines due their mouse + kbd not working:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> 
> Can we get this patch-series reverted from the 5.14.y releases please ?

Sure, but can you also submit patches to get into 5.15.y and 5.16-rc1
that revert these changes as they should still be an issue there, right?

thanks,

greg k-h

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Hans de Goede]

Hi,

On 11/3/21 10:07, Greg Kroah-Hartman wrote:
> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
>> Hi Greg,
>>
>> We (Fedora) have been receiving multiple reports about USB devices stopping
>> working starting with 5.14.14 .
>>
>> An Arch Linux user has found that reverting the first 2 patches from this series:
>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>
>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>
>> See here for the Arch-linux discussion surrounding this:
>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>
>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>> machines due their mouse + kbd not working:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>
>> Can we get this patch-series reverted from the 5.14.y releases please ?
> 
> Sure,

Thanks.

> but can you also submit patches to get into 5.15.y and 5.16-rc1
> that revert these changes as they should still be an issue there, right?

Yes I assume this is still an issue there too, but I was hoping that
Kishon can take a look and maybe actually fix things, since just
reverting presumably regresses whatever these patches were addressing.

We've aprox 1-3 weeks before distros like Arch and Linux will switch
to 5.15.y kernels.  So we have some time to come up with a fix
there, where as for 5.14.y this is hitting users now.

Regards,

Hans

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Kishon Vijay Abraham I]

+ Alan, Chris, Mathias, linux-usb

Hi Hans,

On 03/11/21 6:18 pm, Hans de Goede wrote:
> Hi,
> 
> On 11/3/21 10:07, Greg Kroah-Hartman wrote:
>> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
>>> Hi Greg,
>>>
>>> We (Fedora) have been receiving multiple reports about USB devices stopping
>>> working starting with 5.14.14 .
>>>
>>> An Arch Linux user has found that reverting the first 2 patches from this series:
>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>>
>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>>
>>> See here for the Arch-linux discussion surrounding this:
>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>>
>>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>>> machines due their mouse + kbd not working:
>>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>>
>>> Can we get this patch-series reverted from the 5.14.y releases please ?
>>
>> Sure,
> 
> Thanks.
> 
>> but can you also submit patches to get into 5.15.y and 5.16-rc1
>> that revert these changes as they should still be an issue there, right?
> 
> Yes I assume this is still an issue there too, but I was hoping that
> Kishon can take a look and maybe actually fix things, since just
> reverting presumably regresses whatever these patches were addressing.
> 
> We've aprox 1-3 weeks before distros like Arch and Linux will switch
> to 5.15.y kernels.  So we have some time to come up with a fix
> there, where as for 5.14.y this is hitting users now.

Is the issue with PCIe USB devices or platform USB device? Is it specific to
super speed devices or high speed device?

Thanks,
Kishon

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Alan Stern]

On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote:
> + Alan, Chris, Mathias, linux-usb
> 
> Hi Hans,
> 
> On 03/11/21 6:18 pm, Hans de Goede wrote:
> > Hi,
> > 
> > On 11/3/21 10:07, Greg Kroah-Hartman wrote:
> >> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
> >>> Hi Greg,
> >>>
> >>> We (Fedora) have been receiving multiple reports about USB devices stopping
> >>> working starting with 5.14.14 .
> >>>
> >>> An Arch Linux user has found that reverting the first 2 patches from this series:
> >>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> >>>
> >>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> >>>
> >>> See here for the Arch-linux discussion surrounding this:
> >>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> >>>
> >>> And here are 2 Fedora bug reports of Fedora users being unable to use their
> >>> machines due their mouse + kbd not working:
> >>>
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> >>>
> >>> Can we get this patch-series reverted from the 5.14.y releases please ?
> >>
> >> Sure,
> > 
> > Thanks.
> > 
> >> but can you also submit patches to get into 5.15.y and 5.16-rc1
> >> that revert these changes as they should still be an issue there, right?
> > 
> > Yes I assume this is still an issue there too, but I was hoping that
> > Kishon can take a look and maybe actually fix things, since just
> > reverting presumably regresses whatever these patches were addressing.
> > 
> > We've aprox 1-3 weeks before distros like Arch and Linux will switch
> > to 5.15.y kernels.  So we have some time to come up with a fix
> > there, where as for 5.14.y this is hitting users now.
> 
> Is the issue with PCIe USB devices or platform USB device? Is it specific to
> super speed devices or high speed device?

Look at the bug reports.  They are on standard PCs (so PCIe controllers) 
and some of them involve full speed (mouse and keyboard) devices.  
Although it looks like the problem has little to do with the device and 
a lot to do with the controller.

Is there a good way to get more information about what is going wrong?  
For example, by enabling tracepoints in the xhci-hcd driver?

Alan Stern

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Mathias Nyman]

On 3.11.2021 16.59, Alan Stern wrote:
> On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote:
>> + Alan, Chris, Mathias, linux-usb
>>
>> Hi Hans,
>>
>> On 03/11/21 6:18 pm, Hans de Goede wrote:
>>> Hi,
>>>
>>> On 11/3/21 10:07, Greg Kroah-Hartman wrote:
>>>> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
>>>>> Hi Greg,
>>>>>
>>>>> We (Fedora) have been receiving multiple reports about USB devices stopping
>>>>> working starting with 5.14.14 .
>>>>>
>>>>> An Arch Linux user has found that reverting the first 2 patches from this series:
>>>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>>>>
>>>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>>>>
>>>>> See here for the Arch-linux discussion surrounding this:
>>>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>>>>
>>>>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>>>>> machines due their mouse + kbd not working:
>>>>>
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>>>>
>>>>> Can we get this patch-series reverted from the 5.14.y releases please ?
>>>>
>>>> Sure,
>>>
>>> Thanks.
>>>
>>>> but can you also submit patches to get into 5.15.y and 5.16-rc1
>>>> that revert these changes as they should still be an issue there, right?
>>>
>>> Yes I assume this is still an issue there too, but I was hoping that
>>> Kishon can take a look and maybe actually fix things, since just
>>> reverting presumably regresses whatever these patches were addressing.
>>>
>>> We've aprox 1-3 weeks before distros like Arch and Linux will switch
>>> to 5.15.y kernels.  So we have some time to come up with a fix
>>> there, where as for 5.14.y this is hitting users now.
>>
>> Is the issue with PCIe USB devices or platform USB device? Is it specific to
>> super speed devices or high speed device?
> 
> Look at the bug reports.  They are on standard PCs (so PCIe controllers) 
> and some of them involve full speed (mouse and keyboard) devices.  
> Although it looks like the problem has little to do with the device and 
> a lot to do with the controller.
> 
> Is there a good way to get more information about what is going wrong?  
> For example, by enabling tracepoints in the xhci-hcd driver?
> 
> Alan Stern
> 

To enable xhci traces and dynamic debug at boot please add:
"usbcore.dyndbg=+p xhci_hcd.dyndbg=+p trace_event=xhci-hcd trace_buf_size=80M"
to the kernel cmdline before booting.
(info added to https://bugzilla.redhat.com/show_bug.cgi?id=2019788 as well)

Symptoms look similar to an old race issue where two usb devices were reset at the same time.
xHC HW can't handle this, and to prevent it the hcd->address0_mutex was added.

for details see: 
https://lkml.org/lkml/2016/2/8/312

-Mathias

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Mathias Nyman]

On 4.11.2021 1.54, Mathias Nyman wrote:
> On 3.11.2021 16.59, Alan Stern wrote:
>> On Wed, Nov 03, 2021 at 08:14:35PM +0530, Kishon Vijay Abraham I wrote:
>>> + Alan, Chris, Mathias, linux-usb
>>>
>>> Hi Hans,
>>>
>>> On 03/11/21 6:18 pm, Hans de Goede wrote:
>>>> Hi,
>>>>
>>>> On 11/3/21 10:07, Greg Kroah-Hartman wrote:
>>>>> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
>>>>>> Hi Greg,
>>>>>>
>>>>>> We (Fedora) have been receiving multiple reports about USB devices stopping
>>>>>> working starting with 5.14.14 .
>>>>>>
>>>>>> An Arch Linux user has found that reverting the first 2 patches from this series:
>>>>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>>>>>
>>>>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>>>>>
>>>>>> See here for the Arch-linux discussion surrounding this:
>>>>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>>>>>
>>>>>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>>>>>> machines due their mouse + kbd not working:
>>>>>>
>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>>>>>
>>>>>> Can we get this patch-series reverted from the 5.14.y releases please ?
>>>>>
>>>>> Sure,
>>>>
>>>> Thanks.
>>>>
>>>>> but can you also submit patches to get into 5.15.y and 5.16-rc1
>>>>> that revert these changes as they should still be an issue there, right?
>>>>
>>>> Yes I assume this is still an issue there too, but I was hoping that
>>>> Kishon can take a look and maybe actually fix things, since just
>>>> reverting presumably regresses whatever these patches were addressing.
>>>>
>>>> We've aprox 1-3 weeks before distros like Arch and Linux will switch
>>>> to 5.15.y kernels.  So we have some time to come up with a fix
>>>> there, where as for 5.14.y this is hitting users now.
>>>
>>> Is the issue with PCIe USB devices or platform USB device? Is it specific to
>>> super speed devices or high speed device?
>>
>> Look at the bug reports.  They are on standard PCs (so PCIe controllers) 
>> and some of them involve full speed (mouse and keyboard) devices.  
>> Although it looks like the problem has little to do with the device and 
>> a lot to do with the controller.
>>
>> Is there a good way to get more information about what is going wrong?  
>> For example, by enabling tracepoints in the xhci-hcd driver?
>>
>> Alan Stern
>>
> 
> To enable xhci traces and dynamic debug at boot please add:
> "usbcore.dyndbg=+p xhci_hcd.dyndbg=+p trace_event=xhci-hcd trace_buf_size=80M"
> to the kernel cmdline before booting.
> (info added to https://bugzilla.redhat.com/show_bug.cgi?id=2019788 as well)
> 
> Symptoms look similar to an old race issue where two usb devices were reset at the same time.
> xHC HW can't handle this, and to prevent it the hcd->address0_mutex was added.
> 
> for details see: 
> https://lkml.org/lkml/2016/2/8/312
> 
> -Mathias
> 

Looks like it really is an issue with address0_mutex in hub.c not preventing
two xhci slots from entering default state at the same time during retry.

Seems extending the mutex across retries resolved this, details:
https://bugzilla.redhat.com/show_bug.cgi?id=2019788

I'll post that patch here on linux-usb for review shortly

-Mathias

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Greg Kroah-Hartman]

On Wed, Nov 03, 2021 at 01:48:11PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 11/3/21 10:07, Greg Kroah-Hartman wrote:
> > On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
> >> Hi Greg,
> >>
> >> We (Fedora) have been receiving multiple reports about USB devices stopping
> >> working starting with 5.14.14 .
> >>
> >> An Arch Linux user has found that reverting the first 2 patches from this series:
> >> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> >>
> >> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> >>
> >> See here for the Arch-linux discussion surrounding this:
> >> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> >>
> >> And here are 2 Fedora bug reports of Fedora users being unable to use their
> >> machines due their mouse + kbd not working:
> >>
> >> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> >> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> >>
> >> Can we get this patch-series reverted from the 5.14.y releases please ?
> > 
> > Sure,
> 
> Thanks.
> 
> > but can you also submit patches to get into 5.15.y and 5.16-rc1
> > that revert these changes as they should still be an issue there, right?
> 
> Yes I assume this is still an issue there too, but I was hoping that
> Kishon can take a look and maybe actually fix things, since just
> reverting presumably regresses whatever these patches were addressing.
> 
> We've aprox 1-3 weeks before distros like Arch and Linux will switch
> to 5.15.y kernels.  So we have some time to come up with a fix
> there, where as for 5.14.y this is hitting users now.

I've reverted them from all stable kernels for now.  Unless this gets
figured out by 5.16-rc1, I will revert it from there then as well.

thanks,

greg k-h

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Greg Kroah-Hartman]

On Wed, Nov 03, 2021 at 10:07:18AM +0100, Greg Kroah-Hartman wrote:
> On Wed, Nov 03, 2021 at 10:02:52AM +0100, Hans de Goede wrote:
> > Hi Greg,
> > 
> > We (Fedora) have been receiving multiple reports about USB devices stopping
> > working starting with 5.14.14 .
> > 
> > An Arch Linux user has found that reverting the first 2 patches from this series:
> > https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> > 
> > Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> > 
> > See here for the Arch-linux discussion surrounding this:
> > https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> > 
> > And here are 2 Fedora bug reports of Fedora users being unable to use their
> > machines due their mouse + kbd not working:
> > 
> > https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> > https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> > 
> > Can we get this patch-series reverted from the 5.14.y releases please ?
> 
> Sure, but can you also submit patches to get into 5.15.y and 5.16-rc1
> that revert these changes as they should still be an issue there, right?

I've reverted these in my tree now so that the revert will get into
5.16-rc1.

thanks,

greg k-h

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Thorsten Leemhuis]

On 03.11.21 10:02, Hans de Goede wrote:
> 
> We (Fedora) have been receiving multiple reports about USB devices stopping
> working starting with 5.14.14 .
> 
> An Arch Linux user has found that reverting the first 2 patches from this series:
> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> 
> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> 
> See here for the Arch-linux discussion surrounding this:
> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> 
> And here are 2 Fedora bug reports of Fedora users being unable to use their
> machines due their mouse + kbd not working:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> 
> Can we get this patch-series reverted from the 5.14.y releases please ?
#regzbot ^introduced: b7a0a792f864583207c593b50fd1b752ed89f4c1
#regzbot ignore-activity

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Greg KH]

On Thu, Nov 04, 2021 at 07:53:37AM +0100, Thorsten Leemhuis wrote:
> On 03.11.21 10:02, Hans de Goede wrote:
> > 
> > We (Fedora) have been receiving multiple reports about USB devices stopping
> > working starting with 5.14.14 .
> > 
> > An Arch Linux user has found that reverting the first 2 patches from this series:
> > https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> > 
> > Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> > 
> > See here for the Arch-linux discussion surrounding this:
> > https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> > 
> > And here are 2 Fedora bug reports of Fedora users being unable to use their
> > machines due their mouse + kbd not working:
> > 
> > https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> > https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> > 
> > Can we get this patch-series reverted from the 5.14.y releases please ?
> #regzbot ^introduced: b7a0a792f864583207c593b50fd1b752ed89f4c1
> #regzbot ignore-activity
> 

Process question.  Is there anything I need to do here when this is
resolved?  I'm thinking of reverting the offending patches in 5.16-rc1,
but when/if that happens, how will the regzbot know this?

thanks,

greg k-h

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Thorsten Leemhuis]

aOn 04.11.21 08:04, Greg KH wrote:
> On Thu, Nov 04, 2021 at 07:53:37AM +0100, Thorsten Leemhuis wrote:
>> On 03.11.21 10:02, Hans de Goede wrote:
>>>
>>> We (Fedora) have been receiving multiple reports about USB devices stopping
>>> working starting with 5.14.14 .
>>>
>>> An Arch Linux user has found that reverting the first 2 patches from this series:
>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>>
>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>>
>>> See here for the Arch-linux discussion surrounding this:
>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>>
>>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>>> machines due their mouse + kbd not working:
>>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>>
>>> Can we get this patch-series reverted from the 5.14.y releases please ?
>> #regzbot ^introduced: b7a0a792f864583207c593b50fd1b752ed89f4c1
>> #regzbot ignore-activity
> 
> Process question.  Is there anything I need to do here when this is
> resolved?  I'm thinking of reverting the offending patches in 5.16-rc1,
> but when/if that happens, how will the regzbot know this?

You don't need to do something special, but you need to do something ;-)
Regzbot will know the regression got resolved if the commit message of
the fix (a proper one or a revert) in mainline links to the report with
a Link: tag. Hence, the commit should have something like this:

Link:
https://lore.kernel.org/stable/42bcbea6-5eb8-16c7-336a-2cb72e71bc36@redhat.com/

This is something you and many other developers often do already -- and
something that submitting-patches.rst since a few days states
explicitly: https://git.kernel.org/torvalds/c/1f57bd42b77c

But even if you forget that: As a regression tracker I still have an eye
on things and likely would tell regzbot about it myself, *if* I notice
the fix/revert got merged (and if not, I might send a quick 'hey' what's
up here' mail, which will be a bit annoying for the involved parties,
but that's how it is).

Side note: regbot will even notice if a patch with above Link-Tag gets
posted to one of the mainline list and link to it.


That afaics answered the question, but TWIMC: this report created some
challenges for me/regzbot due to two aspects:

1) the report specifies two commits as culprit
2) the problematic changes are in mainline, 5.15.y and 5.14.y, and I'm
unsure as of now how to handle such situations

For the first aspect I for now settled on "well, just mentioned the
second of the two commits, that's good enough for regzbot, the details
are in the report anyway".

I'm ignoring the second aspect for now and told regzbot only about the
commit in mainline, as my focus for regzbot is on mainline. I guess to
handle this properly, it would be possible to enhance regzbot and then
tall it about 807ac762afee (that's the backport of b7a0a792f864 to 5.14)
as well. But you and Sasha afaics handle things like this already quite
well afaics. But if that would make your life easier, just let me know,
than I'll put it higher on my todo list.

Ciao, Thorsten

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Greg KH]

On Thu, Nov 04, 2021 at 09:50:12AM +0100, Thorsten Leemhuis wrote:
> aOn 04.11.21 08:04, Greg KH wrote:
> > On Thu, Nov 04, 2021 at 07:53:37AM +0100, Thorsten Leemhuis wrote:
> >> On 03.11.21 10:02, Hans de Goede wrote:
> >>>
> >>> We (Fedora) have been receiving multiple reports about USB devices stopping
> >>> working starting with 5.14.14 .
> >>>
> >>> An Arch Linux user has found that reverting the first 2 patches from this series:
> >>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
> >>>
> >>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
> >>>
> >>> See here for the Arch-linux discussion surrounding this:
> >>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
> >>>
> >>> And here are 2 Fedora bug reports of Fedora users being unable to use their
> >>> machines due their mouse + kbd not working:
> >>>
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
> >>>
> >>> Can we get this patch-series reverted from the 5.14.y releases please ?
> >> #regzbot ^introduced: b7a0a792f864583207c593b50fd1b752ed89f4c1
> >> #regzbot ignore-activity
> > 
> > Process question.  Is there anything I need to do here when this is
> > resolved?  I'm thinking of reverting the offending patches in 5.16-rc1,
> > but when/if that happens, how will the regzbot know this?
> 
> You don't need to do something special, but you need to do something ;-)
> Regzbot will know the regression got resolved if the commit message of
> the fix (a proper one or a revert) in mainline links to the report with
> a Link: tag. Hence, the commit should have something like this:
> 
> Link:
> https://lore.kernel.org/stable/42bcbea6-5eb8-16c7-336a-2cb72e71bc36@redhat.com/

Close, the patch will have this link tag:
    Link: https://lore.kernel.org/r/42bcbea6-5eb8-16c7-336a-2cb72e71bc36@redhat.com

Is that ok?

I've committed the reverts to my tree and pushed them out so they show
up in the next linux-next release and will get the reverts to Linus
before 5.16-rc1.  Let's see if the bot can notice them :)

thanks,

greg k-h

Re: 5.14.14+ USB regression caused by "usb: core: hcd: Add support for deferring roothub registration" series

[Thorsten Leemhuis]

On 05.11.21 08:56, Greg KH wrote:
> On Thu, Nov 04, 2021 at 09:50:12AM +0100, Thorsten Leemhuis wrote:
>> aOn 04.11.21 08:04, Greg KH wrote:
>>> On Thu, Nov 04, 2021 at 07:53:37AM +0100, Thorsten Leemhuis wrote:
>>>> On 03.11.21 10:02, Hans de Goede wrote:
>>>>>
>>>>> We (Fedora) have been receiving multiple reports about USB devices stopping
>>>>> working starting with 5.14.14 .
>>>>>
>>>>> An Arch Linux user has found that reverting the first 2 patches from this series:
>>>>> https://lore.kernel.org/all/20210909064200.16216-1-kishon@ti.com/
>>>>>
>>>>> Fixes things (the 3th patch is just some mostly unrelated refactoring/cleanup).
>>>>>
>>>>> See here for the Arch-linux discussion surrounding this:
>>>>> https://bbs.archlinux.org/viewtopic.php?pid=2000956#p2000956
>>>>>
>>>>> And here are 2 Fedora bug reports of Fedora users being unable to use their
>>>>> machines due their mouse + kbd not working:
>>>>>
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019542
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=2019576
>>>>>
>>>>> Can we get this patch-series reverted from the 5.14.y releases please ?
>>>> #regzbot ^introduced: b7a0a792f864583207c593b50fd1b752ed89f4c1
>>>> #regzbot ignore-activity
>>>
>>> Process question.  Is there anything I need to do here when this is
>>> resolved?  I'm thinking of reverting the offending patches in 5.16-rc1,
>>> but when/if that happens, how will the regzbot know this?
>>
>> You don't need to do something special, but you need to do something ;-)
>> Regzbot will know the regression got resolved if the commit message of
>> the fix (a proper one or a revert) in mainline links to the report with
>> a Link: tag. Hence, the commit should have something like this:
>>
>> Link:
>> https://lore.kernel.org/stable/42bcbea6-5eb8-16c7-336a-2cb72e71bc36@redhat.com/
> 
> Close, the patch will have this link tag:
>     Link: https://lore.kernel.org/r/42bcbea6-5eb8-16c7-336a-2cb72e71bc36@redhat.com
> 
> Is that ok?

Yes, of course it is, in fact I just forgot to do s/stable/r/ when
writing above mail. But an url with /stable/ in it would work, too, as
Regzbot in this case will not care what's between lore.kernel.org/ and
the /<msgid> part.

> I've committed the reverts to my tree and pushed them out so they show
> up in the next linux-next release and will get the reverts to Linus
> before 5.16-rc1.  Let's see if the bot can notice them :)

It should even notice them in linux-next and mark the regression as "to
be fixed" until the commits land mainline. Just to be sure I wanted to
wait for this to happen before writing this mail, but forgot it's
Saturday today -- there thus is no new linux-next, so I have to wait
till Monday. :-/

Ciao, Thorsten