* [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
@ 2021-07-14 14:03 David Jeffery
2021-07-14 14:27 ` Andy Shevchenko
2021-07-14 14:29 ` Alan Stern
0 siblings, 2 replies; 7+ messages in thread
From: David Jeffery @ 2021-07-14 14:03 UTC (permalink / raw)
To: linux-usb; +Cc: loberman, andriy.shevchenko, emilne, apanagio, torez
When MSI is used by the ehci driver, it can cause interrupts to be lost which
results in ehci only continuing to work due to its polling fallback. But the
reliance of polling drasticly reduces performance of any I/O through ehci.
Interrupts are lost as ehci's interrupt handler does not safely handle
edge-triggered interrupts. It fails to ensure all interrupt status bits are
cleared, which works with level-triggered interrupts but not the
edge-triggered interrupts typical from using MSI.
To fix this problem, check if the driver may have raced with the hardware
setting additional interrupt status bits and clear status until it is in a
stable state.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
This is an alternative to reverting 306c54d0edb6ba94d39877524dddebaad7770cf2
which is the patch that allowed MSI to be used with ehci.
ehci-hcd.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
index 36f5bf6a0752..2283205d4b40 100644
--- a/drivers/usb/host/ehci-hcd.c
+++ b/drivers/usb/host/ehci-hcd.c
@@ -704,14 +704,18 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
{
struct ehci_hcd *ehci = hcd_to_ehci (hcd);
u32 status, masked_status, pcd_status = 0, cmd;
+ u32 current_status;
int bh;
spin_lock(&ehci->lock);
- status = ehci_readl(ehci, &ehci->regs->status);
+ status = 0;
+ current_status = ehci_readl(ehci, &ehci->regs->status);
+restart:
+ status |= current_status;
/* e.g. cardbus physical eject */
- if (status == ~(u32) 0) {
+ if (current_status == ~(u32) 0) {
ehci_dbg (ehci, "device removed\n");
goto dead;
}
@@ -720,7 +724,7 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
* We don't use STS_FLR, but some controllers don't like it to
* remain on, so mask it out along with the other status bits.
*/
- masked_status = status & (INTR_MASK | STS_FLR);
+ masked_status = current_status & (INTR_MASK | STS_FLR);
/* Shared IRQ? */
if (!masked_status || unlikely(ehci->rh_state == EHCI_RH_HALTED)) {
@@ -730,6 +734,12 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
/* clear (just) interrupts */
ehci_writel(ehci, masked_status, &ehci->regs->status);
+
+ /* for edge interrupts, don't race with an interrupt bit being raised */
+ current_status = ehci_readl(ehci, &ehci->regs->status);
+ if (current_status & INTR_MASK)
+ goto restart;
+
cmd = ehci_readl(ehci, &ehci->regs->command);
bh = 0;
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 14:03 [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI David Jeffery
@ 2021-07-14 14:27 ` Andy Shevchenko
2021-07-14 15:55 ` David Jeffery
2021-07-14 14:29 ` Alan Stern
1 sibling, 1 reply; 7+ messages in thread
From: Andy Shevchenko @ 2021-07-14 14:27 UTC (permalink / raw)
To: David Jeffery; +Cc: linux-usb, loberman, emilne, apanagio, torez
On Wed, Jul 14, 2021 at 10:03:09AM -0400, David Jeffery wrote:
Thanks, I have few minor comments, after addressing them feel free to add
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> When MSI is used by the ehci driver, it can cause interrupts to be lost which
ehci -> EHCI everywhere?
> results in ehci only continuing to work due to its polling fallback. But the
> reliance of polling drasticly reduces performance of any I/O through ehci.
>
> Interrupts are lost as ehci's interrupt handler does not safely handle
> edge-triggered interrupts. It fails to ensure all interrupt status bits are
> cleared, which works with level-triggered interrupts but not the
Drop trailing white space here and double spaces in the couple of places above?
> edge-triggered interrupts typical from using MSI.
>
> To fix this problem, check if the driver may have raced with the hardware
> setting additional interrupt status bits and clear status until it is in a
> stable state.
Fixes tag?
> Signed-off-by: David Jeffery <djeffery@redhat.com>
> Tested-by: Laurence Oberman <loberman@redhat.com>
> ---
>
> This is an alternative to reverting 306c54d0edb6ba94d39877524dddebaad7770cf2
> which is the patch that allowed MSI to be used with ehci.
Thanks!
>
> ehci-hcd.c | 16 +++++++++++++---
> 1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
> index 36f5bf6a0752..2283205d4b40 100644
> --- a/drivers/usb/host/ehci-hcd.c
> +++ b/drivers/usb/host/ehci-hcd.c
> @@ -704,14 +704,18 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> {
> struct ehci_hcd *ehci = hcd_to_ehci (hcd);
> u32 status, masked_status, pcd_status = 0, cmd;
> + u32 current_status;
Perhaps
u32 status, current_status, masked_status, pcd_status = 0;
u32 cmd;
?
> int bh;
>
> spin_lock(&ehci->lock);
>
> - status = ehci_readl(ehci, &ehci->regs->status);
> + status = 0;
>
> + current_status = ehci_readl(ehci, &ehci->regs->status);
> +restart:
> + status |= current_status;
> /* e.g. cardbus physical eject */
> - if (status == ~(u32) 0) {
> + if (current_status == ~(u32) 0) {
> ehci_dbg (ehci, "device removed\n");
> goto dead;
> }
> @@ -720,7 +724,7 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> * We don't use STS_FLR, but some controllers don't like it to
> * remain on, so mask it out along with the other status bits.
> */
> - masked_status = status & (INTR_MASK | STS_FLR);
> + masked_status = current_status & (INTR_MASK | STS_FLR);
>
> /* Shared IRQ? */
> if (!masked_status || unlikely(ehci->rh_state == EHCI_RH_HALTED)) {
> @@ -730,6 +734,12 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
>
> /* clear (just) interrupts */
> ehci_writel(ehci, masked_status, &ehci->regs->status);
> +
> + /* for edge interrupts, don't race with an interrupt bit being raised */
for -> For
> + current_status = ehci_readl(ehci, &ehci->regs->status);
> + if (current_status & INTR_MASK)
> + goto restart;
> +
> cmd = ehci_readl(ehci, &ehci->regs->command);
> bh = 0;
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 14:03 [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI David Jeffery
2021-07-14 14:27 ` Andy Shevchenko
@ 2021-07-14 14:29 ` Alan Stern
2021-07-14 16:10 ` David Jeffery
1 sibling, 1 reply; 7+ messages in thread
From: Alan Stern @ 2021-07-14 14:29 UTC (permalink / raw)
To: David Jeffery
Cc: linux-usb, loberman, andriy.shevchenko, emilne, apanagio, torez
On Wed, Jul 14, 2021 at 10:03:09AM -0400, David Jeffery wrote:
> When MSI is used by the ehci driver, it can cause interrupts to be lost which
> results in ehci only continuing to work due to its polling fallback. But the
> reliance of polling drasticly reduces performance of any I/O through ehci.
>
> Interrupts are lost as ehci's interrupt handler does not safely handle
> edge-triggered interrupts. It fails to ensure all interrupt status bits are
> cleared, which works with level-triggered interrupts but not the
> edge-triggered interrupts typical from using MSI.
>
> To fix this problem, check if the driver may have raced with the hardware
> setting additional interrupt status bits and clear status until it is in a
> stable state.
>
> Signed-off-by: David Jeffery <djeffery@redhat.com>
> Tested-by: Laurence Oberman <loberman@redhat.com>
> ---
>
> This is an alternative to reverting 306c54d0edb6ba94d39877524dddebaad7770cf2
> which is the patch that allowed MSI to be used with ehci.
>
> ehci-hcd.c | 16 +++++++++++++---
> 1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
> index 36f5bf6a0752..2283205d4b40 100644
> --- a/drivers/usb/host/ehci-hcd.c
> +++ b/drivers/usb/host/ehci-hcd.c
> @@ -704,14 +704,18 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> {
> struct ehci_hcd *ehci = hcd_to_ehci (hcd);
> u32 status, masked_status, pcd_status = 0, cmd;
> + u32 current_status;
> int bh;
>
> spin_lock(&ehci->lock);
>
> - status = ehci_readl(ehci, &ehci->regs->status);
> + status = 0;
>
> + current_status = ehci_readl(ehci, &ehci->regs->status);
> +restart:
> + status |= current_status;
> /* e.g. cardbus physical eject */
> - if (status == ~(u32) 0) {
> + if (current_status == ~(u32) 0) {
> ehci_dbg (ehci, "device removed\n");
> goto dead;
> }
Mild stylistic quibble: I generally prefer to have a blank line before a
/* ... */ comment. And it doesn't seem reasonable to have a blank line
between "status = 0" and the current_status assignment, since those are
similar once-only things before the beginning of the "restart" loop.
Also, I would move the "status |= current_status" line after the test
for device removed, since that test doesn't use status.
But obviously none of these things affect the patch's correntness.
> @@ -720,7 +724,7 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> * We don't use STS_FLR, but some controllers don't like it to
> * remain on, so mask it out along with the other status bits.
> */
> - masked_status = status & (INTR_MASK | STS_FLR);
> + masked_status = current_status & (INTR_MASK | STS_FLR);
>
> /* Shared IRQ? */
> if (!masked_status || unlikely(ehci->rh_state == EHCI_RH_HALTED)) {
> @@ -730,6 +734,12 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
>
> /* clear (just) interrupts */
> ehci_writel(ehci, masked_status, &ehci->regs->status);
> +
> + /* for edge interrupts, don't race with an interrupt bit being raised */
> + current_status = ehci_readl(ehci, &ehci->regs->status);
> + if (current_status & INTR_MASK)
> + goto restart;
> +
> cmd = ehci_readl(ehci, &ehci->regs->command);
> bh = 0;
You can choose to submit a new version of the patch with those stylistic
changes, or if you prefer, just stick with this version. Either way,
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Does this issue affect any other PCI-based host controller drivers?
Alan Stern
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 14:27 ` Andy Shevchenko
@ 2021-07-14 15:55 ` David Jeffery
2021-07-14 19:35 ` Andy Shevchenko
0 siblings, 1 reply; 7+ messages in thread
From: David Jeffery @ 2021-07-14 15:55 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: linux-usb, Laurence Oberman, Ewan Milne, apanagio, torez
On Wed, Jul 14, 2021 at 10:27 AM Andy Shevchenko
<andriy.shevchenko@linux.intel.com> wrote:
>
> On Wed, Jul 14, 2021 at 10:03:09AM -0400, David Jeffery wrote:
>
> Thanks, I have few minor comments, after addressing them feel free to add
> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>
> > When MSI is used by the ehci driver, it can cause interrupts to be lost which
>
> ehci -> EHCI everywhere?
>
Are you asking for a capitalization change in the text or asking what
all is affected by the bug?
> > diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
> > index 36f5bf6a0752..2283205d4b40 100644
> > --- a/drivers/usb/host/ehci-hcd.c
> > +++ b/drivers/usb/host/ehci-hcd.c
> > @@ -704,14 +704,18 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> > {
> > struct ehci_hcd *ehci = hcd_to_ehci (hcd);
> > u32 status, masked_status, pcd_status = 0, cmd;
> > + u32 current_status;
>
> Perhaps
>
> u32 status, current_status, masked_status, pcd_status = 0;
> u32 cmd;
>
> ?
>
Is this a style preference? I can change it and just did it in a way
to minimize line changes.
>
> > int bh;
> >
> > spin_lock(&ehci->lock);
> >
> > - status = ehci_readl(ehci, &ehci->regs->status);
> > + status = 0;
> >
> > + current_status = ehci_readl(ehci, &ehci->regs->status);
> > +restart:
> > + status |= current_status;
> > /* e.g. cardbus physical eject */
> > - if (status == ~(u32) 0) {
> > + if (current_status == ~(u32) 0) {
> > ehci_dbg (ehci, "device removed\n");
> > goto dead;
> > }
> > @@ -720,7 +724,7 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> > * We don't use STS_FLR, but some controllers don't like it to
> > * remain on, so mask it out along with the other status bits.
> > */
> > - masked_status = status & (INTR_MASK | STS_FLR);
> > + masked_status = current_status & (INTR_MASK | STS_FLR);
> >
> > /* Shared IRQ? */
> > if (!masked_status || unlikely(ehci->rh_state == EHCI_RH_HALTED)) {
> > @@ -730,6 +734,12 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> >
> > /* clear (just) interrupts */
> > ehci_writel(ehci, masked_status, &ehci->regs->status);
> > +
> > + /* for edge interrupts, don't race with an interrupt bit being raised */
>
> for -> For
>
I'll update it.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 14:29 ` Alan Stern
@ 2021-07-14 16:10 ` David Jeffery
2021-07-14 16:27 ` Alan Stern
0 siblings, 1 reply; 7+ messages in thread
From: David Jeffery @ 2021-07-14 16:10 UTC (permalink / raw)
To: Alan Stern
Cc: linux-usb, Laurence Oberman, Andy Shevchenko, Ewan Milne,
apanagio, torez
On Wed, Jul 14, 2021 at 10:29 AM Alan Stern <stern@rowland.harvard.edu> wrote:
>
> On Wed, Jul 14, 2021 at 10:03:09AM -0400, David Jeffery wrote:
> >
> > diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
> > index 36f5bf6a0752..2283205d4b40 100644
> > --- a/drivers/usb/host/ehci-hcd.c
> > +++ b/drivers/usb/host/ehci-hcd.c
> > @@ -704,14 +704,18 @@ static irqreturn_t ehci_irq (struct usb_hcd *hcd)
> > {
> > struct ehci_hcd *ehci = hcd_to_ehci (hcd);
> > u32 status, masked_status, pcd_status = 0, cmd;
> > + u32 current_status;
> > int bh;
> >
> > spin_lock(&ehci->lock);
> >
> > - status = ehci_readl(ehci, &ehci->regs->status);
> > + status = 0;
> >
> > + current_status = ehci_readl(ehci, &ehci->regs->status);
> > +restart:
> > + status |= current_status;
> > /* e.g. cardbus physical eject */
> > - if (status == ~(u32) 0) {
> > + if (current_status == ~(u32) 0) {
> > ehci_dbg (ehci, "device removed\n");
> > goto dead;
> > }
>
> Mild stylistic quibble: I generally prefer to have a blank line before a
> /* ... */ comment. And it doesn't seem reasonable to have a blank line
> between "status = 0" and the current_status assignment, since those are
> similar once-only things before the beginning of the "restart" loop.
> Also, I would move the "status |= current_status" line after the test
> for device removed, since that test doesn't use status.
>
> But obviously none of these things affect the patch's correntness.
I'll update it with requested changes and resend it.
>
> You can choose to submit a new version of the patch with those stylistic
> changes, or if you prefer, just stick with this version. Either way,
>
> Acked-by: Alan Stern <stern@rowland.harvard.edu>
>
> Does this issue affect any other PCI-based host controller drivers?
>
> Alan Stern
>
Possibly. The uhci driver should have the same issue if hardware
exists with it and has MSI for it. I suspect the ohci driver has a
similar issue, but I'm not familiar enough with its interfaces and
specification to say for sure.
David Jeffery
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 16:10 ` David Jeffery
@ 2021-07-14 16:27 ` Alan Stern
0 siblings, 0 replies; 7+ messages in thread
From: Alan Stern @ 2021-07-14 16:27 UTC (permalink / raw)
To: David Jeffery
Cc: linux-usb, Laurence Oberman, Andy Shevchenko, Ewan Milne,
apanagio, torez
On Wed, Jul 14, 2021 at 12:10:26PM -0400, David Jeffery wrote:
> On Wed, Jul 14, 2021 at 10:29 AM Alan Stern <stern@rowland.harvard.edu> wrote:
> >
> > Does this issue affect any other PCI-based host controller drivers?
> >
> > Alan Stern
> >
>
> Possibly. The uhci driver should have the same issue if hardware
> exists with it and has MSI for it. I suspect the ohci driver has a
> similar issue, but I'm not familiar enough with its interfaces and
> specification to say for sure.
Hmmm. I've never heard of an MSI implementation of either of those,
although it may be possible for OHCI. UHCI was used only by Intel and
one or two others; as far as I know none of them are producing those
controllers any more. Nowadays when people want to support USB-1.1
devices but not USB-3, they use an EHCI controller together with a
"rate-matching" hub, which avoids the need for any UHCI or OHCI
controller.
I was wondering more about dwc2 or other non-EHCI USB-2 controller
drivers.
Alan Stern
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI
2021-07-14 15:55 ` David Jeffery
@ 2021-07-14 19:35 ` Andy Shevchenko
0 siblings, 0 replies; 7+ messages in thread
From: Andy Shevchenko @ 2021-07-14 19:35 UTC (permalink / raw)
To: David Jeffery
Cc: Andy Shevchenko, USB, Laurence Oberman, Ewan Milne, apanagio, torez
On Wed, Jul 14, 2021 at 6:55 PM David Jeffery <djeffery@redhat.com> wrote:
> On Wed, Jul 14, 2021 at 10:27 AM Andy Shevchenko
> <andriy.shevchenko@linux.intel.com> wrote:
> > On Wed, Jul 14, 2021 at 10:03:09AM -0400, David Jeffery wrote:
> >
> > Thanks, I have few minor comments, after addressing them feel free to add
> > Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> >
> > > When MSI is used by the ehci driver, it can cause interrupts to be lost which
> >
> > ehci -> EHCI everywhere?
> >
>
> Are you asking for a capitalization change in the text or asking what
> all is affected by the bug?
I'm suggesting to capitalize the abbreviation (maybe not in all cases,
but where you refer to the hardware, for the driver it's called
ehci-hcd).
...
> > > u32 status, masked_status, pcd_status = 0, cmd;
> > > + u32 current_status;
> >
> > Perhaps
> >
> > u32 status, current_status, masked_status, pcd_status = 0;
> > u32 cmd;
> >
> > ?
>
> Is this a style preference? I can change it and just did it in a way
> to minimize line changes.
Yes, I believe it's a preference of having something unified
semantically on one line.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-07-14 19:36 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-14 14:03 [PATCH] usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI David Jeffery
2021-07-14 14:27 ` Andy Shevchenko
2021-07-14 15:55 ` David Jeffery
2021-07-14 19:35 ` Andy Shevchenko
2021-07-14 14:29 ` Alan Stern
2021-07-14 16:10 ` David Jeffery
2021-07-14 16:27 ` Alan Stern
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.