[PATCH] USB: Fix xhci ERDP update issue

[PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On some situations, software handles TRB events slower than adding TRBs,
xhci_irq will not exit until all events are handled. If xhci_irq just
handles 256 TRBs and exit, the temp variable(event_ring_deq) driver 
records in xhci irq is equal to driver current dequeue pointer. It will 
cause driver not update ERDP and software dequeue pointer lost sync with 
ERDP. On the next xhci_irq, the event ring is full but driver will not 
update ERDP as software dequeue pointer is equal to ERDP.

[  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
[  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 
inflight: CMD OUT
[  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 
00 00 08 00
[  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host 
not responding to stop endpoint command.
[  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
[  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not 
responding, assume dead
[  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 
inflight: CMD
[  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 
00 00 08 00
[  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 
inflight: CMD

Fixed this issue by update software record temp variable when handles 
128 TRB events.

Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
---
  drivers/usb/host/xhci-ring.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index d0b6806..f970799 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
                 if (event_loop++ < TRBS_PER_SEGMENT / 2)
                         continue;
                 xhci_update_erst_dequeue(xhci, event_ring_deq);
+               event_ring_deq = xhci->event_ring->dequeue;

                 /* ring is half-full, force isoc trbs to interrupt more 
often */
                 if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
-- 
2.7.4

Re: [PATCH] USB: Fix xhci ERDP update issue

[Greg KH]

On Mon, Mar 14, 2022 at 03:25:23PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
> On some situations, software handles TRB events slower than adding TRBs,
> xhci_irq will not exit until all events are handled. If xhci_irq just
> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records
> in xhci irq is equal to driver current dequeue pointer. It will cause driver
> not update ERDP and software dequeue pointer lost sync with ERDP. On the
> next xhci_irq, the event ring is full but driver will not update ERDP as
> software dequeue pointer is equal to ERDP.
> 
> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
> inflight: CMD OUT
> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00
> 00 08 00
> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not
> responding to stop endpoint command.
> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding,
> assume dead
> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
> inflight: CMD
> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00
> 00 08 00
> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
> inflight: CMD
> 
> Fixed this issue by update software record temp variable when handles 128
> TRB events.
> 
> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
> ---
>  drivers/usb/host/xhci-ring.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
> index d0b6806..f970799 100644
> --- a/drivers/usb/host/xhci-ring.c
> +++ b/drivers/usb/host/xhci-ring.c
> @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
>                 if (event_loop++ < TRBS_PER_SEGMENT / 2)
>                         continue;
>                 xhci_update_erst_dequeue(xhci, event_ring_deq);
> +               event_ring_deq = xhci->event_ring->dequeue;
> 
>                 /* ring is half-full, force isoc trbs to interrupt more
> often */
>                 if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
> -- 
> 2.7.4

Hi,

This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
a patch that has triggered this response.  He used to manually respond
to these common problems, but in order to save his sanity (he kept
writing the same thing over and over, yet to different people), I was
created.  Hopefully you will not take offence and will fix the problem
in your patch and resubmit it so that it can be accepted into the Linux
kernel tree.

You are receiving this message because of the following common error(s)
as indicated below:

- Your patch is malformed (tabs converted to spaces, linewrapped, etc.)
  and can not be applied.  Please read the file,
  Documentation/email-clients.txt in order to fix this.


If you wish to discuss this problem further, or you have questions about
how to resolve this issue, please feel free to respond to this email and
Greg will reply once he has dug out from the pending patches received
from other developers.

thanks,

greg k-h's patch email bot

Re: [PATCH] USB: Fix xhci ERDP update issue

[Mathias Nyman]

On 14.3.2022 9.25, WeitaoWang-oc@zhaoxin.com wrote:
> On some situations, software handles TRB events slower than adding TRBs,
> xhci_irq will not exit until all events are handled. If xhci_irq just
> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records in xhci irq is equal to driver current dequeue pointer. It will cause driver not update ERDP and software dequeue pointer lost sync with ERDP. On the next xhci_irq, the event ring is full but driver will not update ERDP as software dequeue pointer is equal to ERDP.
> 
> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT
> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00 00 08 00
> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not responding to stop endpoint command.
> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding, assume dead
> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD
> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00 00 08 00
> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD
> 
> Fixed this issue by update software record temp variable when handles 128 TRB events.> 
> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>

Thanks

Solution itself looks good but patch has some minor format issue:


It would also be interesting to know if the full event ring was triggered in a real
life usecase?
If that is the case I might need to look more into it.

Bigger event ring, more rings, faster handler, avoid irqoff time...

Thanks
Mathias 

Re: [PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On 2022/3/15 下午4:08, Mathias Nyman wrote:
> On 14.3.2022 9.25, WeitaoWang-oc@zhaoxin.com wrote:
>> On some situations, software handles TRB events slower than adding TRBs,
>> xhci_irq will not exit until all events are handled. If xhci_irq just
>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records in xhci irq is equal to driver current dequeue pointer. It will cause driver not update ERDP and software dequeue pointer lost sync with ERDP. On the next xhci_irq, the event ring is full but driver will not update ERDP as software dequeue pointer is equal to ERDP.
>>
>> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
>> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT
>> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00 00 08 00
>> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not responding to stop endpoint command.
>> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
>> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding, assume dead
>> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD
>> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00 00 08 00
>> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD
>>
>> Fixed this issue by update software record temp variable when handles 128 TRB events.>
>> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
> 
> Thanks
> 
> Solution itself looks good but patch has some minor format issue:
> 
> 
> It would also be interesting to know if the full event ring was triggered in a real
> life usecase?
> If that is the case I might need to look more into it.
> 
> Bigger event ring, more rings, faster handler, avoid irqoff time...
> 
> Thanks
> Mathias
> .
Some performance test tools such as fio or iometer can be used to reproduce
  this case, If tested with 4KB read or write. xHCI will generate a lost TRB
  events fast than software consume on a certain period of time. Once 
the interrupt is entered, software may handle more than 128 TRBs at a time.
While the software is processing, xHCI is still generating events. This may
has problems caused by the ERDP update mechanism. If update software
  record temp variable when handles 128 TRB events, event ring full will not
  happen any more even though fio test with 4KB read or write.

Thanks
Weitao Wang
> 

Re: [PATCH] USB: Fix xhci ERDP update issue

[Peter Chen]

On Mon, Mar 14, 2022 at 10:34 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, Mar 14, 2022 at 03:25:23PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
> > On some situations, software handles TRB events slower than adding TRBs,
> > xhci_irq will not exit until all events are handled. If xhci_irq just
> > handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records
> > in xhci irq is equal to driver current dequeue pointer. It will cause driver
> > not update ERDP and software dequeue pointer lost sync with ERDP. On the
> > next xhci_irq, the event ring is full but driver will not update ERDP as
> > software dequeue pointer is equal to ERDP.

At the current driver, the ERDP is updated at most 128 TRBs, how is
the above condition
triggered?

Peter



> >
> > [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> > [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
> > inflight: CMD OUT
> > [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00
> > 00 08 00
> > [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not
> > responding to stop endpoint command.
> > [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> > [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding,
> > assume dead
> > [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
> > inflight: CMD
> > [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00
> > 00 08 00
> > [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
> > inflight: CMD
> >
> > Fixed this issue by update software record temp variable when handles 128
> > TRB events.
> >
> > Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
> > ---
> >  drivers/usb/host/xhci-ring.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
> > index d0b6806..f970799 100644
> > --- a/drivers/usb/host/xhci-ring.c
> > +++ b/drivers/usb/host/xhci-ring.c
> > @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
> >                 if (event_loop++ < TRBS_PER_SEGMENT / 2)
> >                         continue;
> >                 xhci_update_erst_dequeue(xhci, event_ring_deq);
> > +               event_ring_deq = xhci->event_ring->dequeue;
> >
> >                 /* ring is half-full, force isoc trbs to interrupt more
> > often */
> >                 if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
> > --
> > 2.7.4
>
> Hi,
>
> This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
> a patch that has triggered this response.  He used to manually respond
> to these common problems, but in order to save his sanity (he kept
> writing the same thing over and over, yet to different people), I was
> created.  Hopefully you will not take offence and will fix the problem
> in your patch and resubmit it so that it can be accepted into the Linux
> kernel tree.
>
> You are receiving this message because of the following common error(s)
> as indicated below:
>
> - Your patch is malformed (tabs converted to spaces, linewrapped, etc.)
>   and can not be applied.  Please read the file,
>   Documentation/email-clients.txt in order to fix this.
>
>
> If you wish to discuss this problem further, or you have questions about
> how to resolve this issue, please feel free to respond to this email and
> Greg will reply once he has dug out from the pending patches received
> from other developers.
>
> thanks,
>
> greg k-h's patch email bot

Re: [PATCH] USB: Fix xhci ERDP update issue

[Mathias Nyman]

On 16.3.2022 13.57, Peter Chen wrote:
> On Mon, Mar 14, 2022 at 10:34 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>>
>> On Mon, Mar 14, 2022 at 03:25:23PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
>>> On some situations, software handles TRB events slower than adding TRBs,
>>> xhci_irq will not exit until all events are handled. If xhci_irq just
>>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records
>>> in xhci irq is equal to driver current dequeue pointer. It will cause driver
>>> not update ERDP and software dequeue pointer lost sync with ERDP. On the
>>> next xhci_irq, the event ring is full but driver will not update ERDP as
>>> software dequeue pointer is equal to ERDP.
> 
> At the current driver, the ERDP is updated at most 128 TRBs, how is
> the above condition
> triggered?
> 
> Peter
> 

Before, and during _one_ interrupt handling xHC hardware writes exactly 256 events
to event ring. ring buffer size is 256 so buffer position 0 and 256 point
to the same place.

Interrupt handler stores software dequeue in a local variable "event_ring_deq".
Handler start handling events, it updates software dequeue, but not local variable.
After 128 events handler updates hardware ERDP.

So at event 128 we got:
Hardware ERDP = 128
software dequeue = 128
event_ring_deq = 0

Handler continue handling events, at event 256 try to update HW ERDP again, but fail due
to this condition in update_erst_dequeue():
      if (event_ring_deq != xhci->event_ring->dequeue)

This fails because event_ring_deq is still 0, and software deq is 256,
pointing to the same place in the event ring.

So at the end of the interrupt handler we have:
HW ERDP = 128
software dequeue = 256 (same as 0)

So in this specific case we fail to update ERDP correctly

-Mathias

Re: [PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On 2022/3/16 19:57, Peter Chen wrote:
> On Mon, Mar 14, 2022 at 10:34 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>>
>> On Mon, Mar 14, 2022 at 03:25:23PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
>>> On some situations, software handles TRB events slower than adding TRBs,
>>> xhci_irq will not exit until all events are handled. If xhci_irq just
>>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records
>>> in xhci irq is equal to driver current dequeue pointer. It will cause driver
>>> not update ERDP and software dequeue pointer lost sync with ERDP. On the
>>> next xhci_irq, the event ring is full but driver will not update ERDP as
>>> software dequeue pointer is equal to ERDP.
> 
> At the current driver, the ERDP is updated at most 128 TRBs, how is
> the above condition
> triggered?
> 
> Peter

If the number of TRB events to be processed in a given interrupt is 256.
ERDP is updated after only the first 128 TRB evnets are processed.
It will not be updated when another 128 TRB evnets are processed as 
event_ring_deq= "xhci->event_ring->dequeue", which will cause the 
software-recorded dequeue pointer is out of sync with ERDP on interrupt 
exit.

Weitao Wang
> 
> 
>>>
>>> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
>>> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
>>> inflight: CMD OUT
>>> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00
>>> 00 08 00
>>> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not
>>> responding to stop endpoint command.
>>> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
>>> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding,
>>> assume dead
>>> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
>>> inflight: CMD
>>> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00
>>> 00 08 00
>>> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
>>> inflight: CMD
>>>
>>> Fixed this issue by update software record temp variable when handles 128
>>> TRB events.
>>>
>>> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
>>> ---
>>>   drivers/usb/host/xhci-ring.c | 1 +
>>>   1 file changed, 1 insertion(+)
>>>
>>> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
>>> index d0b6806..f970799 100644
>>> --- a/drivers/usb/host/xhci-ring.c
>>> +++ b/drivers/usb/host/xhci-ring.c
>>> @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
>>>                  if (event_loop++ < TRBS_PER_SEGMENT / 2)
>>>                          continue;
>>>                  xhci_update_erst_dequeue(xhci, event_ring_deq);
>>> +               event_ring_deq = xhci->event_ring->dequeue;
>>>
>>>                  /* ring is half-full, force isoc trbs to interrupt more
>>> often */
>>>                  if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
>>> --
>>> 2.7.4
>>
>> Hi,
>>
>> This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
>> a patch that has triggered this response.  He used to manually respond
>> to these common problems, but in order to save his sanity (he kept
>> writing the same thing over and over, yet to different people), I was
>> created.  Hopefully you will not take offence and will fix the problem
>> in your patch and resubmit it so that it can be accepted into the Linux
>> kernel tree.
>>
>> You are receiving this message because of the following common error(s)
>> as indicated below:
>>
>> - Your patch is malformed (tabs converted to spaces, linewrapped, etc.)
>>    and can not be applied.  Please read the file,
>>    Documentation/email-clients.txt in order to fix this.
>>
>>
>> If you wish to discuss this problem further, or you have questions about
>> how to resolve this issue, please feel free to respond to this email and
>> Greg will reply once he has dug out from the pending patches received
>> from other developers.
>>
>> thanks,
>>
>> greg k-h's patch email bot
> .

Re: [PATCH] USB: Fix xhci ERDP update issue

[Mathias Nyman]

On 15.3.2022 14.44, WeitaoWang-oc@zhaoxin.com wrote:
> On 2022/3/15 下午4:08, Mathias Nyman wrote:
>> On 14.3.2022 9.25, WeitaoWang-oc@zhaoxin.com wrote:
>>> On some situations, software handles TRB events slower than adding TRBs,
>>> xhci_irq will not exit until all events are handled. If xhci_irq just
>>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records in xhci irq is equal to driver current dequeue pointer. It will cause driver not update ERDP and software dequeue pointer lost sync with ERDP. On the next xhci_irq, the event ring is full but driver will not update ERDP as software dequeue pointer is equal to ERDP.
>>>
>>> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
>>> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT
>>> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00 00 08 00
>>> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not responding to stop endpoint command.
>>> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
>>> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding, assume dead
>>> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD
>>> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00 00 08 00
>>> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD
>>>
>>> Fixed this issue by update software record temp variable when handles 128 TRB events.>
>>> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
>>
>> Thanks
>>
>> Solution itself looks good but patch has some minor format issue:
>>
>>
>> It would also be interesting to know if the full event ring was triggered in a real
>> life usecase?
>> If that is the case I might need to look more into it.
>>
>> Bigger event ring, more rings, faster handler, avoid irqoff time...
>>
>> Thanks
>> Mathias
>> .
> Some performance test tools such as fio or iometer can be used to reproduce
>  this case, If tested with 4KB read or write. xHCI will generate a lost TRB
>  events fast than software consume on a certain period of time. Once the interrupt is entered, software may handle more than 128 TRBs at a time.
> While the software is processing, xHCI is still generating events. This may
> has problems caused by the ERDP update mechanism. If update software
>  record temp variable when handles 128 TRB events, event ring full will not
>  happen any more even though fio test with 4KB read or write.
> 
> Thanks
> Weitao Wang
>>

Thanks for the clarification.

Could you resubmit this after fixing the minor patch format issue?
Also make sure patch passes checkpatch test

Thanks
-Mathias

Re: [PATCH] USB: Fix xhci ERDP update issue

[Peter Chen]

On Wed, Mar 16, 2022 at 8:43 PM Mathias Nyman
<mathias.nyman@linux.intel.com> wrote:
>
> On 16.3.2022 13.57, Peter Chen wrote:
> > On Mon, Mar 14, 2022 at 10:34 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> >>
> >> On Mon, Mar 14, 2022 at 03:25:23PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
> >>> On some situations, software handles TRB events slower than adding TRBs,
> >>> xhci_irq will not exit until all events are handled. If xhci_irq just
> >>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records
> >>> in xhci irq is equal to driver current dequeue pointer. It will cause driver
> >>> not update ERDP and software dequeue pointer lost sync with ERDP. On the
> >>> next xhci_irq, the event ring is full but driver will not update ERDP as
> >>> software dequeue pointer is equal to ERDP.
> >
> > At the current driver, the ERDP is updated at most 128 TRBs, how is
> > the above condition
> > triggered?
> >
> > Peter
> >
>
> Before, and during _one_ interrupt handling xHC hardware writes exactly 256 events
> to event ring. ring buffer size is 256 so buffer position 0 and 256 point
> to the same place.
>
> Interrupt handler stores software dequeue in a local variable "event_ring_deq".
> Handler start handling events, it updates software dequeue, but not local variable.
> After 128 events handler updates hardware ERDP.
>
> So at event 128 we got:
> Hardware ERDP = 128
> software dequeue = 128
> event_ring_deq = 0
>
> Handler continue handling events, at event 256 try to update HW ERDP again, but fail due
> to this condition in update_erst_dequeue():
>       if (event_ring_deq != xhci->event_ring->dequeue)
>
> This fails because event_ring_deq is still 0, and software deq is 256,
> pointing to the same place in the event ring.
>
> So at the end of the interrupt handler we have:
> HW ERDP = 128
> software dequeue = 256 (same as 0)
>
> So in this specific case we fail to update ERDP correctly

Cleared, thanks.

Peter

Re: [PATCH] USB: Fix xhci ERDP update issue

[Peter Chen]

On Thu, Mar 17, 2022 at 1:30 AM WeitaoWang-oc@zhaoxin.com
<WeitaoWang-oc@zhaoxin.com> wrote:
>
> On some situations, software handles TRB events slower than adding TRBs,
> xhci_irq will not exit until all events are handled. If xhci_irq just
> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver
> records in xhci irq is equal to driver current dequeue pointer. It will
> cause driver not update ERDP and software dequeue pointer lost sync with
> ERDP. On the next xhci_irq, the event ring is full but driver will not
> update ERDP as software dequeue pointer is equal to ERDP.
>
> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
> inflight: CMD OUT
> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78
> 00 00 08 00
> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host
> not responding to stop endpoint command.
> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not
> responding, assume dead
> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
> inflight: CMD
> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8
> 00 00 08 00
> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
> inflight: CMD
>
> Fixed this issue by update software record temp variable when handles
> 128 TRB events.
>
> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>

Reviewed-by: Peter Chen <peter.chen@kernel.org>

> ---
>   drivers/usb/host/xhci-ring.c | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
> index d0b6806..f970799 100644
> --- a/drivers/usb/host/xhci-ring.c
> +++ b/drivers/usb/host/xhci-ring.c
> @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
>                  if (event_loop++ < TRBS_PER_SEGMENT / 2)
>                          continue;
>                  xhci_update_erst_dequeue(xhci, event_ring_deq);
> +               event_ring_deq = xhci->event_ring->dequeue;
>
>                  /* ring is half-full, force isoc trbs to interrupt more
> often */
>                  if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
> --
> 2.7.4

Re: [PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On 2022/3/16 21:20, Mathias Nyman wrote:
> On 15.3.2022 14.44, WeitaoWang-oc@zhaoxin.com wrote:
>> On 2022/3/15 下午4:08, Mathias Nyman wrote:
>>> On 14.3.2022 9.25, WeitaoWang-oc@zhaoxin.com wrote:
>>>> On some situations, software handles TRB events slower than adding TRBs,
>>>> xhci_irq will not exit until all events are handled. If xhci_irq just
>>>> handles 256 TRBs and exit, the temp variable(event_ring_deq) driver records in xhci irq is equal to driver current dequeue pointer. It will cause driver not update ERDP and software dequeue pointer lost sync with ERDP. On the next xhci_irq, the event ring is full but driver will not update ERDP as software dequeue pointer is equal to ERDP.
>>>>
>>>> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
>>>> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT
>>>> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00 00 08 00
>>>> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not responding to stop endpoint command.
>>>> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
>>>> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding, assume dead
>>>> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD
>>>> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00 00 08 00
>>>> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD
>>>>
>>>> Fixed this issue by update software record temp variable when handles 128 TRB events.>
>>>> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
>>>
>>> Thanks
>>>
>>> Solution itself looks good but patch has some minor format issue:
>>>
>>>
>>> It would also be interesting to know if the full event ring was triggered in a real
>>> life usecase?
>>> If that is the case I might need to look more into it.
>>>
>>> Bigger event ring, more rings, faster handler, avoid irqoff time...
>>>
>>> Thanks
>>> Mathias
>>> .
>> Some performance test tools such as fio or iometer can be used to reproduce
>>   this case, If tested with 4KB read or write. xHCI will generate a lost TRB
>>   events fast than software consume on a certain period of time. Once the interrupt is entered, software may handle more than 128 TRBs at a time.
>> While the software is processing, xHCI is still generating events. This may
>> has problems caused by the ERDP update mechanism. If update software
>>   record temp variable when handles 128 TRB events, event ring full will not
>>   happen any more even though fio test with 4KB read or write.
>>
>> Thanks
>> Weitao Wang
>>>
> 
> Thanks for the clarification.
> 
> Could you resubmit this after fixing the minor patch format issue?
> Also make sure patch passes checkpatch test
> 
> Thanks
> -Mathias

All right, I'll resubmit this patch.Thanks for your help.

Weitao Wang
> .

Re: [PATCH] USB: Fix xhci ERDP update issue

[Greg KH]

On Fri, Mar 18, 2022 at 08:17:50PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
> On some situations, software handles TRB events slower than adding TRBs.
> If the number of TRB events to be processed in a given interrupt is 256.
> The local variable "event_ring_deq" that records in interrupt handler
> is equal to software_dequeue. It will cause driver not update ERDP,then
> software dequeue pointer is out of sync with ERDP on interrupt exit.
> On the next interrupt, the event ring may full but driver will not
> update ERDP as software_dequeue is equal to ERDP.
> 
> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
> inflight: CMD OUT
> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78
> 00 00 08 00
> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host
> not responding to stop endpoint command.
> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not
> responding, assume dead
> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
> inflight: CMD
> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8
> 00 00 08 00
> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
> inflight: CMD
> 
> To avoid this issue by update software record local variable when
> handles 128 TRB events.
> 
> Fixes: dc0ffbea5729 ("usb: host: xhci: update event ring dequeue pointer on purpose")
> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
> ---
>  drivers/usb/host/xhci-ring.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
> index d0b6806275e0..f9707997969d 100644
> --- a/drivers/usb/host/xhci-ring.c
> +++ b/drivers/usb/host/xhci-ring.c
> @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
>  		if (event_loop++ < TRBS_PER_SEGMENT / 2)
>  			continue;
>  		xhci_update_erst_dequeue(xhci, event_ring_deq);
> +		event_ring_deq = xhci->event_ring->dequeue;
> 
>  		/* ring is half-full, force isoc trbs to interrupt more often */
>  		if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
> -- 
> 2.17.1

Hi,

This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
a patch that has triggered this response.  He used to manually respond
to these common problems, but in order to save his sanity (he kept
writing the same thing over and over, yet to different people), I was
created.  Hopefully you will not take offence and will fix the problem
in your patch and resubmit it so that it can be accepted into the Linux
kernel tree.

You are receiving this message because of the following common error(s)
as indicated below:

- This looks like a new version of a previously submitted patch, but you
  did not list below the --- line any changes from the previous version.
  Please read the section entitled "The canonical patch format" in the
  kernel file, Documentation/SubmittingPatches for what needs to be done
  here to properly describe this.

If you wish to discuss this problem further, or you have questions about
how to resolve this issue, please feel free to respond to this email and
Greg will reply once he has dug out from the pending patches received
from other developers.

thanks,

greg k-h's patch email bot

[PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On some situations, software handles TRB events slower than adding TRBs.
If the number of TRB events to be processed in a given interrupt is 256.
The local variable "event_ring_deq" that records in interrupt handler
is equal to software_dequeue. It will cause driver not update ERDP,then
software dequeue pointer is out of sync with ERDP on interrupt exit.
On the next interrupt, the event ring may full but driver will not
update ERDP as software_dequeue is equal to ERDP.

[  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
[  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
inflight: CMD OUT
[  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78
00 00 08 00
[  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host
not responding to stop endpoint command.
[  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
[  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not
responding, assume dead
[  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
inflight: CMD
[  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8
00 00 08 00
[  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
inflight: CMD

To avoid this issue by update software record local variable when
handles 128 TRB events.

Fixes: dc0ffbea5729 ("usb: host: xhci: update event ring dequeue pointer on purpose")
Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
---
  drivers/usb/host/xhci-ring.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index d0b6806275e0..f9707997969d 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
  		if (event_loop++ < TRBS_PER_SEGMENT / 2)
  			continue;
  		xhci_update_erst_dequeue(xhci, event_ring_deq);
+		event_ring_deq = xhci->event_ring->dequeue;

  		/* ring is half-full, force isoc trbs to interrupt more often */
  		if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
-- 
2.17.1

Re: [PATCH] USB: Fix xhci ERDP update issue

[Greg KH]

On Fri, Mar 18, 2022 at 03:32:14PM +0800, WeitaoWang-oc@zhaoxin.com wrote:
> On some situations, software handles TRB events slower than adding TRBs.
> If the number of TRB events to be processed in a given interrupt is 256.
> The local variable "event_ring_deq" that records in interrupt handler
> is equal to software_dequeue. It will cause driver not update ERDP,then
> software dequeue pointer is out of sync with ERDP on interrupt exit.
> On the next interrupt, the event ring may full but driver will not
> update ERDP as software_dequeue is equal to ERDP.
> 
> [  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
> [  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7
> inflight: CMD OUT
> [  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 00
> 00 08 00
> [  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host not
> responding to stop endpoint command.
> [  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
> [  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not responding,
> assume dead
> [  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6
> inflight: CMD
> [  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 00
> 00 08 00
> [  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5
> inflight: CMD
> 
> Fixed this issue by update software record local variable when handles 128
> TRB events.
> 
> Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
> ---
>  drivers/usb/host/xhci-ring.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
> index d0b6806..f970799 100644
> --- a/drivers/usb/host/xhci-ring.c
> +++ b/drivers/usb/host/xhci-ring.c
> @@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
>  		if (event_loop++ < TRBS_PER_SEGMENT / 2)
>  			continue;
>  		xhci_update_erst_dequeue(xhci, event_ring_deq);
> +		event_ring_deq = xhci->event_ring->dequeue;
> 
>  		/* ring is half-full, force isoc trbs to interrupt more often */
>  		if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
> -- 
> 2.7.4

What commit id does this change fix?

thanks,

greg k-h

[PATCH] USB: Fix xhci ERDP update issue

[WeitaoWang-oc]

On some situations, software handles TRB events slower than adding TRBs.
If the number of TRB events to be processed in a given interrupt is 256.
The local variable "event_ring_deq" that records in interrupt handler
is equal to software_dequeue. It will cause driver not update ERDP,then
software dequeue pointer is out of sync with ERDP on interrupt exit.
On the next interrupt, the event ring may full but driver will not
update ERDP as software_dequeue is equal to ERDP.

[  536.377115] xhci_hcd 0000:00:12.0: ERROR unknown event type 37
[  566.933173] sd 8:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 7 
inflight: CMD OUT
[  566.933181] sd 8:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 17 71 e6 78 
00 00 08 00
[  572.041186] xhci_hcd On some situataions,the0000:00:12.0: xHCI host 
not responding to stop endpoint command.
[  572.057193] xhci_hcd 0000:00:12.0: Host halt failed, -110
[  572.057196] xhci_hcd 0000:00:12.0: xHCI host controller not 
responding, assume dead
[  572.057236] sd 8:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 6 
inflight: CMD
[  572.057240] sd 8:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 38 eb cc d8 
00 00 08 00
[  572.057244] sd 8:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 5 
inflight: CMD

Fixed this issue by update software record local variable when handles 
128 TRB events.

Signed-off-by: Weitao Wang <WeitaoWang-oc@zhaoxin.com>
---
  drivers/usb/host/xhci-ring.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index d0b6806..f970799 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -3141,6 +3141,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd)
  		if (event_loop++ < TRBS_PER_SEGMENT / 2)
  			continue;
  		xhci_update_erst_dequeue(xhci, event_ring_deq);
+		event_ring_deq = xhci->event_ring->dequeue;

  		/* ring is half-full, force isoc trbs to interrupt more often */
  		if (xhci->isoc_bei_interval > AVOID_BEI_INTERVAL_MIN)
-- 
2.7.4