[BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Sebastian von Ohr]

[-- Attachment #1: Type: text/plain, Size: 2157 bytes --]

I'm running an Intel Apollo Lake SoM (Celeron N3350E) which I want to use as 
a USB gadget with the functionFS gadget driver. I have created two bulk 
endpoints for sending and receiving data. The hardware and cabling is only USB 
2.0 capable. In one test case the receive side of the SoM is slowed down 
deliberately (200ms sleep between reads) while the host PC tries to send as 
fast as possible. This setup leads to send timeouts on every second 
transmission on the host PC.

I believe this is an issue with the USB 2.0 LPM feature, more specifically 
hardware LPM done by the host USB controller. I have tested different USB 
descriptors and the issue is gone when removing the USB_LPM_SUPPORT flag from 
the USB 2.0 extension descriptor (actually removing only USB_BESL_SUPPORT seems 
to suffice). Also the issue occurs only on some newer PCs and adding a hub 
(doesn't matter if 2.0 or 3.0 capable) makes the issue go away. I can 
reproduce this issue with a Windows 10 (1909) host running an Intel B360 
chipset. I use libusb v1.0.24 with the WinUSB driver on the host side to send 
data on the bulk endpoint.

See the attached dwc3 trace and registers. It was created using the current 
5.12.1 kernel version. It shows multiple WakeUp [U0] events in short succession 
but never any event showing different link states than U0. The host is doing 8 
transmissions of 16 bytes to the device, but the device only receives 4 of 
these transmissions. The first transmissions always succeeds while the next one 
will timeout on the host. I believe this is because the device is currently not 
ready to receive new data. But instead of sending the data when the delay on 
the receive side is over the request never finishes and times out after 1 
second (or even longer when I increase the timeout value).

Is the USB 2.0 LPM extension even supposed to work with the dwc3 controller? I 
can work around this issue currently by downgrading the device to USB 2.0 only 
(setting bcdUSB to 0x0200). But I believe USB 3.0 capable device must support 
LPM, so this issue might come up again when having USB 3.0 capable hardware.

[-- Attachment #2: dwc3_trace.tar.gz --]
[-- Type: application/x-gzip, Size: 15279 bytes --]

Re: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 3709 bytes --]


Hi,

(before anything, thanks for actually following the steps for bug
reporting. Much, much appreciated)

Sebastian von Ohr <vonohr@smaract.com> writes:

> I'm running an Intel Apollo Lake SoM (Celeron N3350E) which I want to use as 
> a USB gadget with the functionFS gadget driver. I have created two bulk 
> endpoints for sending and receiving data. The hardware and cabling is only USB 
> 2.0 capable. In one test case the receive side of the SoM is slowed down 
> deliberately (200ms sleep between reads) while the host PC tries to send as 
> fast as possible. This setup leads to send timeouts on every second 
> transmission on the host PC.
>
> I believe this is an issue with the USB 2.0 LPM feature, more specifically 
> hardware LPM done by the host USB controller. I have tested different USB 

It's like the host is trying to go down to lower LPM states every 100ms:

     irq/13-dwc3-236     [000] d..1    71.363262: dwc3_event: event (00000401): WakeUp [U0]
     irq/13-dwc3-236     [000] d..1    71.363315: dwc3_event: event (00000401): WakeUp [U0]
     irq/13-dwc3-236     [000] d..1    71.363423: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)

> descriptors and the issue is gone when removing the USB_LPM_SUPPORT flag from 
> the USB 2.0 extension descriptor (actually removing only USB_BESL_SUPPORT seems 
> to suffice). Also the issue occurs only on some newer PCs and adding a hub 

it could be that LPM is disabled on older xHCI revisions. Mathias,
anything you can add here?

> (doesn't matter if 2.0 or 3.0 capable) makes the issue go away. I can 
> reproduce this issue with a Windows 10 (1909) host running an Intel B360 
> chipset. I use libusb v1.0.24 with the WinUSB driver on the host side to send 
> data on the bulk endpoint.

could you share a dump of your descriptors? It could be that the wake-up
latencies are incorrect which tricks the host into trying to go down to
lower LPM states too frequently.

> See the attached dwc3 trace and registers. It was created using the current 
> 5.12.1 kernel version. It shows multiple WakeUp [U0] events in short succession 
> but never any event showing different link states than U0. The host is doing 8 
> transmissions of 16 bytes to the device, but the device only receives 4 of 
> these transmissions. The first transmissions always succeeds while the next one 
> will timeout on the host. I believe this is because the device is currently not 
> ready to receive new data. But instead of sending the data when the delay on 
> the receive side is over the request never finishes and times out after 1 
> second (or even longer when I increase the timeout value).
>
> Is the USB 2.0 LPM extension even supposed to work with the dwc3 controller? I 

yes, it should be supported :-)

> can work around this issue currently by downgrading the device to USB 2.0 only 
> (setting bcdUSB to 0x0200). But I believe USB 3.0 capable device must support 
> LPM, so this issue might come up again when having USB 3.0 capable hardware.

correct, LPM is mandatory for USB3.0+. From the traces, we don't see any
errors, though. Everything looks fine.

Mathias, I have an old memory from my time at Intel when we discussed
this very fact. I can't remember what was the conclusion, but is there
anything the peripheral can do to tell the host to *not* enter U1/U2 so
frequently? If my memory doesn't fail me, I don't think there is
anything at all for the peripheral to do, right? Other than not
supporting U1/U2, which means STALLing the relevat SetFeature
requests. Do you know if that would still be a certifiable solution?

cheers

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

RE: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Sebastian von Ohr]

Thanks for your super-fast reply!

> From: Felipe Balbi [mailto:balbi@kernel.org]
> Sent: Monday, May 3, 2021 3:52 PM
> It's like the host is trying to go down to lower LPM states every 100ms:
> 
>      irq/13-dwc3-236     [000] d..1    71.363262: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1    71.363315: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1    71.363423: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)

How do you arrive at 100ms? These wakeups are around 50 microseconds apart.

> > Is the USB 2.0 LPM extension even supposed to work with the dwc3 controller? I
> 
> yes, it should be supported :-)

I've spent some time looking at the code and I don't understand how the device 
is supposed to wake up the host once it's able to receive new data again. The 
closest thing I could find is the __dwc3_gadget_wakeup function which is called 
by dwc3_send_gadget_ep_cmd. But I don't understand the condition there. The 
wakeup is only executed for DWC3_DEPCMD_STARTTRANSFER, but I understand bulk 
transfers are only started once the endpoint is configured and then only 
DWC3_DEPCMD_UPDATETRANSFER is used.

> could you share a dump of your descriptors? It could be that the wake-up
> latencies are incorrect which tricks the host into trying to go down to
> lower LPM states too frequently.

Bus 001 Device 003: ID 3386:0001  
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x3386 
  idProduct          0x0001 
  bcdDevice            5.12
  iManufacturer           1 SmarAct
  iProduct                2 SmarAct Sensor
  iSerial                 3 PSC-00000038
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          4 SmarAct Config 1
    bmAttributes         0xc0
      Self Powered
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              5 SmarAct Sensor
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           22
  bNumDeviceCaps          2
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x0000010e
      Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000f
      Device can operate at Low Speed (1Mbps)
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat         511 micro seconds
Device Status:     0x0001
  Self Powered

RE: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 5141 bytes --]

Sebastian von Ohr <vonohr@smaract.com> writes:

> Thanks for your super-fast reply!
>
>> From: Felipe Balbi [mailto:balbi@kernel.org]
>> Sent: Monday, May 3, 2021 3:52 PM
>> It's like the host is trying to go down to lower LPM states every 100ms:
>> 
>>      irq/13-dwc3-236     [000] d..1    71.363262: dwc3_event: event (00000401): WakeUp [U0]
>>      irq/13-dwc3-236     [000] d..1    71.363315: dwc3_event: event (00000401): WakeUp [U0]
>>      irq/13-dwc3-236     [000] d..1    71.363423: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)
>
> How do you arrive at 100ms? These wakeups are around 50 microseconds apart.

heh, I should've written 100uS :-) but yeah, they're 50uS apart.

>> > Is the USB 2.0 LPM extension even supposed to work with the dwc3 controller? I
>> 
>> yes, it should be supported :-)
>
> I've spent some time looking at the code and I don't understand how the device 
> is supposed to wake up the host once it's able to receive new data again. The 

For U1/U2 it's mostly handled by the HW itself. The only thing we do is
set the appropriate bits for the relevant SetFeature requests, see ep0.c.

> closest thing I could find is the __dwc3_gadget_wakeup function which is called 
> by dwc3_send_gadget_ep_cmd. But I don't understand the condition there. The 
> wakeup is only executed for DWC3_DEPCMD_STARTTRANSFER, but I understand bulk 
> transfers are only started once the endpoint is configured and then only 
> DWC3_DEPCMD_UPDATETRANSFER is used.

That's a "special note" on the databook :-)

>> could you share a dump of your descriptors? It could be that the wake-up
>> latencies are incorrect which tricks the host into trying to go down to
>> lower LPM states too frequently.
>
> Bus 001 Device 003: ID 3386:0001  
> Device Descriptor:
>   bLength                18
>   bDescriptorType         1
>   bcdUSB               2.10
>   bDeviceClass            0 (Defined at Interface level)
>   bDeviceSubClass         0 
>   bDeviceProtocol         0 
>   bMaxPacketSize0        64
>   idVendor           0x3386 
>   idProduct          0x0001 
>   bcdDevice            5.12
>   iManufacturer           1 SmarAct
>   iProduct                2 SmarAct Sensor
>   iSerial                 3 PSC-00000038
>   bNumConfigurations      1
>   Configuration Descriptor:
>     bLength                 9
>     bDescriptorType         2
>     wTotalLength           32
>     bNumInterfaces          1
>     bConfigurationValue     1
>     iConfiguration          4 SmarAct Config 1
>     bmAttributes         0xc0
>       Self Powered
>     MaxPower                2mA
>     Interface Descriptor:
>       bLength                 9
>       bDescriptorType         4
>       bInterfaceNumber        0
>       bAlternateSetting       0
>       bNumEndpoints           2
>       bInterfaceClass       255 Vendor Specific Class
>       bInterfaceSubClass      0 
>       bInterfaceProtocol      0 
>       iInterface              5 SmarAct Sensor
>       Endpoint Descriptor:
>         bLength                 7
>         bDescriptorType         5
>         bEndpointAddress     0x81  EP 1 IN
>         bmAttributes            2
>           Transfer Type            Bulk
>           Synch Type               None
>           Usage Type               Data
>         wMaxPacketSize     0x0200  1x 512 bytes
>         bInterval               0
>       Endpoint Descriptor:
>         bLength                 7
>         bDescriptorType         5
>         bEndpointAddress     0x01  EP 1 OUT
>         bmAttributes            2
>           Transfer Type            Bulk
>           Synch Type               None
>           Usage Type               Data
>         wMaxPacketSize     0x0200  1x 512 bytes
>         bInterval               1
> Binary Object Store Descriptor:
>   bLength                 5
>   bDescriptorType        15
>   wTotalLength           22
>   bNumDeviceCaps          2
>   USB 2.0 Extension Device Capability:
>     bLength                 7
>     bDescriptorType        16
>     bDevCapabilityType      2
>     bmAttributes   0x0000010e
>       Link Power Management (LPM) Supported
>   SuperSpeed USB Device Capability:
>     bLength                10
>     bDescriptorType        16
>     bDevCapabilityType      3
>     bmAttributes         0x00
>     wSpeedsSupported   0x000f
>       Device can operate at Low Speed (1Mbps)
>       Device can operate at Full Speed (12Mbps)
>       Device can operate at High Speed (480Mbps)
>       Device can operate at SuperSpeed (5Gbps)
>     bFunctionalitySupport   1
>       Lowest fully-functional device speed is Full Speed (12Mbps)
>     bU1DevExitLat          10 micro seconds

Hmm, this is the maximum allowed value

>     bU2DevExitLat         511 micro seconds

This is not. Can you try setting this to 0x7ff and see if the problem
goes away? It could be that your platform needs more time to
wakeup. Then you're going to have to characterize it to figure out how
much this value should be.

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

RE: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Sebastian von Ohr]

[-- Attachment #1: Type: text/plain, Size: 3312 bytes --]

> From: Felipe Balbi [mailto:balbi@kernel.org]

> For U1/U2 it's mostly handled by the HW itself. The only thing we do is
> set the appropriate bits for the relevant SetFeature requests, see ep0.c.

Is this also the case for USB 2.0 LPM? USB 3.0 U1/U2 transitions seem to be 
completely different from USB 2.0. The SetFeature functions in ep0.c only 
handle SuperSpeed and SuperSpeed-plus. My connection is only USB 2.0 high-speed.

> >     bU1DevExitLat          10 micro seconds
> 
> Hmm, this is the maximum allowed value
> 
> >     bU2DevExitLat         511 micro seconds
> 
> This is not. Can you try setting this to 0x7ff and see if the problem
> goes away? It could be that your platform needs more time to
> wakeup. Then you're going to have to characterize it to figure out how
> much this value should be.

I've changed it to 0x7ff but no difference. Also isn't his field for USB 3.0 only?

Meanwhile I've spent some more time looking at the driver code and enabled the 
link change interrupt. I've attached a new trace where we can actually see what 
transitions happen:

     irq/13-dwc3-236     [000] d..1   174.435986: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)
     irq/13-dwc3-236     [000] d..1   174.435988: dwc3_complete_trb: ep1out: trb 000000005384b162 (E2:D2) buf 000000007348a000 size 4080 ctrl 00000818 (hlcS:sC:normal)
     irq/13-dwc3-236     [000] d..1   174.435992: dwc3_gadget_giveback: ep1out: req 00000000f8e0932d length 16/4096 zsI ==> 0
     irq/13-dwc3-236     [000] d..1   174.436497: dwc3_event: event (00050301): Link Change [RX.Detect]
     irq/13-dwc3-236     [000] d..1   174.436544: dwc3_event: event (00020301): Link Change [U2]
        usb_recv-812     [000] d..2   174.636131: dwc3_ep_queue: ep1out: req 00000000f8e0932d length 0/4096 zsI ==> -115
        usb_recv-812     [000] d..2   174.636139: dwc3_prepare_trb: ep1out: trb 0000000085f38bb7 (E3:D2) buf 000000007348b000 size 4096 ctrl 00000819 (HlcS:sC:normal)
        usb_recv-812     [000] d..2   174.636147: dwc3_gadget_ep_cmd: ep1out: cmd 'Update Transfer' [20007] params 00000000 00000000 00000000 --> status: Successful
     irq/13-dwc3-236     [000] d..1   175.438282: dwc3_event: event (00000401): WakeUp [U0]
     irq/13-dwc3-236     [000] d..1   175.438353: dwc3_event: event (00000401): WakeUp [U0]
     irq/13-dwc3-236     [000] d..1   175.438357: dwc3_event: event (00000301): Link Change [U0]
          
We see that 500us after the last transaction the link state is changed to 
RX.Detect (in HS, means Early Suspend) and then shortly after to U2 (in HS, 
means SLEEP). I'm not sure what early suspend is supposed to be as I can't find 
in the USB spec (dwc3 specific?). Then a new receive request is queued, but the 
link state doesn't change even though the host has data to send. Data is only 
transferred way later after the host write times out and tries again.

For a test I've changed some conditions in the driver so that 
__dwc3_gadget_wakeup is also called on transfer updates and the link state 
change also happens when in U2. This change actually fixed my timeout issue. 
However, I'm not sure if this is actually the correct thing to do. I'm by far 
no USB expert and I don't have access to the dwc3 databook.

[-- Attachment #2: dwc3_trace_linkstate.tar.gz --]
[-- Type: application/x-gzip, Size: 16373 bytes --]

Re: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Mathias Nyman]

On 3.5.2021 16.52, Felipe Balbi wrote:
> 
> Hi,
> 
> (before anything, thanks for actually following the steps for bug
> reporting. Much, much appreciated)
> 
> Sebastian von Ohr <vonohr@smaract.com> writes:
> 
>> I'm running an Intel Apollo Lake SoM (Celeron N3350E) which I want to use as 
>> a USB gadget with the functionFS gadget driver. I have created two bulk 
>> endpoints for sending and receiving data. The hardware and cabling is only USB 
>> 2.0 capable. In one test case the receive side of the SoM is slowed down 
>> deliberately (200ms sleep between reads) while the host PC tries to send as 
>> fast as possible. This setup leads to send timeouts on every second 
>> transmission on the host PC.
>>
>> I believe this is an issue with the USB 2.0 LPM feature, more specifically 
>> hardware LPM done by the host USB controller. I have tested different USB 
> 
> It's like the host is trying to go down to lower LPM states every 100ms:
> 
>      irq/13-dwc3-236     [000] d..1    71.363262: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1    71.363315: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1    71.363423: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)
> 
>> descriptors and the issue is gone when removing the USB_LPM_SUPPORT flag from 
>> the USB 2.0 extension descriptor (actually removing only USB_BESL_SUPPORT seems 
>> to suffice). Also the issue occurs only on some newer PCs and adding a hub 

xhci driver enables USB2 LPM only if hardware LPM is supported, driver itself won't do
any L0 -> L1 -> L0 transition. 
It only sets a L1 timeout (inactivirty timer) after  which host sends LPM token.

This can be adjusted on host side in sysfs  file "usb2_lpm_l1_timeout" for thid device.

> 
> it could be that LPM is disabled on older xHCI revisions. Mathias,
> anything you can add here?

xhci driver will only use USB2 LPM if the host supports Hardware LPM capability (HLC bit),
this was optional before xHCI 1.1. 


>> (doesn't matter if 2.0 or 3.0 capable) makes the issue go away. I can 
>> reproduce this issue with a Windows 10 (1909) host running an Intel B360 
>> chipset. I use libusb v1.0.24 with the WinUSB driver on the host side to send 
>> data on the bulk endpoint.

xhci driver in linux only enables USB2 LPM for devices directly conncted to root port.
> 
> could you share a dump of your descriptors? It could be that the wake-up
> latencies are incorrect which tricks the host into trying to go down to
> lower LPM states too frequently.
> 
>> See the attached dwc3 trace and registers. It was created using the current 
>> 5.12.1 kernel version. It shows multiple WakeUp [U0] events in short succession 
>> but never any event showing different link states than U0. The host is doing 8 
>> transmissions of 16 bytes to the device, but the device only receives 4 of 
>> these transmissions. The first transmissions always succeeds while the next one 
>> will timeout on the host. I believe this is because the device is currently not 
>> ready to receive new data. But instead of sending the data when the delay on 
>> the receive side is over the request never finishes and times out after 1 
>> second (or even longer when I increase the timeout value).
>>
>> Is the USB 2.0 LPM extension even supposed to work with the dwc3 controller? I 
> 
> yes, it should be supported :-)
> 
>> can work around this issue currently by downgrading the device to USB 2.0 only 
>> (setting bcdUSB to 0x0200). But I believe USB 3.0 capable device must support 
>> LPM, so this issue might come up again when having USB 3.0 capable hardware.
> 
> correct, LPM is mandatory for USB3.0+. From the traces, we don't see any
> errors, though. Everything looks fine.
> 
> Mathias, I have an old memory from my time at Intel when we discussed
> this very fact. I can't remember what was the conclusion, but is there
> anything the peripheral can do to tell the host to *not* enter U1/U2 so
> frequently? If my memory doesn't fail me, I don't think there is
> anything at all for the peripheral to do, right? Other than not
> supporting U1/U2, which means STALLing the relevat SetFeature
> requests. Do you know if that would still be a certifiable solution?

STALL response to LPM should cause xHCI to disable hardware LPM for this device.

Also NYET response should be visible to xhci driver (L1S) fields in PORTPMSC.
In theory we could try to check this and decide to increase L1 timeout if there are 
frequent NYETs, but there is no driver support for this

Note: xhci specs 4.23.5.1.1 also state:

In the case of a High-speed Bulk OUT Endpoint that has returned a NYEThandshake for
an OUT transaction and then the Hardware LPM mechanism transitions the link to the L1 state,
the xHC shall not initiate a L1 Exit (i.e. wake
up the link) to do a PING transaction. The device is expected to initiate an L1
exit (Remote Wake) when it is ready to accept data.

Does this device support L1 exit (remote wake)?

-Mathias 

RE: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 3895 bytes --]


Hi,

Sebastian von Ohr <vonohr@smaract.com> writes:
>> From: Felipe Balbi [mailto:balbi@kernel.org]
>
>> For U1/U2 it's mostly handled by the HW itself. The only thing we do is
>> set the appropriate bits for the relevant SetFeature requests, see ep0.c.
>
> Is this also the case for USB 2.0 LPM? USB 3.0 U1/U2 transitions seem to be 
> completely different from USB 2.0. The SetFeature functions in ep0.c only 
> handle SuperSpeed and SuperSpeed-plus. My connection is only USB 2.0 high-speed.

should behave the same for USB2

>> >     bU1DevExitLat          10 micro seconds
>> 
>> Hmm, this is the maximum allowed value
>> 
>> >     bU2DevExitLat         511 micro seconds
>> 
>> This is not. Can you try setting this to 0x7ff and see if the problem
>> goes away? It could be that your platform needs more time to
>> wakeup. Then you're going to have to characterize it to figure out how
>> much this value should be.
>
> I've changed it to 0x7ff but no difference. Also isn't his field for USB 3.0 only?

It should be part of one of the BOS descriptors that describes LPM, I
don't recall which one in particular.

> Meanwhile I've spent some more time looking at the driver code and enabled the 
> link change interrupt. I've attached a new trace where we can actually see what 
> transitions happen:
>
>      irq/13-dwc3-236     [000] d..1   174.435986: dwc3_event: event (00006084): ep1out: Transfer In Progress [0] (SIm)
>      irq/13-dwc3-236     [000] d..1   174.435988: dwc3_complete_trb: ep1out: trb 000000005384b162 (E2:D2) buf 000000007348a000 size 4080 ctrl 00000818 (hlcS:sC:normal)
>      irq/13-dwc3-236     [000] d..1   174.435992: dwc3_gadget_giveback: ep1out: req 00000000f8e0932d length 16/4096 zsI ==> 0
>      irq/13-dwc3-236     [000] d..1   174.436497: dwc3_event: event (00050301): Link Change [RX.Detect]
>      irq/13-dwc3-236     [000] d..1   174.436544: dwc3_event: event (00020301): Link Change [U2]
>         usb_recv-812     [000] d..2   174.636131: dwc3_ep_queue: ep1out: req 00000000f8e0932d length 0/4096 zsI ==> -115
>         usb_recv-812     [000] d..2   174.636139: dwc3_prepare_trb: ep1out: trb 0000000085f38bb7 (E3:D2) buf 000000007348b000 size 4096 ctrl 00000819 (HlcS:sC:normal)
>         usb_recv-812     [000] d..2   174.636147: dwc3_gadget_ep_cmd: ep1out: cmd 'Update Transfer' [20007] params 00000000 00000000 00000000 --> status: Successful
>      irq/13-dwc3-236     [000] d..1   175.438282: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1   175.438353: dwc3_event: event (00000401): WakeUp [U0]
>      irq/13-dwc3-236     [000] d..1   175.438357: dwc3_event: event (00000301): Link Change [U0]
>           
> We see that 500us after the last transaction the link state is changed to 
> RX.Detect (in HS, means Early Suspend) and then shortly after to U2 (in HS, 
> means SLEEP). I'm not sure what early suspend is supposed to be as I can't find 
> in the USB spec (dwc3 specific?). Then a new receive request is queued, but the 

yes, that's dwc3-specific

> link state doesn't change even though the host has data to send. Data is only 

it could be we're missing a wakeup somewhere.

> transferred way later after the host write times out and tries again.
>
> For a test I've changed some conditions in the driver so that 
> __dwc3_gadget_wakeup is also called on transfer updates and the link state 
> change also happens when in U2. This change actually fixed my timeout issue. 
> However, I'm not sure if this is actually the correct thing to do. I'm by far 
> no USB expert and I don't have access to the dwc3 databook.

Right, AFAIR the databook was a bit unclear about this. It stated that
it was required only for Start Transfer, but I always had the same
doubt. No idea if the databook has been clarified since then.

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

RE: [BUG REPORT] usb: dwc3: Timeouts with USB 2.0 LPM active

[Sebastian von Ohr]

> From: Felipe Balbi [mailto:balbi@kernel.org]
> > For a test I've changed some conditions in the driver so that
> > __dwc3_gadget_wakeup is also called on transfer updates and the link state
> > change also happens when in U2. This change actually fixed my timeout issue.
> > However, I'm not sure if this is actually the correct thing to do. I'm by far
> > no USB expert and I don't have access to the dwc3 databook.
> 
> Right, AFAIR the databook was a bit unclear about this. It stated that
> it was required only for Start Transfer, but I always had the same
> doubt. No idea if the databook has been clarified since then.

Maybe somebody with access to the databook can clarify this? Since dwc3 defaults
to USB 3.0 speed and LPM enabled this should affect most USB gadgets users and 
I'd like to see this issue fixed. I'm still wondering why there have been no
previous reports for this bug (that I know of). Surely I'm not the only one
using a function FS usb gadget.