Re: USB network gadget / DWC3 issue

[Thinh Nguyen <Thinh.Nguyen@synopsys.com>]

Ferry Toth wrote:
> Op 07-04-2021 om 15:34 schreef Andy Shevchenko:
>> On Wed, Apr 7, 2021 at 3:24 AM Thinh Nguyen
>> <Thinh.Nguyen@synopsys.com> wrote:
>>> Thinh Nguyen wrote:
>> ...
>>
>>>> I took a look at the "bad" and "normal" tracepoints. There are a few
>>>> 1-second delays where the host tried to bring the device back and
>>>> resume from low power:
>>>>
>>>>       ksoftirqd/0-10      [000] d.s.   231.501808:
>>>> dwc3_gadget_ep_cmd: ep3in: cmd 'Update Transfer' [60007] params
>>>> 00000000 00000000 00000000 --> status: Successful
>>>>       ksoftirqd/0-10      [000] d.s.   231.501809: dwc3_readl: addr
>>>> 00000000d68ecd36 value 0000a610
>>>>       ksoftirqd/0-10      [000] d.s.   231.501810: dwc3_writel: addr
>>>> 00000000d68ecd36 value 0000a710
>>>>            <idle>-0       [000] d.h.   232.499418: dwc3_readl: addr
>>>> 00000000a15e0e35 value 00000034
>>>>            <idle>-0       [000] d.h.   232.499423: dwc3_readl: addr
>>>> 00000000bb67b585 value 00001000
>>>>            <idle>-0       [000] d.h.   232.499425: dwc3_writel: addr
>>>> 00000000bb67b585 value 80001000
>>>>            <idle>-0       [000] d.h.   232.499427: dwc3_writel: addr
>>>> 00000000a15e0e35 value 00000034
>>>>       irq/15-dwc3-476     [000] d...   232.499480: dwc3_event: event
>>>> (00000401): WakeUp [U0]
>>>>       irq/15-dwc3-476     [000] d...   232.499492: dwc3_event: event
>>>> (00000401): WakeUp [U0]
>>>>       irq/15-dwc3-476     [000] d...   232.499496: dwc3_event: event
>>>> (00006088): ep2out: Transfer In Progress [0] (SIm)
>>>>       irq/15-dwc3-476     [000] d...   232.499501:
>>>> dwc3_complete_trb: ep2out: trb 00000000c7ce524e (E179:D170) buf
>>>> 0000000008273540 size 1463 ctrl 00000818 (hlcS:sC:normal)
>>>>       irq/15-dwc3-476     [000] d...   232.499518:
>>>> dwc3_gadget_giveback: ep2out: req 0000000012e296cf length 73/1536
>>>> zsI ==> 0
>>>>       irq/15-dwc3-476     [000] d...   232.499562: dwc3_ep_queue:
>>>> ep2out: req 0000000012e296cf length 0/1536 zsI ==> -115
>>>>       irq/15-dwc3-476     [000] d...   232.499601: dwc3_prepare_trb:
>>>> ep2out: trb 000000008c083777 (E180:D170) buf 0000000002a7e9c0 size
>>>> 1536 ctrl 00000819 (HlcS:sC:normal)
>>>>
>>>>
>>>> Your device is operating in highspeed right? Try to turn off LPM from
>>>> host and see if that helps with the speed throttling issue. (If you're
>>>> using xHCI host, then set XHCI_HW_LPM_DISABLE). It may also help with
>>>> the connection issue you saw.
>>>>
>>>> It seems to be an issue from host, but I can't tell for sure unless we
>>>> have some USB traffic analyzer that shows what's going on. Have you
>>>> tried different hosts?
>>>>
>>> You can also disable LPM from the gadget side by setting
>>> dwc->dis_enblslpm_quirk.
>> Ferry, it can be done by adding a corresponding property to the
>> dwc3-pci.c for Intel Merrifield platform. I'll check also for my case
>> and perhaps I can collect some traces in my case later on when I have
>> more time for that.
>>
> Ok thanks all. Here is what I tried:
> 
> Another computer (Acer 720P brainwashed chromebook), I tried both full
> speed and high speed. Still throttling but less bad.
> 
> Then on desktop, with Edison kernel 5.12-rc5 as above + this patch:
> 
> diff --git a/drivers/usb/dwc3/dwc3-pci.c b/drivers/usb/dwc3/dwc3-pci.c
> 
> index 4c5c6972124a..a9268c085840 100644
> 
> --- a/drivers/usb/dwc3/dwc3-pci.c
> 
> +++ b/drivers/usb/dwc3/dwc3-pci.c
> 
> @@ -122,6 +122,7 @@ static const struct property_entry
> dwc3_pci_mrfld_properties[] = {
> 
> PROPERTY_ENTRY_STRING("linux,extcon-name", "mrfld_bcove_pwrsrc"),
> 
> PROPERTY_ENTRY_BOOL("snps,dis_u3_susphy_quirk"),
> 
> PROPERTY_ENTRY_BOOL("snps,dis_u2_susphy_quirk"),
> 
> + PROPERTY_ENTRY_BOOL("snps,dis_enblslpm_quirk"),
> 
> PROPERTY_ENTRY_BOOL("linux,sysdev_is_parent"),
> 
> {}
> 
> };
> 
> This fixes the throttling but reveals I had actually at least 2 bugs:
> 
> 1) throttling due to LPM, this seems solved now, thanks to much!

Now that we can confirm the speed throttling is related to LPM. We can
try to experiment further. (IMO, LPM is an important feature and
totally disabling LPM seems like using a sledgehammer to crack a nut)

I suspect that your phy/HW has a higher low power exit latency. I don't
think you provided any HIRD threshold property in your setup right? So
by default, dwc3 sets the base line BESL value to 1 (or 150us). Unless
you know what your phy/HW is capable of, try to test and increase the
recommended BESL value. The range can be from 0 to 15 where 0 is 150us
and 15 is 10ms. Maybe try 6 (i.e. 1ms).

diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
index 60e850a395a2..423533df8927 100644
--- a/drivers/usb/dwc3/gadget.c
+++ b/drivers/usb/dwc3/gadget.c
@@ -2895,7 +2895,7 @@ static void dwc3_gadget_config_params(struct usb_gadget *g,
                 * recommended BESL baseline to 1 and clamp the BESL deep to be
                 * within 2 to 15.
                 */
-               params->besl_baseline = 1;
+               params->besl_baseline = 6;
                if (dwc->is_utmi_l1_suspend)
                        params->besl_deep =
                                clamp_t(u8, dwc->hird_threshold, 2, 15);



> 
> 2) a problem with usb plug detection
> 
> When I unplug/replug the gadget cable I need to do that at least another
> time before gadget is detected. So unplug/replug/unplug/replug seems to
> work.
> 
> Also this platform has a HW switch to select host/device mode, with
> separate connectors for host and device.
> 
> When I flip the switch to host it immediately changes to host.
> 
> Flipping to device leaves the LEDs on my connected usb hub on, so it's
> still powered (but not operational).
> 
> Flipping fast host/device (within 1/2 sec) hub LEDs turns off. But I
> still need to additionally unplug/replug the gadget cable to get that to
> work.
> 

The connection issue can come from different things. Please narrow it down
and make sure that you don't use any defective cable or bad hub. Even then,
it's difficult to determine whose fault it is from just the dmesg and driver
logs alone without looking at the USB traffic at the packet level.

Btw, is your setup DRD? If you're switching mode, then I know that dwc3 right
now doesn't implement mode switching correctly.

You can see the discussion we have here:
https://lore.kernel.org/linux-usb/20210108015115.27920-1-john.stultz@linaro.org/T/#t

BR,
Thinh