All of lore.kernel.org
 help / color / mirror / Atom feed
* High CPU load produced by USB (DW2)
@ 2018-02-07  9:48 Mirza Krak
       [not found] ` <CALw8SCXuOCpndakFBRbP7FpJKOSH+nzNStkszWD4XTXVSK1zyQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  2018-02-08 11:02 ` Minas Harutyunyan
  0 siblings, 2 replies; 20+ messages in thread
From: Mirza Krak @ 2018-02-07  9:48 UTC (permalink / raw)
  To: linux-usb
  Cc: gregkh, johnyoun-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

Hi.

I initially started a thread on a different mailing list [1], and you
can take look there for some additional background.

I am using a 4.14.15 stable kernel on a RK3288 SoC (FireFly RK3288
board) which uses the DW2 USB core. I have noticed that when
connecting a USB device there is unreasonable high CPU load produced
by the USB interrupts.

Here is one example with a MIDI keyboard connected on USB (not really
doing anything on it)

   79     2 root     SW       0   0%  19% [irq/45-dwc2_hso]
   78     2 root     SW       0   0%  11% [irq/45-ff540000]

Can also see that there is a "HIGH" interrupt count

root@esp5-d1:~# watch cat /proc/interrupts | grep usb
 45:   43681938          0          0          0     GIC-0  57 Level
  ff540000.usb, dwc2_hsotg:usb1
 46:          0          0          0          0     GIC-0  55 Level
  ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
 45:   43829473          0          0          0     GIC-0  57 Level
  ff540000.usb, dwc2_hsotg:usb1
 46:          0          0          0          0     GIC-0  55 Level
  ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
 45:   43972333          0          0          0     GIC-0  57 Level
  ff540000.usb, dwc2_hsotg:usb1
 46:          0          0          0          0     GIC-0  55 Level
  ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
 45:   44118769          0          0          0     GIC-0  57 Level
  ff540000.usb, dwc2_hsotg:usb1
 46:          0          0          0          0     GIC-0  55 Level
  ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
 45:   44267876          0          0          0     GIC-0  57 Level
  ff540000.usb, dwc2_hsotg:usb1
 46:          0          0          0          0     GIC-0  55 Level
  ff580000.usb, ff580000.usb, dwc2_hsotg:usb2

Again not really doing anything with the device, it is just connected.
I have also tested with server different devices with same result (BT
and WLAN).

The CPU load is exposed when I add "threadirqs" to kernel arguments,
otherwise it does not seem that it accounted for (not see in "top")
but the number of interrupts is still high.

Any thoughts on this? Please let me know if there is additional
information that I can provide to help debug this.


[1]. https://www.spinics.net/lists/linux-rt-users/msg18341.html

-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found] ` <CALw8SCXuOCpndakFBRbP7FpJKOSH+nzNStkszWD4XTXVSK1zyQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2018-02-07 14:27   ` gregkh
  2018-02-08  8:50     ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: gregkh @ 2018-02-07 14:27 UTC (permalink / raw)
  To: Mirza Krak
  Cc: linux-usb, johnyoun-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On Wed, Feb 07, 2018 at 10:48:57AM +0100, Mirza Krak wrote:
> Hi.
> 
> I initially started a thread on a different mailing list [1], and you
> can take look there for some additional background.
> 
> I am using a 4.14.15 stable kernel on a RK3288 SoC (FireFly RK3288
> board) which uses the DW2 USB core. I have noticed that when
> connecting a USB device there is unreasonable high CPU load produced
> by the USB interrupts.

That's a normal problem with that kind of hardware.  I've seen devices
with that type of USB core take a 30% cpu usage just when you plug a USB
keyboard into the device.

There's not much the kernel can do about horrible hardware, sorry, go
complain to the device manufacturer :(

> Here is one example with a MIDI keyboard connected on USB (not really
> doing anything on it)
> 
>    79     2 root     SW       0   0%  19% [irq/45-dwc2_hso]
>    78     2 root     SW       0   0%  11% [irq/45-ff540000]

It doesn't matter if anything is happening on the device, USB goes and
polls the device all the time, that's just the way that the protocol
works.  With hardware like this, that polling is done by the kernel
driver/CPU and does not have offload capability to the USB controller to
handle this type of housekeeping.  Because of that, you see tons of CPU
interrupts and a high load.

Again, not much we can do about that, sorry :(

good luck!

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-07 14:27   ` gregkh
@ 2018-02-08  8:50     ` Minas Harutyunyan
  0 siblings, 0 replies; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-08  8:50 UTC (permalink / raw)
  To: gregkh, Mirza Krak
  Cc: linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 2/7/2018 6:28 PM, gregkh wrote:
> On Wed, Feb 07, 2018 at 10:48:57AM +0100, Mirza Krak wrote:
>> Hi.
>>
>> I initially started a thread on a different mailing list [1], and you
>> can take look there for some additional background.
>>
>> I am using a 4.14.15 stable kernel on a RK3288 SoC (FireFly RK3288
>> board) which uses the DW2 USB core. I have noticed that when
>> connecting a USB device there is unreasonable high CPU load produced
>> by the USB interrupts.
> 
> That's a normal problem with that kind of hardware.  I've seen devices
> with that type of USB core take a 30% cpu usage just when you plug a USB
> keyboard into the device.
> 
> There's not much the kernel can do about horrible hardware, sorry, go
> complain to the device manufacturer :(
> 
>> Here is one example with a MIDI keyboard connected on USB (not really
>> doing anything on it)
>>
>>     79     2 root     SW       0   0%  19% [irq/45-dwc2_hso]
>>     78     2 root     SW       0   0%  11% [irq/45-ff540000]
> 
> It doesn't matter if anything is happening on the device, USB goes and
> polls the device all the time, that's just the way that the protocol
> works.  With hardware like this, that polling is done by the kernel
> driver/CPU and does not have offload capability to the USB controller to
> handle this type of housekeeping.  Because of that, you see tons of CPU
> interrupts and a high load.
> 
> Again, not much we can do about that, sorry :(
> 
> good luck!
> 
> greg k-h
> --
> To unsubscribe from this list: send the line "unsubscribe linux-usb" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  https://urldefense.proofpoint.com/v2/url?u=http-3A__vger.kernel.org_majordomo-2Dinfo.html&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=mnEzj_Q87pyYPBxV3CbMXuG55KNzxqhYf7tuz517xjg&s=eftxO7ffuUu2L87Mqwa6K8p7s1mpI__boRvzDI70HvY&e=
> 
dwc2 HW doesn't track (u)frames internally. SW forced to track (u)frames 
by unmasking SOF interrupts. This is why when any device with periodic 
EP connected to port require to unmask SOF's as result seen high CPU 
load. dwc2 SW dynamically mask/unmask SOF interrupts depend on using 
periodic EP by connected device.
There are no any polling in SW.

Lot of HW functionality delegated to dwc2 driver to reduce gate count in HW.
This "horrible" core very small and cheap.

good luck!

Minas
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-07  9:48 High CPU load produced by USB (DW2) Mirza Krak
       [not found] ` <CALw8SCXuOCpndakFBRbP7FpJKOSH+nzNStkszWD4XTXVSK1zyQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2018-02-08 11:02 ` Minas Harutyunyan
       [not found]   ` <410670D7E743164D87FA6160E7907A560113A67F86-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  1 sibling, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-08 11:02 UTC (permalink / raw)
  To: Mirza Krak, linux-usb
  Cc: gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

Hi Mirza,

On 2/7/2018 1:49 PM, Mirza Krak wrote:
> Hi.
> 
> I initially started a thread on a different mailing list [1], and you
> can take look there for some additional background.
> 
> I am using a 4.14.15 stable kernel on a RK3288 SoC (FireFly RK3288
> board) which uses the DW2 USB core. I have noticed that when
> connecting a USB device there is unreasonable high CPU load produced
> by the USB interrupts.
> 
> Here is one example with a MIDI keyboard connected on USB (not really
> doing anything on it)
> 
>     79     2 root     SW       0   0%  19% [irq/45-dwc2_hso]
>     78     2 root     SW       0   0%  11% [irq/45-ff540000]
> 
> Can also see that there is a "HIGH" interrupt count
> 
> root@esp5-d1:~# watch cat /proc/interrupts | grep usb
>   45:   43681938          0          0          0     GIC-0  57 Level
>    ff540000.usb, dwc2_hsotg:usb1
>   46:          0          0          0          0     GIC-0  55 Level
>    ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
>   45:   43829473          0          0          0     GIC-0  57 Level
>    ff540000.usb, dwc2_hsotg:usb1
>   46:          0          0          0          0     GIC-0  55 Level
>    ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
>   45:   43972333          0          0          0     GIC-0  57 Level
>    ff540000.usb, dwc2_hsotg:usb1
>   46:          0          0          0          0     GIC-0  55 Level
>    ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
>   45:   44118769          0          0          0     GIC-0  57 Level
>    ff540000.usb, dwc2_hsotg:usb1
>   46:          0          0          0          0     GIC-0  55 Level
>    ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
>   45:   44267876          0          0          0     GIC-0  57 Level
>    ff540000.usb, dwc2_hsotg:usb1
>   46:          0          0          0          0     GIC-0  55 Level
>    ff580000.usb, ff580000.usb, dwc2_hsotg:usb2
> 
> Again not really doing anything with the device, it is just connected.
> I have also tested with server different devices with same result (BT
> and WLAN).
> 
> The CPU load is exposed when I add "threadirqs" to kernel arguments,
> otherwise it does not seem that it accounted for (not see in "top")
> but the number of interrupts is still high.
> 
> Any thoughts on this? Please let me know if there is additional
> information that I can provide to help debug this.
> 
> 
> [1]. https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_linux-2Drt-2Dusers_msg18341.html&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=DnUeBv1jR8jz4npf8WClUsYhhcd0eYqW0aDc4l7V96o&s=6fpUB5epwB1otF9_hwacG1JIkWJaHN0duhAyZIaiMqM&e=
> 

Could you please provide additional info:
1. Does your core support descriptor DMA mode? (bits 31 and 30 of GHWCFG4).
2. Version of core (GSNPSID)?
3. Do you connect any HS HUB between dwc core port and keyboard? Is it 
available connection scenarios when Split traffic generated?
Thanks,
Minas
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]   ` <410670D7E743164D87FA6160E7907A560113A67F86-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-08 13:06     ` Mirza Krak
  2018-02-08 13:53       ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Mirza Krak @ 2018-02-08 13:06 UTC (permalink / raw)
  To: Minas Harutyunyan
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 8 February 2018 at 12:02, Minas Harutyunyan
<Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
> Hi Mirza,
>
> On 2/7/2018 1:49 PM, Mirza Krak wrote:

< snip >

>
> Could you please provide additional info:
> 1. Does your core support descriptor DMA mode? (bits 31 and 30 of GHWCFG4).
> 2. Version of core (GSNPSID)?

It does seem to support descriptor DMA mode, and content of both
GHWCFG4 and GSNPSID are posted below.

root@esp5-d1:~# dmesg | grep -E "GHWCFG4|GSNPSID"
[    1.109298] dwc2 ff540000.usb: GSNPSID @0xF0940040 : 0x4F54310A
[    1.109344] dwc2 ff540000.usb: GHWCFG4 @0xF0940050 : 0xC8004030
[    1.489016] dwc2 ff580000.usb: GSNPSID @0xF09C0040 : 0x4F54310A
[    1.489063] dwc2 ff580000.usb: GHWCFG4 @0xF09C0050 : 0xDBF04030

Full register dump at boot (no devices connected) can be found at [1].

> 3. Do you connect any HS HUB between dwc core port and keyboard? Is it
> available connection scenarios when Split traffic generated?

There is actually an HS HUB (FE1_QFP48) between dwc core and keyboard,
I was not aware of it but it is mounted on the board.


[1]. https://gist.github.com/mirzak/a6245dddd4670f3b908a9783f9db8b0d
-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-08 13:06     ` Mirza Krak
@ 2018-02-08 13:53       ` Minas Harutyunyan
       [not found]         ` <410670D7E743164D87FA6160E7907A560113A68287-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-08 13:53 UTC (permalink / raw)
  To: Mirza Krak, Minas Harutyunyan
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 2/8/2018 5:07 PM, Mirza Krak wrote:
> On 8 February 2018 at 12:02, Minas Harutyunyan
> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>> Hi Mirza,
>>
>> On 2/7/2018 1:49 PM, Mirza Krak wrote:
> 
> < snip >
> 
>>
>> Could you please provide additional info:
>> 1. Does your core support descriptor DMA mode? (bits 31 and 30 of GHWCFG4).
>> 2. Version of core (GSNPSID)?
> 
> It does seem to support descriptor DMA mode, and content of both
> GHWCFG4 and GSNPSID are posted below.
> 
> root@esp5-d1:~# dmesg | grep -E "GHWCFG4|GSNPSID"
> [    1.109298] dwc2 ff540000.usb: GSNPSID @0xF0940040 : 0x4F54310A
> [    1.109344] dwc2 ff540000.usb: GHWCFG4 @0xF0940050 : 0xC8004030
> [    1.489016] dwc2 ff580000.usb: GSNPSID @0xF09C0040 : 0x4F54310A
> [    1.489063] dwc2 ff580000.usb: GHWCFG4 @0xF09C0050 : 0xDBF04030
> 
> Full register dump at boot (no devices connected) can be found at [1].
> 
>> 3. Do you connect any HS HUB between dwc core port and keyboard? Is it
>> available connection scenarios when Split traffic generated?
> 
> There is actually an HS HUB (FE1_QFP48) between dwc core and keyboard,
> I was not aware of it but it is mounted on the board.
> 
> 
> [1]. https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_mirzak_a6245dddd4670f3b908a9783f9db8b0d&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=FuRmhDEOJlbUvuyApaxG_YmB514C3IOGSzs6Mq_fzC0&s=ELF0UP4IluOiL35JlnFjiltWZx-BcTIsTaPjaWb0RFk&e=
> 
Actually your core supported DDMA mode, moreover core allow SW to 
dynamically switch between BDMA and DDMA after core reset.
If dwc2 core work in DDMA mode then core can track (u)frames internally 
and unmasking SOF's not required, but Host in DDMA doesn't support Split 
transfers. This is why, in your case (FS/LS keyboard connected to HS 
HUB), core switched to BDMA and unmasking SOF interrupt to track (u)frames.

Thanks,
Minas
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]         ` <410670D7E743164D87FA6160E7907A560113A68287-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-14  8:57           ` Mirza Krak
  2018-02-14 12:07             ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Mirza Krak @ 2018-02-14  8:57 UTC (permalink / raw)
  To: Minas Harutyunyan
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 8 February 2018 at 14:53, Minas Harutyunyan
<Minas.Harutyunyan@synopsys.com> wrote:
> On 2/8/2018 5:07 PM, Mirza Krak wrote:
>> On 8 February 2018 at 12:02, Minas Harutyunyan
>> <Minas.Harutyunyan@synopsys.com> wrote:
>>> Hi Mirza,
>>>
>>> On 2/7/2018 1:49 PM, Mirza Krak wrote:
>>
>> < snip >
>>
>>>
>>> Could you please provide additional info:
>>> 1. Does your core support descriptor DMA mode? (bits 31 and 30 of GHWCFG4).
>>> 2. Version of core (GSNPSID)?
>>
>> It does seem to support descriptor DMA mode, and content of both
>> GHWCFG4 and GSNPSID are posted below.
>>
>> root@esp5-d1:~# dmesg | grep -E "GHWCFG4|GSNPSID"
>> [    1.109298] dwc2 ff540000.usb: GSNPSID @0xF0940040 : 0x4F54310A
>> [    1.109344] dwc2 ff540000.usb: GHWCFG4 @0xF0940050 : 0xC8004030
>> [    1.489016] dwc2 ff580000.usb: GSNPSID @0xF09C0040 : 0x4F54310A
>> [    1.489063] dwc2 ff580000.usb: GHWCFG4 @0xF09C0050 : 0xDBF04030
>>
>> Full register dump at boot (no devices connected) can be found at [1].
>>
>>> 3. Do you connect any HS HUB between dwc core port and keyboard? Is it
>>> available connection scenarios when Split traffic generated?
>>
>> There is actually an HS HUB (FE1_QFP48) between dwc core and keyboard,
>> I was not aware of it but it is mounted on the board.
>>
>>
>> [1]. https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_mirzak_a6245dddd4670f3b908a9783f9db8b0d&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=FuRmhDEOJlbUvuyApaxG_YmB514C3IOGSzs6Mq_fzC0&s=ELF0UP4IluOiL35JlnFjiltWZx-BcTIsTaPjaWb0RFk&e=
>>
> Actually your core supported DDMA mode, moreover core allow SW to
> dynamically switch between BDMA and DDMA after core reset.
> If dwc2 core work in DDMA mode then core can track (u)frames internally
> and unmasking SOF's not required, but Host in DDMA doesn't support Split
> transfers. This is why, in your case (FS/LS keyboard connected to HS
> HUB), core switched to BDMA and unmasking SOF interrupt to track (u)frames.

Thank you for your valuable respons.

It is indeed as you mentioned the combination of HS and FS/LS devices
that trigger the high CPU load. Running HS only devices does not
produce any noticeable load.

But I am confused by one thing, I have two different boards with the
same SoC and same version of the DWC2 IP. Both boards have an internal
USB hub for the external connectors, though it seems that they have
different types of HUB' s. And I only have problems with one of them.
The second board does not have a problem mixing HS and FS/LS devices.
Is it possible that the USB HUB has some kind of "offloading" for the
Split transfers?

Tinkerboard USB hub (works well when mixing HS and FS/LS devices)
----------------------------

$ lsusb -t
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 1, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 2, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 3, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 4, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 5, Class=Audio, Driver=, 480M
    |__ Port 1: Dev 2, If 255, Class=Human Interface Device, Driver=usbhid, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 3: Dev 4, If 0, Class=Audio, Driver=, 12M
        |__ Port 3: Dev 4, If 1, Class=Audio, Driver=, 12M
        |__ Port 4: Dev 3, If 0, Class=Human Interface Device,
Driver=usbhid, 12M

$ lsusb -d 05e3:0610 -v

Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0
  bDeviceProtocol         2 TT per port
  bMaxPacketSize0        64
  idVendor           0x05e3 Genesys Logic, Inc.
  idProduct          0x0610 4-port hub
  bcdDevice           32.98
  iManufacturer           0
  iProduct                1 USB2.0 Hub
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           41
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0
      bInterfaceProtocol      1 Single TT
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0
      bInterfaceProtocol      2 TT per port
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             4
  wHubCharacteristic 0x00e0
    Ganged power switching
    Ganged overcurrent protection
    TT think time 32 FS bits
    Port indicators
  bPwrOn2PwrGood       50 * 2 milli seconds
  bHubContrCurrent    100 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0100 power
   Port 2: 0000.0100 power
   Port 3: 0000.0103 power enable connect
   Port 4: 0000.0103 power enable connect
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0
  bDeviceProtocol         0 Full speed (or root) hub
  bMaxPacketSize0        64
  bNumConfigurations      1
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered


FireFly USB HUB (does not work well when mixing HS and FS/LS devices)
-----------------------

$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=, 480M
        |__ Port 3: Dev 4, If 0, Class=Audio, Driver=snd-usb-audio, 12M
        |__ Port 3: Dev 4, If 1, Class=Audio, Driver=snd-usb-audio, 12M

$ lsusb -d 1a40:0101 -v

Bus 001 Device 002: ID 1a40:0101 Terminus Technology Inc. Hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0
  bDeviceProtocol         2 TT per port
  bMaxPacketSize0        64
  idVendor           0x1a40 Terminus Technology Inc.
  idProduct          0x0101 Hub
  bcdDevice            1.00
  iManufacturer           0
  iProduct                1 USB 2.0 Hub [MTT]
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           41
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0
      bInterfaceProtocol      1 Single TT
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0
      bInterfaceProtocol      2 TT per port
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             4
  wHubCharacteristic 0x0080
    Ganged power switching
    Ganged overcurrent protection
    TT think time 8 FS bits
    Port indicators
  bPwrOn2PwrGood       50 * 2 milli seconds
  bHubContrCurrent    100 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0503 highspeed power enable connect
   Port 2: 0000.0100 power
   Port 3: 0000.0103 power enable connect
   Port 4: 0000.0100 power
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0
  bDeviceProtocol         0 Full speed (or root) hub
  bMaxPacketSize0        64
  bNumConfigurations      1
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered

-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-14  8:57           ` Mirza Krak
@ 2018-02-14 12:07             ` Minas Harutyunyan
       [not found]               ` <410670D7E743164D87FA6160E7907A560113A6B365-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-14 12:07 UTC (permalink / raw)
  To: Mirza Krak, Minas Harutyunyan
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 2/14/2018 12:57 PM, Mirza Krak wrote:
> On 8 February 2018 at 14:53, Minas Harutyunyan
> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>> On 2/8/2018 5:07 PM, Mirza Krak wrote:
>>> On 8 February 2018 at 12:02, Minas Harutyunyan
>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>> Hi Mirza,
>>>>
>>>> On 2/7/2018 1:49 PM, Mirza Krak wrote:
>>>
>>> < snip >
>>>
>>>>
>>>> Could you please provide additional info:
>>>> 1. Does your core support descriptor DMA mode? (bits 31 and 30 of GHWCFG4).
>>>> 2. Version of core (GSNPSID)?
>>>
>>> It does seem to support descriptor DMA mode, and content of both
>>> GHWCFG4 and GSNPSID are posted below.
>>>
>>> root@esp5-d1:~# dmesg | grep -E "GHWCFG4|GSNPSID"
>>> [    1.109298] dwc2 ff540000.usb: GSNPSID @0xF0940040 : 0x4F54310A
>>> [    1.109344] dwc2 ff540000.usb: GHWCFG4 @0xF0940050 : 0xC8004030
>>> [    1.489016] dwc2 ff580000.usb: GSNPSID @0xF09C0040 : 0x4F54310A
>>> [    1.489063] dwc2 ff580000.usb: GHWCFG4 @0xF09C0050 : 0xDBF04030
>>>
>>> Full register dump at boot (no devices connected) can be found at [1].
>>>
>>>> 3. Do you connect any HS HUB between dwc core port and keyboard? Is it
>>>> available connection scenarios when Split traffic generated?
>>>
>>> There is actually an HS HUB (FE1_QFP48) between dwc core and keyboard,
>>> I was not aware of it but it is mounted on the board.
>>>
>>>
>>> [1]. https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_mirzak_a6245dddd4670f3b908a9783f9db8b0d&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=FuRmhDEOJlbUvuyApaxG_YmB514C3IOGSzs6Mq_fzC0&s=ELF0UP4IluOiL35JlnFjiltWZx-BcTIsTaPjaWb0RFk&e=
>>>
>> Actually your core supported DDMA mode, moreover core allow SW to
>> dynamically switch between BDMA and DDMA after core reset.
>> If dwc2 core work in DDMA mode then core can track (u)frames internally
>> and unmasking SOF's not required, but Host in DDMA doesn't support Split
>> transfers. This is why, in your case (FS/LS keyboard connected to HS
>> HUB), core switched to BDMA and unmasking SOF interrupt to track (u)frames.
> 
> Thank you for your valuable respons.
> 
> It is indeed as you mentioned the combination of HS and FS/LS devices
> that trigger the high CPU load. Running HS only devices does not
> produce any noticeable load.
> 
> But I am confused by one thing, I have two different boards with the
> same SoC and same version of the DWC2 IP. Both boards have an internal
> USB hub for the external connectors, though it seems that they have
> different types of HUB' s. And I only have problems with one of them.
> The second board does not have a problem mixing HS and FS/LS devices.
> Is it possible that the USB HUB has some kind of "offloading" for the
> Split transfers?
> 
> Tinkerboard USB hub (works well when mixing HS and FS/LS devices)
> ----------------------------
> 
> $ lsusb -t
> /:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
>      |__ Port 1: Dev 2, If 0, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 1, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 2, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 3, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 4, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 5, Class=Audio, Driver=, 480M
>      |__ Port 1: Dev 2, If 255, Class=Human Interface Device, Driver=usbhid, 480M
> /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
> /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
>      |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
>          |__ Port 3: Dev 4, If 0, Class=Audio, Driver=, 12M
>          |__ Port 3: Dev 4, If 1, Class=Audio, Driver=, 12M
>          |__ Port 4: Dev 3, If 0, Class=Human Interface Device,
> Driver=usbhid, 12M
> 
> $ lsusb -d 05e3:0610 -v
> 
> Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
> Device Descriptor:
>    bLength                18
>    bDescriptorType         1
>    bcdUSB               2.00
>    bDeviceClass            9 Hub
>    bDeviceSubClass         0
>    bDeviceProtocol         2 TT per port
>    bMaxPacketSize0        64
>    idVendor           0x05e3 Genesys Logic, Inc.
>    idProduct          0x0610 4-port hub
>    bcdDevice           32.98
>    iManufacturer           0
>    iProduct                1 USB2.0 Hub
>    iSerial                 0
>    bNumConfigurations      1
>    Configuration Descriptor:
>      bLength                 9
>      bDescriptorType         2
>      wTotalLength           41
>      bNumInterfaces          1
>      bConfigurationValue     1
>      iConfiguration          0
>      bmAttributes         0xe0
>        Self Powered
>        Remote Wakeup
>      MaxPower              100mA
>      Interface Descriptor:
>        bLength                 9
>        bDescriptorType         4
>        bInterfaceNumber        0
>        bAlternateSetting       0
>        bNumEndpoints           1
>        bInterfaceClass         9 Hub
>        bInterfaceSubClass      0
>        bInterfaceProtocol      1 Single TT
>        iInterface              0
>        Endpoint Descriptor:
>          bLength                 7
>          bDescriptorType         5
>          bEndpointAddress     0x81  EP 1 IN
>          bmAttributes            3
>            Transfer Type            Interrupt
>            Synch Type               None
>            Usage Type               Data
>          wMaxPacketSize     0x0001  1x 1 bytes
>          bInterval              12
>      Interface Descriptor:
>        bLength                 9
>        bDescriptorType         4
>        bInterfaceNumber        0
>        bAlternateSetting       1
>        bNumEndpoints           1
>        bInterfaceClass         9 Hub
>        bInterfaceSubClass      0
>        bInterfaceProtocol      2 TT per port
>        iInterface              0
>        Endpoint Descriptor:
>          bLength                 7
>          bDescriptorType         5
>          bEndpointAddress     0x81  EP 1 IN
>          bmAttributes            3
>            Transfer Type            Interrupt
>            Synch Type               None
>            Usage Type               Data
>          wMaxPacketSize     0x0001  1x 1 bytes
>          bInterval              12
> Hub Descriptor:
>    bLength               9
>    bDescriptorType      41
>    nNbrPorts             4
>    wHubCharacteristic 0x00e0
>      Ganged power switching
>      Ganged overcurrent protection
>      TT think time 32 FS bits
>      Port indicators
>    bPwrOn2PwrGood       50 * 2 milli seconds
>    bHubContrCurrent    100 milli Ampere
>    DeviceRemovable    0x00
>    PortPwrCtrlMask    0xff
>   Hub Port Status:
>     Port 1: 0000.0100 power
>     Port 2: 0000.0100 power
>     Port 3: 0000.0103 power enable connect
>     Port 4: 0000.0103 power enable connect
> Device Qualifier (for other device speed):
>    bLength                10
>    bDescriptorType         6
>    bcdUSB               2.00
>    bDeviceClass            9 Hub
>    bDeviceSubClass         0
>    bDeviceProtocol         0 Full speed (or root) hub
>    bMaxPacketSize0        64
>    bNumConfigurations      1
> can't get debug descriptor: Resource temporarily unavailable
> Device Status:     0x0001
>    Self Powered
> 
> 
> FireFly USB HUB (does not work well when mixing HS and FS/LS devices)
> -----------------------
> 
> $ lsusb -t
> /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
> /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc2/1p, 480M
>      |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
>          |__ Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=, 480M
>          |__ Port 3: Dev 4, If 0, Class=Audio, Driver=snd-usb-audio, 12M
>          |__ Port 3: Dev 4, If 1, Class=Audio, Driver=snd-usb-audio, 12M
> 
> $ lsusb -d 1a40:0101 -v
> 
> Bus 001 Device 002: ID 1a40:0101 Terminus Technology Inc. Hub
> Device Descriptor:
>    bLength                18
>    bDescriptorType         1
>    bcdUSB               2.00
>    bDeviceClass            9 Hub
>    bDeviceSubClass         0
>    bDeviceProtocol         2 TT per port
>    bMaxPacketSize0        64
>    idVendor           0x1a40 Terminus Technology Inc.
>    idProduct          0x0101 Hub
>    bcdDevice            1.00
>    iManufacturer           0
>    iProduct                1 USB 2.0 Hub [MTT]
>    iSerial                 0
>    bNumConfigurations      1
>    Configuration Descriptor:
>      bLength                 9
>      bDescriptorType         2
>      wTotalLength           41
>      bNumInterfaces          1
>      bConfigurationValue     1
>      iConfiguration          0
>      bmAttributes         0xe0
>        Self Powered
>        Remote Wakeup
>      MaxPower              100mA
>      Interface Descriptor:
>        bLength                 9
>        bDescriptorType         4
>        bInterfaceNumber        0
>        bAlternateSetting       0
>        bNumEndpoints           1
>        bInterfaceClass         9 Hub
>        bInterfaceSubClass      0
>        bInterfaceProtocol      1 Single TT
>        iInterface              0
>        Endpoint Descriptor:
>          bLength                 7
>          bDescriptorType         5
>          bEndpointAddress     0x81  EP 1 IN
>          bmAttributes            3
>            Transfer Type            Interrupt
>            Synch Type               None
>            Usage Type               Data
>          wMaxPacketSize     0x0001  1x 1 bytes
>          bInterval              12
>      Interface Descriptor:
>        bLength                 9
>        bDescriptorType         4
>        bInterfaceNumber        0
>        bAlternateSetting       1
>        bNumEndpoints           1
>        bInterfaceClass         9 Hub
>        bInterfaceSubClass      0
>        bInterfaceProtocol      2 TT per port
>        iInterface              0
>        Endpoint Descriptor:
>          bLength                 7
>          bDescriptorType         5
>          bEndpointAddress     0x81  EP 1 IN
>          bmAttributes            3
>            Transfer Type            Interrupt
>            Synch Type               None
>            Usage Type               Data
>          wMaxPacketSize     0x0001  1x 1 bytes
>          bInterval              12
> Hub Descriptor:
>    bLength               9
>    bDescriptorType      41
>    nNbrPorts             4
>    wHubCharacteristic 0x0080
>      Ganged power switching
>      Ganged overcurrent protection
>      TT think time 8 FS bits
>      Port indicators
>    bPwrOn2PwrGood       50 * 2 milli seconds
>    bHubContrCurrent    100 milli Ampere
>    DeviceRemovable    0x00
>    PortPwrCtrlMask    0xff
>   Hub Port Status:
>     Port 1: 0000.0503 highspeed power enable connect
>     Port 2: 0000.0100 power
>     Port 3: 0000.0103 power enable connect
>     Port 4: 0000.0100 power
> Device Qualifier (for other device speed):
>    bLength                10
>    bDescriptorType         6
>    bcdUSB               2.00
>    bDeviceClass            9 Hub
>    bDeviceSubClass         0
>    bDeviceProtocol         0 Full speed (or root) hub
>    bMaxPacketSize0        64
>    bNumConfigurations      1
> can't get debug descriptor: Resource temporarily unavailable
> Device Status:     0x0001
>    Self Powered
> 

I reviewed your interrupt count log again. About 140,000 interrupts in 2 
seconds, obviously it's not SOF only interrupts. More probably, its NAK 
respond interrupts to SSPLIT/CSPLIT transactions. For this case I can 
recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb: 
dwc2: host: Don't retry NAKed transactions right away" which already 
merged to 4.16-rc1.

In your setups you see different behavior on different HUBs. Your HUBs 
have different "TT think time": 8 and 32. In USB2.0 spec "TT think time" 
described as follow "TT requires at most 8/32 FS bit times of inter
transaction gap on a full-/low-speed downstream bus". So, your "worst" 
HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT 
transactions which replied by NAK. As result you see about 4 time more 
interrupts comparing to "good" HUB. Could you please check interrupts 
count for "good" HUB and check "4 time" hypothesis.

Thanks,
Minas

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]               ` <410670D7E743164D87FA6160E7907A560113A6B365-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-15 13:20                 ` Mirza Krak
  2018-02-16  5:59                   ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Mirza Krak @ 2018-02-15 13:20 UTC (permalink / raw)
  To: Minas Harutyunyan
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 14 February 2018 at 13:07, Minas Harutyunyan
<Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>> On 8 February 2018 at 14:53, Minas Harutyunyan
>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:

< snip >

>
> I reviewed your interrupt count log again. About 140,000 interrupts in 2
> seconds, obviously it's not SOF only interrupts. More probably, its NAK
> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
> dwc2: host: Don't retry NAKed transactions right away" which already
> merged to 4.16-rc1.
>
> In your setups you see different behavior on different HUBs. Your HUBs
> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
> described as follow "TT requires at most 8/32 FS bit times of inter
> transaction gap on a full-/low-speed downstream bus". So, your "worst"
> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
> transactions which replied by NAK. As result you see about 4 time more
> interrupts comparing to "good" HUB. Could you please check interrupts
> count for "good" HUB and check "4 time" hypothesis.

Did some further testing. The "good" HUB is actually as bad as the
"bad" HUB, and it was my setup that caused the different behavior. Did
not use the same devices etc.

Once I made sure that the configuration and setup was the same on both
board I could see that the behaved similarly. And that is the
following interrupt load:

- BT USB (FS) = ~80k interrupts / second
- Keyboard (FS) = ~80k interrupts / second
- WiFI USB (HS) = ~8k interrupts / second

After applying the suggested patch [1], it is steady around 8k
interrupts / second no matter what device I connect (HS/FS,
HUB/NO-HUB). Which is acceptable and usable.

Should this patch be a candidate for stable? Because in the current
state, this combination DWC2 + HS HUB + FS device, is not "usable".

[1]. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/usb/dwc2?h=v4.16-rc1&id=38d2b5fb75c15923fb89c32134516a623515bce4

-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-15 13:20                 ` Mirza Krak
@ 2018-02-16  5:59                   ` Minas Harutyunyan
       [not found]                     ` <410670D7E743164D87FA6160E7907A560113A6C4CA-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-16  5:59 UTC (permalink / raw)
  To: Mirza Krak, Minas Harutyunyan
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 2/15/2018 5:20 PM, Mirza Krak wrote:
> On 14 February 2018 at 13:07, Minas Harutyunyan
> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
> 
> < snip >
> 
>>
>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>> dwc2: host: Don't retry NAKed transactions right away" which already
>> merged to 4.16-rc1.
>>
>> In your setups you see different behavior on different HUBs. Your HUBs
>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>> described as follow "TT requires at most 8/32 FS bit times of inter
>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>> transactions which replied by NAK. As result you see about 4 time more
>> interrupts comparing to "good" HUB. Could you please check interrupts
>> count for "good" HUB and check "4 time" hypothesis.
> 
> Did some further testing. The "good" HUB is actually as bad as the
> "bad" HUB, and it was my setup that caused the different behavior. Did
> not use the same devices etc.
> 
> Once I made sure that the configuration and setup was the same on both
> board I could see that the behaved similarly. And that is the
> following interrupt load:
> 
> - BT USB (FS) = ~80k interrupts / second
> - Keyboard (FS) = ~80k interrupts / second
> - WiFI USB (HS) = ~8k interrupts / second
> 
> After applying the suggested patch [1], it is steady around 8k
> interrupts / second no matter what device I connect (HS/FS,
> HUB/NO-HUB). Which is acceptable and usable.

Great!
> 
> Should this patch be a candidate for stable? Because in the current
> state, this combination DWC2 + HS HUB + FS device, is not "usable".
> 
Ask the author of patch or dwc2 maintainer.

Thanks,
Minas


> [1]. https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pub_scm_linux_kernel_git_torvalds_linux.git_commit_drivers_usb_dwc2-3Fh-3Dv4.16-2Drc1-26id-3D38d2b5fb75c15923fb89c32134516a623515bce4&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=6z9Al9FrHR_ZqbbtSAsD16pvOL2S3XHxQnSzq8kusyI&m=5Np0whDSSNNvE3TAVBpFa0JUmvQbS3qWdpC0PQssosY&s=CYTOKYUsKlYbJKodeDN_ZybQpBL0ULIyf2Y_MeZgV30&e=
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]                     ` <410670D7E743164D87FA6160E7907A560113A6C4CA-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-16 20:07                       ` Marek Vasut
  2018-02-19  8:19                         ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Marek Vasut @ 2018-02-16 20:07 UTC (permalink / raw)
  To: Minas Harutyunyan, Mirza Krak
  Cc: linux-usb, gregkh, John.Youn-HKixBCOQz3hWk0Htik3J/w,
	linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>> On 14 February 2018 at 13:07, Minas Harutyunyan
>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>
>> < snip >
>>
>>>
>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>> merged to 4.16-rc1.
>>>
>>> In your setups you see different behavior on different HUBs. Your HUBs
>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>> transactions which replied by NAK. As result you see about 4 time more
>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>> count for "good" HUB and check "4 time" hypothesis.
>>
>> Did some further testing. The "good" HUB is actually as bad as the
>> "bad" HUB, and it was my setup that caused the different behavior. Did
>> not use the same devices etc.
>>
>> Once I made sure that the configuration and setup was the same on both
>> board I could see that the behaved similarly. And that is the
>> following interrupt load:
>>
>> - BT USB (FS) = ~80k interrupts / second
>> - Keyboard (FS) = ~80k interrupts / second
>> - WiFI USB (HS) = ~8k interrupts / second
>>
>> After applying the suggested patch [1], it is steady around 8k
>> interrupts / second no matter what device I connect (HS/FS,
>> HUB/NO-HUB). Which is acceptable and usable.
> 
> Great!

So 8k IRQs per second is what I should expect from this HW with HS
device attached, that's normal and cannot be reduced ?

-- 
Best regards,
Marek Vasut
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-16 20:07                       ` Marek Vasut
@ 2018-02-19  8:19                         ` Minas Harutyunyan
       [not found]                           ` <410670D7E743164D87FA6160E7907A560113A6E9E6-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-19  8:19 UTC (permalink / raw)
  To: Marek Vasut, Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 2/17/2018 12:07 AM, Marek Vasut wrote:
> On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
>> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>>> On 14 February 2018 at 13:07, Minas Harutyunyan
>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>
>>> < snip >
>>>
>>>>
>>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>>> merged to 4.16-rc1.
>>>>
>>>> In your setups you see different behavior on different HUBs. Your HUBs
>>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>>> transactions which replied by NAK. As result you see about 4 time more
>>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>>> count for "good" HUB and check "4 time" hypothesis.
>>>
>>> Did some further testing. The "good" HUB is actually as bad as the
>>> "bad" HUB, and it was my setup that caused the different behavior. Did
>>> not use the same devices etc.
>>>
>>> Once I made sure that the configuration and setup was the same on both
>>> board I could see that the behaved similarly. And that is the
>>> following interrupt load:
>>>
>>> - BT USB (FS) = ~80k interrupts / second
>>> - Keyboard (FS) = ~80k interrupts / second
>>> - WiFI USB (HS) = ~8k interrupts / second
>>>
>>> After applying the suggested patch [1], it is steady around 8k
>>> interrupts / second no matter what device I connect (HS/FS,
>>> HUB/NO-HUB). Which is acceptable and usable.
>>
>> Great!
> 
> So 8k IRQs per second is what I should expect from this HW with HS
> device attached, that's normal and cannot be reduced ?
> 
If core acting as Host in Buffer DMA mode then 8k IRQs per second (SOF 
interrupts) is expected if connected device(s) has periodic endpoint.

Thanks,
Minas

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]                           ` <410670D7E743164D87FA6160E7907A560113A6E9E6-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-19  8:51                             ` Marek Vasut
  2018-02-19 10:11                               ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Marek Vasut @ 2018-02-19  8:51 UTC (permalink / raw)
  To: Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 02/19/2018 09:19 AM, Minas Harutyunyan wrote:
> On 2/17/2018 12:07 AM, Marek Vasut wrote:
>> On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
>>> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>>>> On 14 February 2018 at 13:07, Minas Harutyunyan
>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>
>>>> < snip >
>>>>
>>>>>
>>>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>>>> merged to 4.16-rc1.
>>>>>
>>>>> In your setups you see different behavior on different HUBs. Your HUBs
>>>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>>>> transactions which replied by NAK. As result you see about 4 time more
>>>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>>>> count for "good" HUB and check "4 time" hypothesis.
>>>>
>>>> Did some further testing. The "good" HUB is actually as bad as the
>>>> "bad" HUB, and it was my setup that caused the different behavior. Did
>>>> not use the same devices etc.
>>>>
>>>> Once I made sure that the configuration and setup was the same on both
>>>> board I could see that the behaved similarly. And that is the
>>>> following interrupt load:
>>>>
>>>> - BT USB (FS) = ~80k interrupts / second
>>>> - Keyboard (FS) = ~80k interrupts / second
>>>> - WiFI USB (HS) = ~8k interrupts / second
>>>>
>>>> After applying the suggested patch [1], it is steady around 8k
>>>> interrupts / second no matter what device I connect (HS/FS,
>>>> HUB/NO-HUB). Which is acceptable and usable.
>>>
>>> Great!
>>
>> So 8k IRQs per second is what I should expect from this HW with HS
>> device attached, that's normal and cannot be reduced ?
>>
> If core acting as Host in Buffer DMA mode then 8k IRQs per second (SOF 
> interrupts) is expected if connected device(s) has periodic endpoint.

Is there a way to reduce that or is that the absolute minimum in HS mode?

-- 
Best regards,
Marek Vasut

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-19  8:51                             ` Marek Vasut
@ 2018-02-19 10:11                               ` Minas Harutyunyan
       [not found]                                 ` <410670D7E743164D87FA6160E7907A560113A6EB2B-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-19 10:11 UTC (permalink / raw)
  To: Marek Vasut, Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 2/19/2018 12:51 PM, Marek Vasut wrote:
> On 02/19/2018 09:19 AM, Minas Harutyunyan wrote:
>> On 2/17/2018 12:07 AM, Marek Vasut wrote:
>>> On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
>>>> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>>>>> On 14 February 2018 at 13:07, Minas Harutyunyan
>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>
>>>>> < snip >
>>>>>
>>>>>>
>>>>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>>>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>>>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>>>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>>>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>>>>> merged to 4.16-rc1.
>>>>>>
>>>>>> In your setups you see different behavior on different HUBs. Your HUBs
>>>>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>>>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>>>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>>>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>>>>> transactions which replied by NAK. As result you see about 4 time more
>>>>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>>>>> count for "good" HUB and check "4 time" hypothesis.
>>>>>
>>>>> Did some further testing. The "good" HUB is actually as bad as the
>>>>> "bad" HUB, and it was my setup that caused the different behavior. Did
>>>>> not use the same devices etc.
>>>>>
>>>>> Once I made sure that the configuration and setup was the same on both
>>>>> board I could see that the behaved similarly. And that is the
>>>>> following interrupt load:
>>>>>
>>>>> - BT USB (FS) = ~80k interrupts / second
>>>>> - Keyboard (FS) = ~80k interrupts / second
>>>>> - WiFI USB (HS) = ~8k interrupts / second
>>>>>
>>>>> After applying the suggested patch [1], it is steady around 8k
>>>>> interrupts / second no matter what device I connect (HS/FS,
>>>>> HUB/NO-HUB). Which is acceptable and usable.
>>>>
>>>> Great!
>>>
>>> So 8k IRQs per second is what I should expect from this HW with HS
>>> device attached, that's normal and cannot be reduced ?
>>>
>> If core acting as Host in Buffer DMA mode then 8k IRQs per second (SOF
>> interrupts) is expected if connected device(s) has periodic endpoint.
> 
> Is there a way to reduce that or is that the absolute minimum in HS mode?
> 
We already discussed, in this email thread earlier, why SOF interrupts 
required and unmasked.
Only in case when connected device with CTRL+BLK EP's only (like flash 
drive) and directly connected to cores root HUB, SOF's will be masked.
Thanks,
Minas

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]                                 ` <410670D7E743164D87FA6160E7907A560113A6EB2B-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-02-19 17:02                                   ` Marek Vasut
  2018-02-20  5:51                                     ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Marek Vasut @ 2018-02-19 17:02 UTC (permalink / raw)
  To: Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 02/19/2018 11:11 AM, Minas Harutyunyan wrote:
> On 2/19/2018 12:51 PM, Marek Vasut wrote:
>> On 02/19/2018 09:19 AM, Minas Harutyunyan wrote:
>>> On 2/17/2018 12:07 AM, Marek Vasut wrote:
>>>> On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
>>>>> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>>>>>> On 14 February 2018 at 13:07, Minas Harutyunyan
>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>>>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>>
>>>>>> < snip >
>>>>>>
>>>>>>>
>>>>>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>>>>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>>>>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>>>>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>>>>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>>>>>> merged to 4.16-rc1.
>>>>>>>
>>>>>>> In your setups you see different behavior on different HUBs. Your HUBs
>>>>>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>>>>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>>>>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>>>>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>>>>>> transactions which replied by NAK. As result you see about 4 time more
>>>>>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>>>>>> count for "good" HUB and check "4 time" hypothesis.
>>>>>>
>>>>>> Did some further testing. The "good" HUB is actually as bad as the
>>>>>> "bad" HUB, and it was my setup that caused the different behavior. Did
>>>>>> not use the same devices etc.
>>>>>>
>>>>>> Once I made sure that the configuration and setup was the same on both
>>>>>> board I could see that the behaved similarly. And that is the
>>>>>> following interrupt load:
>>>>>>
>>>>>> - BT USB (FS) = ~80k interrupts / second
>>>>>> - Keyboard (FS) = ~80k interrupts / second
>>>>>> - WiFI USB (HS) = ~8k interrupts / second
>>>>>>
>>>>>> After applying the suggested patch [1], it is steady around 8k
>>>>>> interrupts / second no matter what device I connect (HS/FS,
>>>>>> HUB/NO-HUB). Which is acceptable and usable.
>>>>>
>>>>> Great!
>>>>
>>>> So 8k IRQs per second is what I should expect from this HW with HS
>>>> device attached, that's normal and cannot be reduced ?
>>>>
>>> If core acting as Host in Buffer DMA mode then 8k IRQs per second (SOF
>>> interrupts) is expected if connected device(s) has periodic endpoint.
>>
>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>
> We already discussed, in this email thread earlier, why SOF interrupts 
> required and unmasked.
> Only in case when connected device with CTRL+BLK EP's only (like flash 
> drive) and directly connected to cores root HUB, SOF's will be masked.

That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.

-- 
Best regards,
Marek Vasut

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-02-19 17:02                                   ` Marek Vasut
@ 2018-02-20  5:51                                     ` Minas Harutyunyan
       [not found]                                       ` <410670D7E743164D87FA6160E7907A560113A6F1B4-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-02-20  5:51 UTC (permalink / raw)
  To: Marek Vasut, Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 2/19/2018 9:03 PM, Marek Vasut wrote:
> On 02/19/2018 11:11 AM, Minas Harutyunyan wrote:
>> On 2/19/2018 12:51 PM, Marek Vasut wrote:
>>> On 02/19/2018 09:19 AM, Minas Harutyunyan wrote:
>>>> On 2/17/2018 12:07 AM, Marek Vasut wrote:
>>>>> On 02/16/2018 06:59 AM, Minas Harutyunyan wrote:
>>>>>> On 2/15/2018 5:20 PM, Mirza Krak wrote:
>>>>>>> On 14 February 2018 at 13:07, Minas Harutyunyan
>>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>>>> On 2/14/2018 12:57 PM, Mirza Krak wrote:
>>>>>>>>> On 8 February 2018 at 14:53, Minas Harutyunyan
>>>>>>>>> <Minas.Harutyunyan-HKixBCOQz3hWk0Htik3J/w@public.gmane.org> wrote:
>>>>>>>
>>>>>>> < snip >
>>>>>>>
>>>>>>>>
>>>>>>>> I reviewed your interrupt count log again. About 140,000 interrupts in 2
>>>>>>>> seconds, obviously it's not SOF only interrupts. More probably, its NAK
>>>>>>>> respond interrupts to SSPLIT/CSPLIT transactions. For this case I can
>>>>>>>> recommend you to apply patch from Douglas Anderson: "[PATCH v2] usb:
>>>>>>>> dwc2: host: Don't retry NAKed transactions right away" which already
>>>>>>>> merged to 4.16-rc1.
>>>>>>>>
>>>>>>>> In your setups you see different behavior on different HUBs. Your HUBs
>>>>>>>> have different "TT think time": 8 and 32. In USB2.0 spec "TT think time"
>>>>>>>> described as follow "TT requires at most 8/32 FS bit times of inter
>>>>>>>> transaction gap on a full-/low-speed downstream bus". So, your "worst"
>>>>>>>> HUB with "TT think time"=8 sending more frequently SSPLIT/CSPLIT
>>>>>>>> transactions which replied by NAK. As result you see about 4 time more
>>>>>>>> interrupts comparing to "good" HUB. Could you please check interrupts
>>>>>>>> count for "good" HUB and check "4 time" hypothesis.
>>>>>>>
>>>>>>> Did some further testing. The "good" HUB is actually as bad as the
>>>>>>> "bad" HUB, and it was my setup that caused the different behavior. Did
>>>>>>> not use the same devices etc.
>>>>>>>
>>>>>>> Once I made sure that the configuration and setup was the same on both
>>>>>>> board I could see that the behaved similarly. And that is the
>>>>>>> following interrupt load:
>>>>>>>
>>>>>>> - BT USB (FS) = ~80k interrupts / second
>>>>>>> - Keyboard (FS) = ~80k interrupts / second
>>>>>>> - WiFI USB (HS) = ~8k interrupts / second
>>>>>>>
>>>>>>> After applying the suggested patch [1], it is steady around 8k
>>>>>>> interrupts / second no matter what device I connect (HS/FS,
>>>>>>> HUB/NO-HUB). Which is acceptable and usable.
>>>>>>
>>>>>> Great!
>>>>>
>>>>> So 8k IRQs per second is what I should expect from this HW with HS
>>>>> device attached, that's normal and cannot be reduced ?
>>>>>
>>>> If core acting as Host in Buffer DMA mode then 8k IRQs per second (SOF
>>>> interrupts) is expected if connected device(s) has periodic endpoint.
>>>
>>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>>
>> We already discussed, in this email thread earlier, why SOF interrupts
>> required and unmasked.
>> Only in case when connected device with CTRL+BLK EP's only (like flash
>> drive) and directly connected to cores root HUB, SOF's will be masked.
> 
> That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.
> 
Could you please send verbose lsusb on connected to dwc2 device and 
driver debug log.
Thanks,
Minas

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]                                       ` <410670D7E743164D87FA6160E7907A560113A6F1B4-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-03-05 17:37                                         ` Marek Vasut
  2018-03-06  6:45                                           ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Marek Vasut @ 2018-03-05 17:37 UTC (permalink / raw)
  To: Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

[-- Attachment #1: Type: text/plain, Size: 651 bytes --]

On 02/20/2018 06:51 AM, Minas Harutyunyan wrote:
[...]
>>>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>>>
>>> We already discussed, in this email thread earlier, why SOF interrupts
>>> required and unmasked.
>>> Only in case when connected device with CTRL+BLK EP's only (like flash
>>> drive) and directly connected to cores root HUB, SOF's will be masked.
>>
>> That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.
>>
> Could you please send verbose lsusb on connected to dwc2 device

See attached

> and driver debug log.

What exactly do you mean by this one ?

-- 
Best regards,
Marek Vasut

[-- Attachment #2: usb.log --]
[-- Type: text/x-log, Size: 8073 bytes --]


Bus 001 Device 003: ID 0781:5581 SanDisk Corp. Ultra
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x0781 SanDisk Corp.
  idProduct          0x5581 Ultra
  bcdDevice            1.00
  iManufacturer           1 SanDisk
  iProduct                2 Ultra
  iSerial                 3 4C530001110620109113
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              224mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           22
  bNumDeviceCaps          2
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat         256 micro seconds
Device Status:     0x0000
  (Bus Powered)

Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         2 TT per port
  bMaxPacketSize0        64
  idVendor           0x0424 Standard Microsystems Corp.
  idProduct          0x2514 USB 2.0 Hub
  bcdDevice            b.b3
  iManufacturer           0 
  iProduct                0 
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           41
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      1 Single TT
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      2 TT per port
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             4
  wHubCharacteristic 0x0009
    Per-port power switching
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood       50 * 2 milli seconds
  bHubContrCurrent      1 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0503 highspeed power enable connect
   Port 2: 0000.0100 power
   Port 3: 0000.0100 power
   Port 4: 0000.0100 power
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         0 Full speed (or root) hub
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 Unused
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x1d6b Linux Foundation
  idProduct          0x0002 2.0 root hub
  bcdDevice            4.09
  iManufacturer           3 Linux 4.9.78-rt61-yocto-preempt-rt+ dwc2_hsotg
  iProduct                2 DWC OTG Controller
  iSerial                 1 ffb00000.usb
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           25
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             1
  wHubCharacteristic 0x0008
    Ganged power switching
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood        1 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0503 highspeed power enable connect
Device Status:     0x0001
  Self Powered

[-- Attachment #3: Type: text/plain, Size: 200 bytes --]

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-03-05 17:37                                         ` Marek Vasut
@ 2018-03-06  6:45                                           ` Minas Harutyunyan
  2018-03-06  8:09                                             ` Minas Harutyunyan
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-03-06  6:45 UTC (permalink / raw)
  To: Marek Vasut, Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

Hi,

On 3/5/2018 11:14 PM, Marek Vasut wrote:
> On 02/20/2018 06:51 AM, Minas Harutyunyan wrote:
> [...]
>>>>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>>>>
>>>> We already discussed, in this email thread earlier, why SOF interrupts
>>>> required and unmasked.
>>>> Only in case when connected device with CTRL+BLK EP's only (like flash
>>>> drive) and directly connected to cores root HUB, SOF's will be masked.
>>>
>>> That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.
>>>
>> Could you please send verbose lsusb on connected to dwc2 device
> 
> See attached
> 
>> and driver debug log.
> 
> What exactly do you mean by this one ?
Enable debugging messages and verbose debugging messages for dwc2 from 
make menuconfig. Provide dmesg starting from dwc2 load till HS device 
connected to dwc2 port and enumerated.

Thanks,
Minas

> 

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
  2018-03-06  6:45                                           ` Minas Harutyunyan
@ 2018-03-06  8:09                                             ` Minas Harutyunyan
       [not found]                                               ` <410670D7E743164D87FA6160E7907A560113AB3F3D-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
  0 siblings, 1 reply; 20+ messages in thread
From: Minas Harutyunyan @ 2018-03-06  8:09 UTC (permalink / raw)
  To: Marek Vasut, Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

Hi,

On 3/6/2018 10:45 AM, Minas Harutyunyan wrote:
> Hi,
> 
> On 3/5/2018 11:14 PM, Marek Vasut wrote:
>> On 02/20/2018 06:51 AM, Minas Harutyunyan wrote:
>> [...]
>>>>>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>>>>>
>>>>> We already discussed, in this email thread earlier, why SOF interrupts
>>>>> required and unmasked.
>>>>> Only in case when connected device with CTRL+BLK EP's only (like flash
>>>>> drive) and directly connected to cores root HUB, SOF's will be masked.
>>>>
>>>> That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.
>>>>
>>> Could you please send verbose lsusb on connected to dwc2 device
>>
>> See attached
>>
>>> and driver debug log.
>>
>> What exactly do you mean by this one ?
> Enable debugging messages and verbose debugging messages for dwc2 from
> make menuconfig. Provide dmesg starting from dwc2 load till HS device
> connected to dwc2 port and enumerated.
> 
> Thanks,
> Minas
> 
>>
> 
> 
Driver debug log not required.
Based on lsusb output: to dwc2 port connected "Standard Microsystems 
Corp. USB 2.0 Hub" to which connected your HS device "SanDisk Corp. 
Ultra". Because of connected HUB, which have periodic endpoint 
(Interrupt IN EP1) like all HUB's, dwc2 forced in Buffer DMA mode unmask 
SOF interrupts.

Thanks,
Minas

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: High CPU load produced by USB (DW2)
       [not found]                                               ` <410670D7E743164D87FA6160E7907A560113AB3F3D-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
@ 2018-03-07 22:42                                                 ` Marek Vasut
  0 siblings, 0 replies; 20+ messages in thread
From: Marek Vasut @ 2018-03-07 22:42 UTC (permalink / raw)
  To: Minas Harutyunyan, Mirza Krak
  Cc: gregkh, linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	linux-usb, John.Youn-HKixBCOQz3hWk0Htik3J/w

On 03/06/2018 09:09 AM, Minas Harutyunyan wrote:
> Hi,
> 
> On 3/6/2018 10:45 AM, Minas Harutyunyan wrote:
>> Hi,
>>
>> On 3/5/2018 11:14 PM, Marek Vasut wrote:
>>> On 02/20/2018 06:51 AM, Minas Harutyunyan wrote:
>>> [...]
>>>>>>> Is there a way to reduce that or is that the absolute minimum in HS mode?
>>>>>>>
>>>>>> We already discussed, in this email thread earlier, why SOF interrupts
>>>>>> required and unmasked.
>>>>>> Only in case when connected device with CTRL+BLK EP's only (like flash
>>>>>> drive) and directly connected to cores root HUB, SOF's will be masked.
>>>>>
>>>>> That's the setup I have on Altera SoCFPGA, yet the SOFs are still present.
>>>>>
>>>> Could you please send verbose lsusb on connected to dwc2 device
>>>
>>> See attached
>>>
>>>> and driver debug log.
>>>
>>> What exactly do you mean by this one ?
>> Enable debugging messages and verbose debugging messages for dwc2 from
>> make menuconfig. Provide dmesg starting from dwc2 load till HS device
>> connected to dwc2 port and enumerated.
>>
>> Thanks,
>> Minas
>>
>>>
>>
>>
> Driver debug log not required.
> Based on lsusb output: to dwc2 port connected "Standard Microsystems
> Corp. USB 2.0 Hub" to which connected your HS device "SanDisk Corp.
> Ultra". Because of connected HUB, which have periodic endpoint
> (Interrupt IN EP1) like all HUB's, dwc2 forced in Buffer DMA mode unmask
> SOF interrupts.

I don't understand the last part, can you rephrase ?

What does that mean regarding the SOF question above ?

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2018-03-07 22:42 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-07  9:48 High CPU load produced by USB (DW2) Mirza Krak
     [not found] ` <CALw8SCXuOCpndakFBRbP7FpJKOSH+nzNStkszWD4XTXVSK1zyQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-02-07 14:27   ` gregkh
2018-02-08  8:50     ` Minas Harutyunyan
2018-02-08 11:02 ` Minas Harutyunyan
     [not found]   ` <410670D7E743164D87FA6160E7907A560113A67F86-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-08 13:06     ` Mirza Krak
2018-02-08 13:53       ` Minas Harutyunyan
     [not found]         ` <410670D7E743164D87FA6160E7907A560113A68287-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-14  8:57           ` Mirza Krak
2018-02-14 12:07             ` Minas Harutyunyan
     [not found]               ` <410670D7E743164D87FA6160E7907A560113A6B365-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-15 13:20                 ` Mirza Krak
2018-02-16  5:59                   ` Minas Harutyunyan
     [not found]                     ` <410670D7E743164D87FA6160E7907A560113A6C4CA-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-16 20:07                       ` Marek Vasut
2018-02-19  8:19                         ` Minas Harutyunyan
     [not found]                           ` <410670D7E743164D87FA6160E7907A560113A6E9E6-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-19  8:51                             ` Marek Vasut
2018-02-19 10:11                               ` Minas Harutyunyan
     [not found]                                 ` <410670D7E743164D87FA6160E7907A560113A6EB2B-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-02-19 17:02                                   ` Marek Vasut
2018-02-20  5:51                                     ` Minas Harutyunyan
     [not found]                                       ` <410670D7E743164D87FA6160E7907A560113A6F1B4-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-03-05 17:37                                         ` Marek Vasut
2018-03-06  6:45                                           ` Minas Harutyunyan
2018-03-06  8:09                                             ` Minas Harutyunyan
     [not found]                                               ` <410670D7E743164D87FA6160E7907A560113AB3F3D-ouFQeoKRuQ8bQ7k9MBbv4fufCSb+aD3WLzEdoUbNIic@public.gmane.org>
2018-03-07 22:42                                                 ` Marek Vasut

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.