Re: lan78xx: About 8000 usb interrupts per second when idle

Re: lan78xx: About 8000 usb interrupts per second when idle

[Paul Zimmerman]

Lukas Wunner wrote:
> On Tue, Apr 09, 2019 at 09:28:16AM +0000, Minas Harutyunyan wrote:
>> Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
>>> The second one: 8000 usb interrupts per second when idle.
>>> This is abnormal. any idea? Is it due to the lan78xx?
>>
>> dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
>> So, 8000 interrupts per second is expectant behavior.
>
> The dwc_otg driver patched into the Raspberry Pi Foundation's
> kernel seems to make do with much fewer interrupts and much
> lower CPU load.  How does it do that and how could dwc2 be
> made to do the same?

It connects an interrupt from the USB Phy to the FIQ (fast interrupt
request) pin on the Arm CPU. The dwc2 driver could do the same, except
I don't think the Linux kernel supports FIQ interrupts, so that would
need to be added. It would also be unique to Arm and to the SOC that
is used on the Pi.

> Would it be possible for you to provide
> me with documentation on the chip? The Synopsis website
> requires registration for downloads and registration requires
> a Synopsis customer ID.

It's highly unlikely that Minas can provide you with the databook if
you are not a customer. I used to work for Synopsys and wrote the
original dwc2 driver. The company's policy did not allow non-customers
to be given the databook. But you can find a copy on the web from a
couple of different sites, google for dwc_otg. In any case, it would
not cover the FIQ method that is used in the Pi driver, you would
probably need the Phy databook for that.

> It seems the Foundation's dwc_otg driver was forked from code
> that later begat dwc2.

Yes, I ported dwc2 from the Synopsys dwc_otg driver. But the FIQ
interrupt stuff was not part of the dwc_otg driver, it was added by
the Raspberry Pi foundation guys specifically to work around problems
on the Pi platform.

HTH,
Paul

Re: lan78xx: About 8000 usb interrupts per second when idle

[Alan Stern]

On Mon, 15 Apr 2019, Doug Anderson wrote:

> Hi,
> 
> On Sun, Apr 14, 2019 at 12:46 PM Lukas Wunner <lukas@wunner.de> wrote:
> >
> > On Tue, Apr 09, 2019 at 09:28:16AM +0000, Minas Harutyunyan wrote:
> > > Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
> > > > The second one: 8000 usb interrupts per second when idle.
> > > > This is abnormal. any idea? Is it due to the lan78xx?
> > >
> > > dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> > > So, 8000 interrupts per second is expectant behavior.
> >
> > The dwc_otg driver patched into the Raspberry Pi Foundation's
> > kernel seems to make do with much fewer interrupts and much
> > lower CPU load.  How does it do that and how could dwc2 be
> > made to do the same?  Would it be possible for you to provide
> > me with documentation on the chip?  The Synopsis website
> > requires registration for downloads and registration requires
> > a Synopsis customer ID.
> >
> > It seems the Foundation's dwc_otg driver was forked from code
> > that later begat dwc2.
> 
> Your information might be misleading.  The downstream dwc2 driver for
> Raspberry PI handles the SoF interrupts at FIQ (fast interrupt) time.
> The idea here is that this is to prioritize it above all other things
> in the system since FIQ can fire even if we're currently in another
> interrupt handler.
> 
> IIRC:
> 
> * FIQs don't get counted in /proc/interrupts.  So probably you really
> are getting 8000 FIQs per second still, you just don't know it.
> 
> * FIQs don't count towards CPU load calculations, so it looks like the
> CPU is less loaded by this than it really is.  I have no evidence
> here, but I seem to remember someone telling me this, so if you
> believe this is wrong then ignore it.
> 
> 
> That all being said, though the purpose of using the FIQ is to improve
> the latency of handling SoF interrupts, it is also plausible that when
> it was written it also had the side effect of making the code more
> efficient.  I mean, there's theoretically maybe some built-in
> efficiency by skipping all the Linux interrupt infrastructure, but I'd
> bet that's not a huge deal and a bigger deal is how inefficient the
> mainline dwc2 driver is at handling interrupts.  I doubt you'd manage
> to get FIQ support for something like this on mainline, but you could
> possible spend more time improving the efficiency of the interrupt
> handler.  I spent some time on this a while ago but it was just small
> things--I didn't gut it and re-think how to make it faster.
> 
> 
> Note also that I spent a bit of time a few years ago making the
> upstream dwc2 driver more robust despite some of its inefficiencies.
> In the end it was fairly robust, though if you wanted to do something
> like audio or USB webcams with it you'd still struggle without higher
> CPU speeds or patches like <https://crbug.com/820961>.  I'm still of
> the belief that, unless the downstream driver has ported over the
> uFrame work I did, that the upstream dwc2 driver will be compatible
> with more more combinations of devices/hubs than the downstream one.

I can confirm that the upstream dwc2 driver has fixed some bugs that
are (or were at the time) present in dwc_otg.  It was years ago and I
don't remember the details, but one of the problems involved
communication with a full-speed device.  Possibly isochronous
transfers; I'm not sure.  The transfers were failing, and a USB bus
analyzer showed that the reason was an invalid byte sequence sent by
the Raspberry Pi while using dwc_otg.  dwc2 sent the correct sequence.

I don't know how much development work is currently going into dwc_otg, 
but I suspect it's a lot less than the amount of work being put into 
dwc2.

Alan Stern

Re: lan78xx: About 8000 usb interrupts per second when idle

[Doug Anderson]

Hi,

On Sun, Apr 14, 2019 at 12:46 PM Lukas Wunner <lukas@wunner.de> wrote:
>
> On Tue, Apr 09, 2019 at 09:28:16AM +0000, Minas Harutyunyan wrote:
> > Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
> > > The second one: 8000 usb interrupts per second when idle.
> > > This is abnormal. any idea? Is it due to the lan78xx?
> >
> > dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> > So, 8000 interrupts per second is expectant behavior.
>
> The dwc_otg driver patched into the Raspberry Pi Foundation's
> kernel seems to make do with much fewer interrupts and much
> lower CPU load.  How does it do that and how could dwc2 be
> made to do the same?  Would it be possible for you to provide
> me with documentation on the chip?  The Synopsis website
> requires registration for downloads and registration requires
> a Synopsis customer ID.
>
> It seems the Foundation's dwc_otg driver was forked from code
> that later begat dwc2.

Your information might be misleading.  The downstream dwc2 driver for
Raspberry PI handles the SoF interrupts at FIQ (fast interrupt) time.
The idea here is that this is to prioritize it above all other things
in the system since FIQ can fire even if we're currently in another
interrupt handler.

IIRC:

* FIQs don't get counted in /proc/interrupts.  So probably you really
are getting 8000 FIQs per second still, you just don't know it.

* FIQs don't count towards CPU load calculations, so it looks like the
CPU is less loaded by this than it really is.  I have no evidence
here, but I seem to remember someone telling me this, so if you
believe this is wrong then ignore it.


That all being said, though the purpose of using the FIQ is to improve
the latency of handling SoF interrupts, it is also plausible that when
it was written it also had the side effect of making the code more
efficient.  I mean, there's theoretically maybe some built-in
efficiency by skipping all the Linux interrupt infrastructure, but I'd
bet that's not a huge deal and a bigger deal is how inefficient the
mainline dwc2 driver is at handling interrupts.  I doubt you'd manage
to get FIQ support for something like this on mainline, but you could
possible spend more time improving the efficiency of the interrupt
handler.  I spent some time on this a while ago but it was just small
things--I didn't gut it and re-think how to make it faster.


Note also that I spent a bit of time a few years ago making the
upstream dwc2 driver more robust despite some of its inefficiencies.
In the end it was fairly robust, though if you wanted to do something
like audio or USB webcams with it you'd still struggle without higher
CPU speeds or patches like <https://crbug.com/820961>.  I'm still of
the belief that, unless the downstream driver has ported over the
uFrame work I did, that the upstream dwc2 driver will be compatible
with more more combinations of devices/hubs than the downstream one.


-Doug

RE: lan78xx: About 8000 usb interrupts per second when idle

[Woojung.Huh]

Hi Lukas & Minas,

> On Tue, Apr 09, 2019 at 09:28:16AM +0000, Minas Harutyunyan wrote:
> > Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
> > > The second one: 8000 usb interrupts per second when idle.
> > > This is abnormal. any idea? Is it due to the lan78xx?
> >
> > dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> > So, 8000 interrupts per second is expectant behavior.
> 
> The dwc_otg driver patched into the Raspberry Pi Foundation's
> kernel seems to make do with much fewer interrupts and much
> lower CPU load.  How does it do that and how could dwc2 be
> made to do the same?  Would it be possible for you to provide
> me with documentation on the chip?  The Synopsis website
> requires registration for downloads and registration requires
> a Synopsis customer ID.
Thanks for update information.
Can you share your info with us?

Regards,
Woojung

Re: lan78xx: About 8000 usb interrupts per second when idle

[Lukas Wunner]

On Tue, Apr 09, 2019 at 09:28:16AM +0000, Minas Harutyunyan wrote:
> Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
> > The second one: 8000 usb interrupts per second when idle.
> > This is abnormal. any idea? Is it due to the lan78xx?
> 
> dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> So, 8000 interrupts per second is expectant behavior.

The dwc_otg driver patched into the Raspberry Pi Foundation's
kernel seems to make do with much fewer interrupts and much
lower CPU load.  How does it do that and how could dwc2 be
made to do the same?  Would it be possible for you to provide
me with documentation on the chip?  The Synopsis website
requires registration for downloads and registration requires
a Synopsis customer ID.

It seems the Foundation's dwc_otg driver was forked from code
that later begat dwc2.

Thanks,

Lukas

Re: lan78xx: About 8000 usb interrupts per second when idle

[Oliver Neukum]

On Mi, 2019-04-10 at 09:37 +0000, Jisheng Zhang wrote:
> On Tue, 9 Apr 2019 09:28:16 +0000 Minas Harutyunyan wrote:

Hi,

> > dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> > So, 8000 interrupts per second is expectant behavior.
> 
> Makes sense. 8 microframes each 1ms, so 8*1000 = 8000 interrupts/s

The remedy for that would be using autosuspend. That just tells us that
support for autosuspend is not optional.

	Regards
		Oliver

Re: lan78xx: About 8000 usb interrupts per second when idle

[Jisheng Zhang]

On Tue, 9 Apr 2019 09:28:16 +0000 Minas Harutyunyan wrote:


> 
> Hi Stefan,
> 
> > Hi Minas,
> >
> > Am 09.04.19 um 08:54 schrieb Jisheng Zhang:  
> >> On Mon, 8 Apr 2019 16:05:51 +0800 Jisheng Zhang wrote:
> >>  
> >>> Hi Stefan,
> >>>
> >>> On Mon, 8 Apr 2019 09:57:14 +0200 Stefan Wahren wrote:
> >>>  
> >>>>
> >>>> Hi Jisheng,
> >>>>
> >>>>
> >>>> thanks for sending this patch.  I already reported this issue
> >>>> last year [1], but Microchip wasn't able to provide a fix.
> >>>>
> >>>> I will give it a try.
> >>>>  
> >>>>> The second one: 8000 usb interrupts per second when idle.
> >>>>> This is abnormal. any idea? Is it due to the lan78xx?  
> >>>> Does this downstream patch [2] improve your situation?  
> >>> Thanks a lot. Will try the patch tonight.  
> >> I tried this patch, it doesn't help. One more observation: if I
> >> ifconfig eth0 down, I can still get 8000 usb interrupts per second.
> >>  Not sure whether this is usb issue or not.  
> >
> > Jisheng reports a lot of USB interrupts while the lan78xx driver on
> > Raspberry Pi 3B+ is in idle.
> >
> > Any suggestions to narrow this down?  
> 
> dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
> So, 8000 interrupts per second is expectant behavior.

Makes sense. 8 microframes each 1ms, so 8*1000 = 8000 interrupts/s

RE: lan78xx: About 8000 usb interrupts per second when idle

[Minas Harutyunyan]

Hi Stefan,

> Hi Minas,
> 
> Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
>> On Mon, 8 Apr 2019 16:05:51 +0800 Jisheng Zhang wrote:
>> 
>>> Hi Stefan,
>>> 
>>> On Mon, 8 Apr 2019 09:57:14 +0200 Stefan Wahren wrote:
>>> 
>>>> 
>>>> Hi Jisheng,
>>>> 
>>>> 
>>>> thanks for sending this patch.  I already reported this issue
>>>> last year [1], but Microchip wasn't able to provide a fix.
>>>> 
>>>> I will give it a try.
>>>> 
>>>>> The second one: 8000 usb interrupts per second when idle.
>>>>> This is abnormal. any idea? Is it due to the lan78xx?
>>>> Does this downstream patch [2] improve your situation?
>>> Thanks a lot. Will try the patch tonight.
>> I tried this patch, it doesn't help. One more observation: if I 
>> ifconfig eth0 down, I can still get 8000 usb interrupts per second.
>>  Not sure whether this is usb issue or not.
> 
> Jisheng reports a lot of USB interrupts while the lan78xx driver on
> Raspberry Pi 3B+ is in idle.
> 
> Any suggestions to narrow this down?

dwc2 in host mode enable SOF interrupts if any periodic EP are in use.
So, 8000 interrupts per second is expectant behavior.

> 
> Stefan
> 
>> 
>> Thanks
>>

lan78xx: About 8000 usb interrupts per second when idle

[Stefan Wahren]

Hi Minas,

Am 09.04.19 um 08:54 schrieb Jisheng Zhang:
> On Mon, 8 Apr 2019 16:05:51 +0800 Jisheng Zhang wrote:
>
>> Hi Stefan,
>>
>> On Mon, 8 Apr 2019 09:57:14 +0200 Stefan Wahren wrote:
>>
>>>
>>> Hi Jisheng,
>>>
>>>
>>> thanks for sending this patch.  I already reported this issue last year
>>> [1], but Microchip wasn't able to provide a fix.
>>>
>>> I will give it a try.
>>>   
>>>> The second one: 8000 usb interrupts per second when idle. This is abnormal.
>>>> any idea? Is it due to the lan78xx?    
>>> Does this downstream patch [2] improve your situation?  
>> Thanks a lot. Will try the patch tonight.
> I tried this patch, it doesn't help. One more observation: if I
> ifconfig eth0 down, I can still get 8000 usb interrupts per second. Not sure
> whether this is usb issue or not.

Jisheng reports a lot of USB interrupts while the lan78xx driver on
Raspberry Pi 3B+ is in idle.

Any suggestions to narrow this down?

Stefan

>
> Thanks
>