ADSL/ATM linklayer tc shaping regression fix commits for stable

ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

Hi Greg (and google search),

ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to
3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").

I can see that 3.9-stable is marked EOL, and 3.8-stable also seems
closed.  I guess it is up to Greg, how far the commits described below
can be backported.  Below is the commit needed for a 3.9 backport.

This was not the only regression introduced by the regression commit,
several fixes were required:

(Refactor/change commit for easy 3.9 backport step (in v3.10-rc1)):
 commit 6906f4ed6f8 (htb: add HTB_DIRECT_QLEN attribute)

The following commit fixes are in 3.10-rc5:
 commit 01cb71d2d47 (net_sched: restore "overhead xxx" handling)
 commit 5343a7f8be1 (net_sched: htb: do not mix 1ns and 64ns time units)

The following commit fixes are in 3.10-rc6:
 commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array)
 (this fix goes further back, but needed for linklayer atm, see below[1])

Refactor improvements (v3.11-rc1):
 commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

The linklayer ATM/ADSL fix, reached 3.11-rc6:
 commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)


[1] Eric Dumazet also found a general "linklayer atm" regression
(dating way-back), which could cause rate-tables to get wrongly shared
between the same rates with-and-without linklayer atm settings.
Addressed/fixed in (3.10-rc6:):
 - commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array)
When configuring two completely equal rates, the kernel detects that
these two equal rates can share the same rate-table.  But the kernel
didn't check if the rate-table data had been modified, which is done
in the linklayer atm case.
In practice, this often isn't a problem, as overhead parameter is
usually combined with the linklayer parameter.


The regression commit 56b765b79 ("htb: improved accuracy at high
rates"), is not all bad, it did provided a significantly improved rate
shaping accuracy.  The old rate-table based system, which this commit
removed, would get inaccurate at rates above 100Mbit/s.

(ps. I'm the original author of http://www.adsl-optimizer.dk/ and
http://sourceforge.net/projects/adsl-optimizer/)
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Greg Kroah-Hartman]

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> Hi Greg (and google search),
> 
> ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to
> 3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").
> 
> I can see that 3.9-stable is marked EOL, and 3.8-stable also seems
> closed.  I guess it is up to Greg, how far the commits described below
> can be backported.  Below is the commit needed for a 3.9 backport.

As you say, 3.8 and 3.9 are end-of-life, so there's nothing I can, or
even want, to do with them, sorry.

If there are issues with 3.10, that's a different story.

> Refactor improvements (v3.11-rc1):
>  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)
> 
> The linklayer ATM/ADSL fix, reached 3.11-rc6:
>  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

David sends me the networking patches for the stable tree, and if he
thinks these are applicable, then I'll take them.

thanks,

greg k-h

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

On Tue, 20 Aug 2013 08:16:50 -0700
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
[...]
 
> If there are issues with 3.10, that's a different story.
> 
> > Refactor improvements (v3.11-rc1):
> >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

Needed because it fixes the accuracy of the rate calc.

> > The linklayer ATM/ADSL fix, reached 3.11-rc6:
> >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

> David sends me the networking patches for the stable tree, and if he
> thinks these are applicable, then I'll take them.

DaveM, please?  What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Dave Taht]

[-- Attachment #1.1: Type: text/plain, Size: 2182 bytes --]

On Wed, Aug 21, 2013 at 5:03 AM, Jesper Dangaard Brouer
<brouer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>wrote:

>
> On Tue, 20 Aug 2013 08:16:50 -0700
> Greg Kroah-Hartman <gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org> wrote:
> > On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> [...]
>
> > If there are issues with 3.10, that's a different story.
> >
> > > Refactor improvements (v3.11-rc1):
> > >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute()
> improvements)
>
> Needed because it fixes the accuracy of the rate calc.
>
> > > The linklayer ATM/ADSL fix, reached 3.11-rc6:
> > >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)
>
> Needed because we broke userspace interface.
>
> > David sends me the networking patches for the stable tree, and if he
> > thinks these are applicable, then I'll take them.
>
> DaveM, please?  What do you want me to do, submit this req/patches somehow?
>
> (Can see that is not currently on your stable queue via:
> http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)
>
>
+1 Dealing with the busted DSL stuff since 3.8 has been no fun and it would
be nice if these last fixes made it into 3.10.X at least so router/fw
distros can pick them up more easily. The difference in dsl behavior since
the bustage in 3.8 to the saner now is remarkable.

Probably not meeting the criteria for 3.10-stable inclusion (as they aren't
exactly bugfixes) are the tiny improvements to the flow dissector made in
3.11 for better ipip, ipv6 encapsulation, and 802.1ad support, but I
thought I'd mention them so other fq_codel folk pick them up that need them.

Not ready for any tree and increasingly doubtful for 3.12 are some
improvements under test to codel and cisco's sch_pie, if anyone wants to
play with them, let me know off list.


> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Sr. Network Kernel Developer at Red Hat
>   Author of http://www.iptv-analyzer.org
>   LinkedIn: http://www.linkedin.com/in/brouer
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #1.2: Type: text/html, Size: 3345 bytes --]

[-- Attachment #2: Type: text/plain, Size: 168 bytes --]

_______________________________________________
Bloat mailing list
Bloat-JXvr2/1DY2fm6VMwtOF2vx4hnT+Y9+D1@public.gmane.org
https://lists.bufferbloat.net/listinfo/bloat

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[David Miller]

From: Jesper Dangaard Brouer <brouer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Date: Wed, 21 Aug 2013 14:03:50 +0200

> 
> On Tue, 20 Aug 2013 08:16:50 -0700
> Greg Kroah-Hartman <gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org> wrote:
>> On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> [...]
>  
>> If there are issues with 3.10, that's a different story.
>> 
>> > Refactor improvements (v3.11-rc1):
>> >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)
> 
> Needed because it fixes the accuracy of the rate calc.
> 
>> > The linklayer ATM/ADSL fix, reached 3.11-rc6:
>> >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)
> 
> Needed because we broke userspace interface.
> 
>> David sends me the networking patches for the stable tree, and if he
>> thinks these are applicable, then I'll take them.
> 
> DaveM, please?  What do you want me to do, submit this req/patches somehow?
> 
> (Can see that is not currently on your stable queue via:
> http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

I'll put it there, you just have to formally ask me to do it that's
all.

Asking Greg to do it directly is a complete waste of time. :-)

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

On Wed, 21 Aug 2013 11:20:23 -0700 (PDT)
David Miller <davem@davemloft.net> wrote:

> From: Jesper Dangaard Brouer <brouer@redhat.com>
> Date: Wed, 21 Aug 2013 14:03:50 +0200
> 
> > 
> > On Tue, 20 Aug 2013 08:16:50 -0700
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> >> On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> > [...]
> >  
> >> If there are issues with 3.10, that's a different story.
> >> 
> >> > Refactor improvements (v3.11-rc1):
> >> >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)
> > 
> > Needed because it fixes the accuracy of the rate calc.
> > 
> >> > The linklayer ATM/ADSL fix, reached 3.11-rc6:
> >> >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)
> > 
> > Needed because we broke userspace interface.
> > 
> >> David sends me the networking patches for the stable tree, and if he
> >> thinks these are applicable, then I'll take them.
> > 
> > DaveM, please?  What do you want me to do, submit this req/patches somehow?
> > 
> > (Can see that is not currently on your stable queue via:
> > http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)
> 
> I'll put it there, you just have to formally ask me to do it that's
> all.

Thanks, I assume that this was formally enough and you are taking the patches.

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
   to see if my patch is already on your stable queue
2) if not, ask you and cc netdev@vger.kernel.org
3) you will handle the interaction with Greg and stable@vger.kernel.org

Correct?


> Asking Greg to do it directly is a complete waste of time. :-)

I'll remember that.  I was just following Documentation/stable_kernel_rules.txt,
I guess I should have known better as all my previous stable patches
have automagically been picked up and processed by you :-)

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[David Miller]

From: Jesper Dangaard Brouer <brouer@redhat.com>
Date: Thu, 22 Aug 2013 10:04:26 +0200

> So, for future reference:
> 
> Stable patches for the networking tree, I should:
> 1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
>    to see if my patch is already on your stable queue
> 2) if not, ask you and cc netdev@vger.kernel.org
> 3) you will handle the interaction with Greg and stable@vger.kernel.org
> 
> Correct?

Yes, and that's how it's been with the networking for years.

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

On Thu, 22 Aug 2013 01:23:48 -0700 (PDT)
David Miller <davem@davemloft.net> wrote:

> From: Jesper Dangaard Brouer <brouer@redhat.com>
> Date: Thu, 22 Aug 2013 10:04:26 +0200
> 
> > So, for future reference:
> > 
> > Stable patches for the networking tree, I should:
> > 1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
> >    to see if my patch is already on your stable queue
> > 2) if not, ask you and cc netdev@vger.kernel.org
> > 3) you will handle the interaction with Greg and stable@vger.kernel.org
> > 
> > Correct?
> 
> Yes, and that's how it's been with the networking for years.

Yes, and in the future, documentation of this will be available here:
 Documentation/networking/netdev-FAQ.txt

Right now is only available via net-next:
 https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/tree/Documentation/networking/netdev-FAQ.txt
or
 https://lwn.net/Articles/559211/

Guess, we should add a pointer to this new doc from
Documentation/stable_kernel_rules.txt once it hits Linus'es tree.

Now I cannot find any more ways to spell-it-out, for people searching google ;-)
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Li Zefan]

Hi David,

On 2013/8/22 16:23, David Miller wrote:
> From: Jesper Dangaard Brouer <brouer@redhat.com>
> Date: Thu, 22 Aug 2013 10:04:26 +0200
> 
>> So, for future reference:
>>
>> Stable patches for the networking tree, I should:
>> 1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
>>    to see if my patch is already on your stable queue
>> 2) if not, ask you and cc netdev@vger.kernel.org
>> 3) you will handle the interaction with Greg and stable@vger.kernel.org
>>
>> Correct?
> 
> Yes, and that's how it's been with the networking for years.
> 

How about these two stable requests?

http://article.gmane.org/gmane.linux.kernel.stable/61247/match=weidong
http://article.gmane.org/gmane.linux.kernel.stable/61248/match=weidong

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[Li Zefan]

Hi David,

On 2013/8/22 16:23, David Miller wrote:
> From: Jesper Dangaard Brouer <brouer@redhat.com>
> Date: Thu, 22 Aug 2013 10:04:26 +0200
> 
>> So, for future reference:
>>
>> Stable patches for the networking tree, I should:
>> 1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
>>    to see if my patch is already on your stable queue
>> 2) if not, ask you and cc netdev@vger.kernel.org
>> 3) you will handle the interaction with Greg and stable@vger.kernel.org
>>
>> Correct?
> 
> Yes, and that's how it's been with the networking for years.
> 

How about these two stable requests?

http://article.gmane.org/gmane.linux.kernel.stable/61247
http://article.gmane.org/gmane.linux.kernel.stable/61248

Re: ADSL/ATM linklayer tc shaping regression fix commits for stable

[David Miller]

From: Li Zefan <lizefan@huawei.com>
Date: Fri, 23 Aug 2013 15:59:58 +0800

> How about these two stable requests?

They are in my inbox, and I'll get to them when I get to them.