Fwd: tc question about ingress bandwidth splitting

Fwd: tc question about ingress bandwidth splitting

[Philip Prindeville]

Had originally posted this to LARTC but realized that “netdev” is probably the better forum.

Was hoping someone familiar with the nuts and bolts of tc and scheduler minutiae could help me come up with a configuration to use as a starting point, then I could tweak it, gather some numbers, make graphs etc, and write a LARTC or LWN article around the findings.

I’d be trying to do shaping in both directions.  Sure, egress shaping is trivial and obviously works.

But I was also thinking about ingress shaping on the last hop, i.e. as traffic flows into the last-hop CPE router, and limiting/delaying it so that the entire end-to-end path is appropriately perceived by the sender, since the effective bandwidth of a [non-multipath] route is the min bandwidth of all individual hops, right?

So that min could be experienced at the final hop before the receiver as delay injected between packets to shape the bitrate.

How far off-base am I?

And what would some tc scripting look like to measure my thesis?



> Begin forwarded message:
> 
> From: Philip Prindeville <philipp_subx@redfish-solutions.com>
> Subject: tc question about ingress bandwidth splitting
> Date: March 22, 2020 at 3:56:46 PM MDT
> To: lartc@vger.kernel.org
> 
> Hi all,
> 
> I asked around on IRC but no one seems to know the answer, so I thought I’d go to the source…
> 
> I have a SoHo router with two physical subnets, which we’ll call “production” (eth0) and “guest” (eth1), and the egress interface “wan” (eth5).
> 
> The uplink is G.PON 50/10 mbps.  I’d like to cap the usage on “guest” to 10/2 mbps.  Any unused bandwidth from “guest” goes to “production”.
> 
> I thought about marking the traffic coming in off “wan" (the public interface).  Then using HTB to have a 50 mbps cap at the root, and allocating 10mb/s to the child “guest”.  The other sibling would be “production”, and he gets the remaining traffic.
> 
> Upstream would be the reverse, marking ingress traffic from “guest” with a separate tag.  Allocating upstream root on “wan” with 10 mbps, and the child “guest” getting 2 mbps.  The remainder goes to the sibling “production”.
> 
> Should be straightforward enough, right? (Well, forwarding is more straightforward than traffic terminating on the router itself, I guess… bonus points for getting that right, too.)
> 
> I’m hoping that the limiting will work adequately so that the end-to-end path has adequate congestion avoidance happening, and that upstream doesn’t overrun the receiver and cause a lot of packets to be dropped on the last hop (work case of wasted bandwidth).  Not sure if I need special accommodations for bursting or if that would just delay the “settling” of congestion avoidance into steady-state.
> 
> Also not sure if ECN is worth marking at this point.  Congestion control is supposed to work better than congestion avoidance, right?
> 
> Anyone know what the steps would look like to accomplish the above?
> 
> A bunch of people responded, “yeah, I’ve been wanting to do that too…” when I brought up my question, so if I get a good solution I’ll submit a FAQ entry.
> 
> Thanks,
> 
> -Philip
> 

Re: Fwd: tc question about ingress bandwidth splitting

[Toke Høiland-Jørgensen]

Philip Prindeville <philipp_subx@redfish-solutions.com> writes:

> Had originally posted this to LARTC but realized that “netdev” is
> probably the better forum.
>
> Was hoping someone familiar with the nuts and bolts of tc and
> scheduler minutiae could help me come up with a configuration to use
> as a starting point, then I could tweak it, gather some numbers, make
> graphs etc, and write a LARTC or LWN article around the findings.
>
> I’d be trying to do shaping in both directions. Sure, egress shaping
> is trivial and obviously works.
>
> But I was also thinking about ingress shaping on the last hop, i.e. as
> traffic flows into the last-hop CPE router, and limiting/delaying it
> so that the entire end-to-end path is appropriately perceived by the
> sender, since the effective bandwidth of a [non-multipath] route is
> the min bandwidth of all individual hops, right?

Indeed, we have been using ingress shaping to combat bufferbloat for
years, and it works quite well (although you may have to set it a few %
lower than your actual line speed). There's even a separate mode in
sch_cake specifically for this purpose.

> So that min could be experienced at the final hop before the receiver
> as delay injected between packets to shape the bitrate.
>
> How far off-base am I?
>
> And what would some tc scripting look like to measure my thesis?

Take a look at sqm-scripts: https://github.com/tohojo/sqm-scripts

It's basically a collection of scripts to setup the kind of bandwidth
shaper you're talking about, with various configuration options. It
is packaged for OpenWrt, but you can also install it on a regular Linux
box.

Now, it doesn't specifically do the kind of guest/production split
you're talking about. However, it does have a script (simple.qos) that
does a three-tier shaping based on different DiffServ markings. If you
start from that, you should be able to change the classification and
bandwidth tiers to suit your purposes.

Having said that, however...

...Are you sure you really need to split bandwidth that way? Usually,
people do this because they don't want the 'guest' traffic to negatively
impact 'their own' usage of the network. But really, with a correctly
de-bloated link, this is much less of an issue than people think. And
with the per-host isolation feature of sch_cake[0], it becomes even less
so.

Not saying you are definitely wrong to pursue this kind of throttling of
your guest network, of course. Just encouraging you to keep an open mind
and test out the other feature first; you may find that it solves your
the problem well enough to be worth the decrease in complexity :)

-Toke


[0] See the section 'To enable Per-Host Isolation' here: https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details#making_cake_sing_and_dance_on_a_tight_rope_without_a_safety_net_aka_advanced_features