SFQ + speed caps

SFQ + speed caps

[Dimitris Kotsonis]

Hi everybody,

I am trying to find a solution to the following problem. I have a lan 
where pcs get a fair amount of the internet connection from a linux 
router. In order to do this I am using SFQ with "flow map key dst and 
0xff". This way each pc gets its own SFQ band. This setup is completely 
automatic and works exceptionally well. Two active pcs will get 50% of 
the line, three active pcs will get 33% of the line etc.

Now here comes the problem. What if in addition to a fair distribution 
of the connection I wanted to cap the speed of one pc to a specified 
hard limit. I can do something similar to this...

interface
  |
  +---- HTB --- class 1 (capped pc)
         |
         +----- class 2  (everything else)
                  |
                  +--- SFQ

  .. but this way I am getting the capped pc out of the SFQ round robin 
allowing it to monopolize the line up to its hard limit and in excess of 
what is currently fair. If, for example, the capped pc speed is set at 
40% of the line speed and there are 5 active pcs on the lan then the 
capped pc will managed to steal more than it's fair share of the line. 
It could manage to go all the way up to 40% instead of 20%.

It appears that HTB must be placed on top of SFQ for this to work:

interface
  |
+--- SFQ
         |
+---- HTB --- class 1 (capped pc)
                |
                +----- class 2  (everything else)

Only problem is, you can't attach a qdisc on top of another qdisc. You 
can only attach a qdisc on top of a class and SFQ is a classless qdisc.

So is there a solution to my problem ? Can you shape some flows 
dynamically and in addition to that shape one of those flows statically 
? I am shifting through the LARTC documentation trying to figure some 
way to do this but I can't think of any.

Any thoughts ?

Regards,

Dimitris.

Re: SFQ + speed caps

[Alan Goodman]

>  .. but this way I am getting the capped pc out of the SFQ round robin 
> allowing it to monopolize the line up to its hard limit and in excess 
> of what is currently fair. If, for example, the capped pc speed is set 
> at 40% of the line speed and there are 5 active pcs on the lan then 
> the capped pc will managed to steal more than it's fair share of the 
> line. It could manage to go all the way up to 40% instead of 20%.
>
> It appears that HTB must be placed on top of SFQ for this to work:
>
> interface
>  |
> +--- SFQ
>         |
> +---- HTB --- class 1 (capped pc)
>                |
>                +----- class 2  (everything else)
>
> Only problem is, you can't attach a qdisc on top of another qdisc. You 
> can only attach a qdisc on top of a class and SFQ is a classless qdisc.

I believe that this is correct.  You cant feed out of sfq into any 
classfull qdisc.  Your simple SFQ based fairness will also very probably 
break badly if someone sends or receives small packets too...  The 
person sending the larger packets will get a much larger share than the 
person sending smaller packets...

Using htb you could have each user hitting a separate class, each with a 
RATE and a common CEIL so that they can all burst if the line isnt 
contended...  Then feed each of those that info BFIFO with limit set 
just over 2x MTU sized packets (if the connection is relatively slow).  
Alternatively if your O/S has it check out fq_codel.

Alan

Re: SFQ + speed caps

[Dimitris Kotsonis]

>
> Using htb you could have each user hitting a separate class, each with 
> a RATE and a common CEIL so that they can all burst if the line isnt 
> contended...  Then feed each of those that info BFIFO with limit set 
> just over 2x MTU sized packets (if the connection is relatively 
> slow).  Alternatively if your O/S has it check out fq_codel.
>
I could do that but for a class C lan I would have to create 256 
seperate classes and 256 separate filters. Each for every ip. Whereas in 
SFQ you only have to specify that the hashing algorithm should work on 
the dst field & 0xFF and everything works automatically. For more than 
256 ips things will get really clattered. Also I wonder if speed is an 
issue here with so many rules.

I haven't heard about fq_codel. I will give it a look.

Thanks for the suggestions.

Regards,

Dimitris

Re: SFQ + speed caps

[Dimitris Kotsonis]

On 8/7/2014 3:42 PM, Remy Mudingay wrote:
> HI Dimitris,
>
> The solution is straight forward :
>
> interface
>   |
>   +---- HTB --- class 1 parent class
>          |
>          +----- HTB class 2  (everything else) 90% guaranteed/ 100% cap
>          |        |
>          |       +--- SFQ
>          |
>          +----- HTB class 3 (capped pc) 10% guaranteed/ 20% cap
>

Hi Remy,

 From what I can tell what you posted is identical to the hypothetical 
scenario that I said will not work.

Here is the problem with it. Let's say that there are 20 pcs battling 
for bandwidth in class 2. SFQ will guarantee that each will get 
approximately 1/20th of the bandwidth. The capped pc however is 
unfettered by the SFQ round robin so it can potentially consume the 
entire chunk allocated to it by HTB in class 3, which is more than 1/20. 
So the capped pc can potentially (e.g. using a large number of sockets) 
get more bandwidth than the uncapped pcs.

So what I am looking for is for a way to implement a global SFQ instead 
of one for each HTB class. For a way to place the entire HTB logic on 
top of a single SFQ.

Regards,

Dimitris

Re: SFQ + speed caps

[Dimitris Kotsonis]

On 8/7/2014 6:16 PM, Remy Mudingay wrote:
> Hi Dimitris,
>
> I disagree. Firstly, if the total number of PC's on your LAN is 30, 
> then set the rate (gurantee) of class 3 (capped PC) to 1/30 (3.3%).

But the whole point is that I don't know how many PCs are active in the 
lan at any given point. When I am saying that 30 PCs use the line what I 
am saying is that 30 PCS are battling for bandwidth right this instant. 
I am not saying that 30 PCs are sitting behind the router. A few seconds 
later there could be 10 PCs, or 50. For example, if a minute later only 
3 PCs are active then the capped PC should be allotted 1/3 of the 
bandwidth in class 3 (up from 1/30 it was a minute ago).

Remember that SFQ (hashed on ips instead of flows) can adjust itself 
automatically to the number of currently active PCs and how much 
bandwidth each should receive. This is not the case with HTB where 
percentages have to be plugged in beforehand.

I hope this makes it clear (and I have not misunderstood what you said)

Regards,

Dimitris

Re: SFQ + speed caps

[Andy Furniss]

Dimitris Kotsonis wrote:
>
>>
>> Using htb you could have each user hitting a separate class, each
>> with a RATE and a common CEIL so that they can all burst if the
>> line isnt contended...  Then feed each of those that info BFIFO
>> with limit set just over 2x MTU sized packets (if the connection is
>> relatively slow).  Alternatively if your O/S has it check out
>> fq_codel.
>>
> I could do that but for a class C lan I would have to create 256
> seperate classes and 256 separate filters. Each for every ip. Whereas
> in SFQ you only have to specify that the hashing algorithm should
> work on the dst field & 0xFF and everything works automatically. For
> more than 256 ips things will get really clattered. Also I wonder if
> speed is an issue here with so many rules.

I don't think 256 is that much depending on h/w of course, You can hash
filters and using htb would let you do more eg. give priority to small
packets per user.

If you really want sfq, then to limit one user and go onto one sfq I
think you would need to look into ifb so you could double process some
packets.