Problem using tc-sfq with tc-flow for arbitrary hashing

Problem using tc-sfq with tc-flow for arbitrary hashing

[Carmelo Cascone]

Hi all,

I have a problem in configuring tc-sfq along with tc-flow. I would like to hash packets not using the UDP/TCP 5-tuple (default beahvior of tc-sfq), but simply the pair <ipsrc, ipdst>. I’m having hard time using tc-flow and finding help on the web, as such I decided to drop an email to this list.

The commands and error I get are:

$ tc qdisc add dev eth6 root handle 1: sfq
$ tc filter add dev eth6 parent 1:0 protocol ip prio 1 flow hash keys src,dst perturb 30 divisor 1024
RTNETLINK answers: Invalid argument
We have an error talking to the kernel

I’m issuing these commands on a Debian 9 with a kernel v4.9.16
Please note that the tc filter command is the same used in the tc-sfq documentation.

What am I doing wrong? What is the invalid argument?

Thanks in advance for your help.

Carmelo Cascone
—
http://ccascone.net

Re: Problem using tc-sfq with tc-flow for arbitrary hashing

[Andy Furniss]

Carmelo Cascone wrote:
> Hi all,
>
> I have a problem in configuring tc-sfq along with tc-flow. I would like to hash packets not using the UDP/TCP 5-tuple (default beahvior of tc-sfq), but simply the pair <ipsrc, ipdst>. I’m having hard time using tc-flow and finding help on the web, as such I decided to drop an email to this list.
>
> The commands and error I get are:
>
> $ tc qdisc add dev eth6 root handle 1: sfq
> $ tc filter add dev eth6 parent 1:0 protocol ip prio 1 flow hash keys src,dst perturb 30 divisor 1024
> RTNETLINK answers: Invalid argument
> We have an error talking to the kernel
>
> I’m issuing these commands on a Debian 9 with a kernel v4.9.16
> Please note that the tc filter command is the same used in the tc-sfq documentation.
>
> What am I doing wrong? What is the invalid argument?

Looking through many years old Qos folder I found a file with an example.

I don't know why it's not documented, but but adding handle 1 to the
filter command makes your example work for me, in the sense that it's
not rejected - didn't test further.

Historically I would have said putting sfq on root of an eth wouldn't
do much, but with BQLs maybe things have changed.

Re: Problem using tc-sfq with tc-flow for arbitrary hashing

[Carmelo Cascone]

Andy,

Thank you very much for the hint. The command is now accepted.

Unfortunately, I see some strange behaviors when using tc-flow, in the sense that sometimes the bandwidth allocation on a 10Gb/s NIC is far form being fair. It looks like the whole bandwidth is capitalized by a single flow, even if I use a small number of flows, even if I wait a couple of perturbation intervals.
Some other times it works perfectly (apart for throughput degradation, ~8.5Gb/s). Not sure what the cause is, but if I use tc-sfq alone without tc-flow iI don’t see this behavior, it always works as expected.

I will test further, but if anyone as some hint, that is appreciated.

Thanks,
Carmelo

> On Mar 28, 2017, at 18:07, Andy Furniss <adf.lists@gmail.com> wrote:
> 
> Carmelo Cascone wrote:
>> Hi all,
>> 
>> I have a problem in configuring tc-sfq along with tc-flow. I would like to hash packets not using the UDP/TCP 5-tuple (default beahvior of tc-sfq), but simply the pair <ipsrc, ipdst>. I’m having hard time using tc-flow and finding help on the web, as such I decided to drop an email to this list.
>> 
>> The commands and error I get are:
>> 
>> $ tc qdisc add dev eth6 root handle 1: sfq
>> $ tc filter add dev eth6 parent 1:0 protocol ip prio 1 flow hash keys src,dst perturb 30 divisor 1024
>> RTNETLINK answers: Invalid argument
>> We have an error talking to the kernel
>> 
>> I’m issuing these commands on a Debian 9 with a kernel v4.9.16
>> Please note that the tc filter command is the same used in the tc-sfq documentation.
>> 
>> What am I doing wrong? What is the invalid argument?
> 
> Looking through many years old Qos folder I found a file with an example.
> 
> I don't know why it's not documented, but but adding handle 1 to the
> filter command makes your example work for me, in the sense that it's
> not rejected - didn't test further.
> 
> Historically I would have said putting sfq on root of an eth wouldn't
> do much, but with BQLs maybe things have changed.
> 
> 

Re: Problem using tc-sfq with tc-flow for arbitrary hashing

[Andy Furniss]

Carmelo Cascone wrote:
> Andy,
>
> Thank you very much for the hint. The command is now accepted.
>
> Unfortunately, I see some strange behaviors when using tc-flow, in
> the sense that sometimes the bandwidth allocation on a 10Gb/s NIC is
> far form being fair. It looks like the whole bandwidth is capitalized
> by a single flow, even if I use a small number of flows, even if I
> wait a couple of perturbation intervals. Some other times it works
> perfectly (apart for throughput degradation, ~8.5Gb/s). Not sure what
> the cause is, but if I use tc-sfq alone without tc-flow iI don’t see
> this behavior, it always works as expected.
>
> I will test further, but if anyone as some hint, that is
> appreciated.

I have no experience with 10gig nics, but years ago putting sfq on a
lowly 100mbit nic didn't work well due to it just sitting before a large
fifo.

Things may be different now with BQLs if enabled - but I have never tested.

Random thoughts -

10 gig nic is likely to do some hardware offload, so the kernel may be
sending huge packets out for locally generated traffic vs smaller for
routed.

ethtool -k will show -K to control - but turning off on 10 gig will I
imagine be expensive CPU wise.

Local traffic may limit its self vs routed with tcp pacing.

SFQ default qlen is only 127 IIRC - which is small at such high rates.

SFQ is possibly not coming into play until a larger queue fills up.