BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[Daniel Lakeland]

This same problem as detailed here

http://lists.openwall.net/netdev/2010/03/26/36

or here:

https://www.spinics.net/lists/lartc/msg22541.html

bit me today

I tried either

ip rule add dsfield CS6 table 100

or

ip rule add dsfield 0xc0 table 100

or replace dsfield with tos, all return:

RTNETLINK answers: Invalid argument

on the other hand, for ip route it will accept the ds/tos values

ip route add default dsfield CS6 dev dummy0

or

ip route add default dsfield 0xc0 dev dummy0

but packets tagged with CS6 don't go to dummy0 they go the regular 
default route

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[Stephen Hemminger]

On Wed, 13 Dec 2017 09:40:08 -0800
Daniel Lakeland <dlakelan@street-artists.org> wrote:

> This same problem as detailed here
> 
> http://lists.openwall.net/netdev/2010/03/26/36

This mail reports an issue from 7 years ago, much nas
changed since then.

> 
> or here:
> 
> https://www.spinics.net/lists/lartc/msg22541.html
> 
> bit me today
> 
> I tried either
> 
> ip rule add dsfield CS6 table 100
> 
> or
> 
> ip rule add dsfield 0xc0 table 100
> 
> or replace dsfield with tos, all return:
> 
> RTNETLINK answers: Invalid argument
> 
> on the other hand, for ip route it will accept the ds/tos values
> 
> ip route add default dsfield CS6 dev dummy0
> 
> or
> 
> ip route add default dsfield 0xc0 dev dummy0
> 
> but packets tagged with CS6 don't go to dummy0 they go the regular 
> default route
> 
> 

The kernel is complaining that ip rule is not valid, (ie not iproute2 issue).
Not sure exactly why or where in fib_rules.c this is happening.

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[Daniel Lakeland]

On December 13, 2017 10:12:59 AM PST, Stephen Hemminger <stephen@networkplumber.org> wrote:
>On Wed, 13 Dec 2017 09:40:08 -0800
>Daniel Lakeland <dlakelan@street-artists.org> wrote:
>
>> This same problem as detailed here
>> 
>> http://lists.openwall.net/netdev/2010/03/26/36
>
>This mail reports an issue from 7 years ago, much nas
>changed since then.
>

I figure it's still biting me because no one reported it. This is with modern Debian testing system.
 
>
>The kernel is complaining that ip rule is not valid, (ie not iproute2
>issue).

Note that like some of those other people I was able to get ip rule to accept tos values with just low order bits set... On my phone now so can't test an example but it was a tos like 0x0c or something.

I'm really not familiar with internals or who's in charge of what I just wanted to be sure this issue hit some kernel netdev people's radar instead of dropping on the floor!!

>Not sure exactly why or where in fib_rules.c this is happening.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[Daniel Lakeland]

Following up my previous email with output from the machine:
Note that like some of those other people I was able to get ip rule to 
accept tos values with just low order bits set

Here is example of how ip rule accepts low order dsfield bits but not modern DSCP type bits, also including some version info

dlakelan@pingpong:~$ sudo ip rule add dsfield 0x0c table 100
dlakelan@pingpong:~$ ip rule show
0:	from all lookup local
32765:	from all tos 0x0c lookup 100
32766:	from all lookup main
32767:	from all lookup default


dlakelan@pingpong:~$ sudo ip rule add dsfield 0xc0 table 100
RTNETLINK answers: Invalid argument


dlakelan@pingpong:~$ cat /proc/version
Linux version 4.12.0-1-amd64 (debian-kernel@lists.debian.org) (gcc version 6.4.0 20170805 (Debian 6.4.0-3) ) #1 SMP Debian 4.12.6-1 (2017-08-12)


dlakelan@pingpong:~$ ip -V
ip utility, iproute2-ss160518

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[David Ahern]

On 12/13/17 12:05 PM, Daniel Lakeland wrote:
> Following up my previous email with output from the machine:
> Note that like some of those other people I was able to get ip rule to
> accept tos values with just low order bits set
> 
> Here is example of how ip rule accepts low order dsfield bits but not
> modern DSCP type bits, also including some version info
> 
> dlakelan@pingpong:~$ sudo ip rule add dsfield 0x0c table 100
> dlakelan@pingpong:~$ ip rule show
> 0:    from all lookup local
> 32765:    from all tos 0x0c lookup 100
> 32766:    from all lookup main
> 32767:    from all lookup default
> 
> 
> dlakelan@pingpong:~$ sudo ip rule add dsfield 0xc0 table 100
> RTNETLINK answers: Invalid argument

In fib4_rule_configure, this the check that is failing:

    if (frh->tos & ~IPTOS_TOS_MASK)
        goto errout;

and EINVAL is returned.

IPv4 routes has not checking on tos -- it is passed from user and
rtm_tos to fc_tos to fib alias tos.

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[Daniel Lakeland]

On 12/13/2017 02:40 PM, David Ahern wrote:
>
> In fib4_rule_configure, this the check that is failing:
>
>      if (frh->tos & ~IPTOS_TOS_MASK)
>          goto errout;
>
> and EINVAL is returned.
>
> IPv4 routes has not checking on tos -- it is passed from user and
> rtm_tos to fc_tos to fib alias tos.

it seems to me that this IPTOS_TOS_MASK check should be either gotten 
rid of, or equal to 0x03 in modern usage. The bottom 2 bits are ECN and 
I suppose someone might want to route based on congestion... and hence 
maybe the mask should be dropped entirely, but if you refuse to allow 
routes on ECN then you'd want 0x03 as the mask

it seems to me this is left over from before DSCP.

apparently most people don't route on DSCP or work around this with 
firewall marks, and so this doesn't cause trouble enough to have been 
reported before?

I think the follow up question is does anyone have any idea why someone 
who set up routes with dsfield settings is not seeing packets routed? 
The kernel may not handle ip rule with DSCP, but it takes

ip route add default dsfield CS6 dev veth0

just fine... and shows up in the route table, but for example the person 
is not seeing CS6 marked packets going to veth2 and instead is seeing 
them routed to veth0 the default route...

Re: BUG REPORT: iproute2 seems to have bug with dsfield/tos in ip-rule and ip-route

[David Ahern]

On 12/13/17 3:52 PM, Daniel Lakeland wrote:
> On 12/13/2017 02:40 PM, David Ahern wrote:
>>
>> In fib4_rule_configure, this the check that is failing:
>>
>>      if (frh->tos & ~IPTOS_TOS_MASK)
>>          goto errout;
>>
>> and EINVAL is returned.
>>
>> IPv4 routes has not checking on tos -- it is passed from user and
>> rtm_tos to fc_tos to fib alias tos.
> 
> it seems to me that this IPTOS_TOS_MASK check should be either gotten
> rid of, or equal to 0x03 in modern usage. The bottom 2 bits are ECN and
> I suppose someone might want to route based on congestion... and hence
> maybe the mask should be dropped entirely, but if you refuse to allow
> routes on ECN then you'd want 0x03 as the mask
> 
> it seems to me this is left over from before DSCP.
> 
> apparently most people don't route on DSCP or work around this with
> firewall marks, and so this doesn't cause trouble enough to have been
> reported before?
> 
> I think the follow up question is does anyone have any idea why someone
> who set up routes with dsfield settings is not seeing packets routed?
> The kernel may not handle ip rule with DSCP, but it takes
> 
> ip route add default dsfield CS6 dev veth0
> 
> just fine... and shows up in the route table, but for example the person
> is not seeing CS6 marked packets going to veth2 and instead is seeing
> them routed to veth0 the default route...
> 
> 

If you are running a modern kernel (>= ~4.5) there are fib tracepoints
you can use to try to answer that:

   perf record -e fib:fib_table_lookup -a -g
   perf script [-G]

I've had some doubts about tos handling in the output path but have not
had the time (or motivation) to dig into it. Specifically, the tos
adjustments in ip_route_output_key_hash look weird to me.