RFC: ethtool support for n-tuple filter programming

RFC: ethtool support for n-tuple filter programming

[Peter P Waskiewicz Jr]

All,

I'm looking to add support to ethtool that would allow programming of
full n-tuple filters into underlying devices.  Currently, ixgbe has
support for these types of perfect match or mostly match (masked)
filters.  I imagine other hardware exists that also has support for
this, so I'd like to make this interface usable for everyone.

Note that this is similar behavior in the iproute2 tools, but it's
different enough, in my opinion, to warrant being in ethtool.  The
iproute2 tools (specifically tc) manipulate the qdiscs to add filters in
the kernel packet schedulers.  This proposed solution is managing the
hardware in the underlying device, which iproute2 tools currently don't
touch.  Hopefully this is obvious for those reviewing this proposal.

What I currently have as possible inputs to ethtool are:

- src/dst IP address: 32-bits each, 128-bits each for IPv6
- src/dst port: 16-bits each (TCP/UDP)
- VLAN tag: 15-bits
- L4 type: 8-bits (TCP/UDP/SCTP currently, can grow later)
- User specified field: currently 32-bits, can be anything a driver
wants to use
- Action: signed 16-bits (-1 indicates drop, any other value is the Rx
queue to steer the flow to)

Now all of these fields, except action, can also have a mask supplied to
them, but it's not mandatory.

An example ethtool command with this support could be:

# ethtool -F ethX dst-ip 0x0101a8c0 src-ip 0x0001a8c0 0x00ffffff
dst-port 0x1600 src-port 0x0000 0x0000 usr 0x8906 act 5

This will program a filter that will filter traffic coming from
192.168.1.0/24 to 192.168.1.1, port 22, from any source port, and will
place all those matches packets into Rx queue 5.  It also specified a
user-defined field of 0x8906, which a driver can use at its own
discretion (or omit completely).

Then running the ethtool -f ethX command could dump all currently
programmed filters.

Any comments, thoughts, suggestions, or ideas are welcome.

Cheers,
-PJ Waskiewicz

Re: RFC: ethtool support for n-tuple filter programming

[Caitlin Bestler]

The approach you are proposing assumes what type of packet filters
that L2 hardware could support.

Why not simply use existing filtering rules that overshoot the target,
such as netfilter, and ask the
device specific tool to indicate what set of these rules it can support?


On Fri, Nov 6, 2009 at 10:57 AM, Peter P Waskiewicz Jr
<peter.p.waskiewicz.jr@intel.com> wrote:
> All,
>
> I'm looking to add support to ethtool that would allow programming of
> full n-tuple filters into underlying devices.  Currently, ixgbe has
> support for these types of perfect match or mostly match (masked)
> filters.  I imagine other hardware exists that also has support for
> this, so I'd like to make this interface usable for everyone.
>
> Note that this is similar behavior in the iproute2 tools, but it's
> different enough, in my opinion, to warrant being in ethtool.  The
> iproute2 tools (specifically tc) manipulate the qdiscs to add filters in
> the kernel packet schedulers.  This proposed solution is managing the
> hardware in the underlying device, which iproute2 tools currently don't
> touch.  Hopefully this is obvious for those reviewing this proposal.
>
> What I currently have as possible inputs to ethtool are:
>
> - src/dst IP address: 32-bits each, 128-bits each for IPv6
> - src/dst port: 16-bits each (TCP/UDP)
> - VLAN tag: 15-bits
> - L4 type: 8-bits (TCP/UDP/SCTP currently, can grow later)
> - User specified field: currently 32-bits, can be anything a driver
> wants to use
> - Action: signed 16-bits (-1 indicates drop, any other value is the Rx
> queue to steer the flow to)
>
> Now all of these fields, except action, can also have a mask supplied to
> them, but it's not mandatory.
>
> An example ethtool command with this support could be:
>
> # ethtool -F ethX dst-ip 0x0101a8c0 src-ip 0x0001a8c0 0x00ffffff
> dst-port 0x1600 src-port 0x0000 0x0000 usr 0x8906 act 5
>
> This will program a filter that will filter traffic coming from
> 192.168.1.0/24 to 192.168.1.1, port 22, from any source port, and will
> place all those matches packets into Rx queue 5.  It also specified a
> user-defined field of 0x8906, which a driver can use at its own
> discretion (or omit completely).
>
> Then running the ethtool -f ethX command could dump all currently
> programmed filters.
>
> Any comments, thoughts, suggestions, or ideas are welcome.
>
> Cheers,
> -PJ Waskiewicz
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: RFC: ethtool support for n-tuple filter programming

[Peter P Waskiewicz Jr]

On Fri, 2009-11-06 at 11:12 -0800, Caitlin Bestler wrote:
> The approach you are proposing assumes what type of packet filters
> that L2 hardware could support.
> 
> Why not simply use existing filtering rules that overshoot the target,
> such as netfilter, and ask the
> device specific tool to indicate what set of these rules it can support?

Are you proposing that netfilter is modified to pass the filters down to
the hardware if it supports it?  netfilter doesn't steer flows though to
queues (or flow ID's in the kernel), plus that's putting HW-specific
capabilities into netfilter.  I'm not sure we want to do that.

Please correct me if I'm wrong with interpreting your suggestion.

Thanks,
-PJ

> 
> 
> On Fri, Nov 6, 2009 at 10:57 AM, Peter P Waskiewicz Jr
> <peter.p.waskiewicz.jr@intel.com> wrote:
> > All,
> >
> > I'm looking to add support to ethtool that would allow programming of
> > full n-tuple filters into underlying devices.  Currently, ixgbe has
> > support for these types of perfect match or mostly match (masked)
> > filters.  I imagine other hardware exists that also has support for
> > this, so I'd like to make this interface usable for everyone.
> >
> > Note that this is similar behavior in the iproute2 tools, but it's
> > different enough, in my opinion, to warrant being in ethtool.  The
> > iproute2 tools (specifically tc) manipulate the qdiscs to add filters in
> > the kernel packet schedulers.  This proposed solution is managing the
> > hardware in the underlying device, which iproute2 tools currently don't
> > touch.  Hopefully this is obvious for those reviewing this proposal.
> >
> > What I currently have as possible inputs to ethtool are:
> >
> > - src/dst IP address: 32-bits each, 128-bits each for IPv6
> > - src/dst port: 16-bits each (TCP/UDP)
> > - VLAN tag: 15-bits
> > - L4 type: 8-bits (TCP/UDP/SCTP currently, can grow later)
> > - User specified field: currently 32-bits, can be anything a driver
> > wants to use
> > - Action: signed 16-bits (-1 indicates drop, any other value is the Rx
> > queue to steer the flow to)
> >
> > Now all of these fields, except action, can also have a mask supplied to
> > them, but it's not mandatory.
> >
> > An example ethtool command with this support could be:
> >
> > # ethtool -F ethX dst-ip 0x0101a8c0 src-ip 0x0001a8c0 0x00ffffff
> > dst-port 0x1600 src-port 0x0000 0x0000 usr 0x8906 act 5
> >
> > This will program a filter that will filter traffic coming from
> > 192.168.1.0/24 to 192.168.1.1, port 22, from any source port, and will
> > place all those matches packets into Rx queue 5.  It also specified a
> > user-defined field of 0x8906, which a driver can use at its own
> > discretion (or omit completely).
> >
> > Then running the ethtool -f ethX command could dump all currently
> > programmed filters.
> >
> > Any comments, thoughts, suggestions, or ideas are welcome.
> >
> > Cheers,
> > -PJ Waskiewicz
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: ethtool support for n-tuple filter programming

[Bill Fink]

On Fri, 06 Nov 2009, Peter P Waskiewicz Jr wrote:

> On Fri, 2009-11-06 at 11:12 -0800, Caitlin Bestler wrote:
> > The approach you are proposing assumes what type of packet filters
> > that L2 hardware could support.
> > 
> > Why not simply use existing filtering rules that overshoot the target,
> > such as netfilter, and ask the
> > device specific tool to indicate what set of these rules it can support?
> 
> Are you proposing that netfilter is modified to pass the filters down to
> the hardware if it supports it?  netfilter doesn't steer flows though to
> queues (or flow ID's in the kernel), plus that's putting HW-specific
> capabilities into netfilter.  I'm not sure we want to do that.
> 
> Please correct me if I'm wrong with interpreting your suggestion.

Plus I believe using netfilter has a significant performance penalty,
and it would be desirable to use such a feature without incurring
this penalty when there was otherwise no need to use netfilter.

						-Bill



> > On Fri, Nov 6, 2009 at 10:57 AM, Peter P Waskiewicz Jr
> > <peter.p.waskiewicz.jr@intel.com> wrote:
> > > All,
> > >
> > > I'm looking to add support to ethtool that would allow programming of
> > > full n-tuple filters into underlying devices.  Currently, ixgbe has
> > > support for these types of perfect match or mostly match (masked)
> > > filters.  I imagine other hardware exists that also has support for
> > > this, so I'd like to make this interface usable for everyone.
> > >
> > > Note that this is similar behavior in the iproute2 tools, but it's
> > > different enough, in my opinion, to warrant being in ethtool.  The
> > > iproute2 tools (specifically tc) manipulate the qdiscs to add filters in
> > > the kernel packet schedulers.  This proposed solution is managing the
> > > hardware in the underlying device, which iproute2 tools currently don't
> > > touch.  Hopefully this is obvious for those reviewing this proposal.
> > >
> > > What I currently have as possible inputs to ethtool are:
> > >
> > > - src/dst IP address: 32-bits each, 128-bits each for IPv6
> > > - src/dst port: 16-bits each (TCP/UDP)
> > > - VLAN tag: 15-bits
> > > - L4 type: 8-bits (TCP/UDP/SCTP currently, can grow later)
> > > - User specified field: currently 32-bits, can be anything a driver
> > > wants to use
> > > - Action: signed 16-bits (-1 indicates drop, any other value is the Rx
> > > queue to steer the flow to)
> > >
> > > Now all of these fields, except action, can also have a mask supplied to
> > > them, but it's not mandatory.
> > >
> > > An example ethtool command with this support could be:
> > >
> > > # ethtool -F ethX dst-ip 0x0101a8c0 src-ip 0x0001a8c0 0x00ffffff
> > > dst-port 0x1600 src-port 0x0000 0x0000 usr 0x8906 act 5
> > >
> > > This will program a filter that will filter traffic coming from
> > > 192.168.1.0/24 to 192.168.1.1, port 22, from any source port, and will
> > > place all those matches packets into Rx queue 5.  It also specified a
> > > user-defined field of 0x8906, which a driver can use at its own
> > > discretion (or omit completely).
> > >
> > > Then running the ethtool -f ethX command could dump all currently
> > > programmed filters.
> > >
> > > Any comments, thoughts, suggestions, or ideas are welcome.

Re: RFC: ethtool support for n-tuple filter programming

[Rick Jones]

Bill Fink wrote:
> On Fri, 06 Nov 2009, Peter P Waskiewicz Jr wrote:
> 
> 
>>On Fri, 2009-11-06 at 11:12 -0800, Caitlin Bestler wrote:
>>
>>>The approach you are proposing assumes what type of packet filters
>>>that L2 hardware could support.
>>>
>>>Why not simply use existing filtering rules that overshoot the target,
>>>such as netfilter, and ask the
>>>device specific tool to indicate what set of these rules it can support?
>>
>>Are you proposing that netfilter is modified to pass the filters down to
>>the hardware if it supports it?  netfilter doesn't steer flows though to
>>queues (or flow ID's in the kernel), plus that's putting HW-specific
>>capabilities into netfilter.  I'm not sure we want to do that.
>>
>>Please correct me if I'm wrong with interpreting your suggestion.
> 
> 
> Plus I believe using netfilter has a significant performance penalty,
> and it would be desirable to use such a feature without incurring
> this penalty when there was otherwise no need to use netfilter.

At the risk of typing words into someone's keyboard, I interpreted it as 
suggesting using the filtering language of netfilter or something similar, not 
necessarily netfilter itself?

rick jones

Re: RFC: ethtool support for n-tuple filter programming

[David Miller]

From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Date: Fri, 06 Nov 2009 10:57:21 -0800

> Any comments, thoughts, suggestions, or ideas are welcome.

We can use the existing datastructures and defines used for
ETHTOOL_GRX* for new ethtool commands that do filtering.

NIU can filter on these tuples too.

Re: RFC: ethtool support for n-tuple filter programming

[Peter P Waskiewicz Jr]

On Sat, 2009-11-07 at 21:27 -0700, David Miller wrote:
> From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
> Date: Fri, 06 Nov 2009 10:57:21 -0800
> 
> > Any comments, thoughts, suggestions, or ideas are welcome.
> 
> We can use the existing datastructures and defines used for
> ETHTOOL_GRX* for new ethtool commands that do filtering.
> 
> NIU can filter on these tuples too.

Are you suggesting to extend the flow hash stuff that is already in
ethtool (same ioctl)?  If so, that makes sense.  I'll go ahead and get
some patches together for review.

Thanks,
-PJ

Re: RFC: ethtool support for n-tuple filter programming

[David Miller]

From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Date: Sun, 08 Nov 2009 21:49:21 -0800

> On Sat, 2009-11-07 at 21:27 -0700, David Miller wrote:
>> From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
>> Date: Fri, 06 Nov 2009 10:57:21 -0800
>> 
>> > Any comments, thoughts, suggestions, or ideas are welcome.
>> 
>> We can use the existing datastructures and defines used for
>> ETHTOOL_GRX* for new ethtool commands that do filtering.
>> 
>> NIU can filter on these tuples too.
> 
> Are you suggesting to extend the flow hash stuff that is already in
> ethtool (same ioctl)?  If so, that makes sense.  I'll go ahead and get
> some patches together for review.

Not exactly.

Rather, I'm saying to use a new ethtool command, but make use of the
existing flow hash datastructures as much as possible.

Re: RFC: ethtool support for n-tuple filter programming

[Peter P Waskiewicz Jr]

On Sun, 2009-11-08 at 23:38 -0700, David Miller wrote:
> From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
> Date: Sun, 08 Nov 2009 21:49:21 -0800
> 
> > On Sat, 2009-11-07 at 21:27 -0700, David Miller wrote:
> >> From: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
> >> Date: Fri, 06 Nov 2009 10:57:21 -0800
> >> 
> >> > Any comments, thoughts, suggestions, or ideas are welcome.
> >> 
> >> We can use the existing datastructures and defines used for
> >> ETHTOOL_GRX* for new ethtool commands that do filtering.
> >> 
> >> NIU can filter on these tuples too.
> > 
> > Are you suggesting to extend the flow hash stuff that is already in
> > ethtool (same ioctl)?  If so, that makes sense.  I'll go ahead and get
> > some patches together for review.
> 
> Not exactly.
> 
> Rather, I'm saying to use a new ethtool command, but make use of the
> existing flow hash datastructures as much as possible.

Ah ok.  Makes even more sense.

Thanks David.

-PJ

Re: RFC: ethtool support for n-tuple filter programming

[Caitlin Bestler]

On Sat, Nov 7, 2009 at 3:28 PM, Rick Jones <rick.jones2@hp.com> wrote:
>
> At the risk of typing words into someone's keyboard, I interpreted it as
> suggesting using the filtering language of netfilter or something similar,
> not necessarily netfilter itself?
>

Correct, a netfilter-friendly interface to the driver could be invoked by
lower-overhead entities that netfilter and the driver would not care.

However the real goal would be to still use netfilter, which would become
a low-overhead entity if it could delegate 90% of the rules it enforced to
smart hardware.

The fundamental suggestion is to start with a filter specification that is
clearly too rich for any Ethernet device, and let the Ethernet devices
decide how quickly they want to catch up. As opposed to standardizing
how smart a smart Ethernet device is and potentially leaving some hardware
capabilities made inaccessible.

I'll point out that once you assume an Ethernet Device is capable of doing
TCP/UDP checksum offload and LSO/LRO then clearly you have recognized
that it is an L4 aware device. Designing its filtering rules as though it were
an L2-only device does not allow it to take advantage of the L4 parsing that
many/most Ethernet NICs already do.

Re: RFC: ethtool support for n-tuple filter programming

[Patrick McHardy]

Caitlin Bestler wrote:
> On Sat, Nov 7, 2009 at 3:28 PM, Rick Jones <rick.jones2@hp.com> wrote:
>> At the risk of typing words into someone's keyboard, I interpreted it as
>> suggesting using the filtering language of netfilter or something similar,
>> not necessarily netfilter itself?
>>
> 
> Correct, a netfilter-friendly interface to the driver could be invoked by
> lower-overhead entities that netfilter and the driver would not care.
> 
> However the real goal would be to still use netfilter, which would become
> a low-overhead entity if it could delegate 90% of the rules it enforced to
> smart hardware.

That's not possible in a compatible fashion with ip_tables because of
counters, logging, rules affecting traffic from multiple interfaces,
targets not supported in hardware (which I presume will simply be
"DROP") etc.

Counters are actually the worst feature standing in the way of this,
but even without them, you could usually only offload the first n
dropping rules that don't use any features not supported in hardware
and only affect the specific interface. Any "ACCEPT" rule is most
likely followed by further drop rules, so packets actually need to
hit those rules in software to exit table processing.

It gets even worse if you consider ingress TC actions directing the
packets to different interfaces or mangling them.

> The fundamental suggestion is to start with a filter specification that is
> clearly too rich for any Ethernet device, and let the Ethernet devices
> decide how quickly they want to catch up. As opposed to standardizing
> how smart a smart Ethernet device is and potentially leaving some hardware
> capabilities made inaccessible.
> 
> I'll point out that once you assume an Ethernet Device is capable of doing
> TCP/UDP checksum offload and LSO/LRO then clearly you have recognized
> that it is an L4 aware device. Designing its filtering rules as though it were
> an L2-only device does not allow it to take advantage of the L4 parsing that
> many/most Ethernet NICs already do.