hw offload for new protocols

hw offload for new protocols

[Michal Swiatkowski]

Hi,

I would like to add matching on values from protocol that isn't yet
supported in kernel, but my hw has abilitty to offload it (session id
in pfcp for example).
What is a correct way to implement it in kernel? I was searching on ML
for threads about that but I didn't find answer to all my concerns.

I assume that for hw offload we should reuse tc flower, which already
has great ability to offload bunch of widely used protocols. To match on
my session id value I will for sure have to add another field in tc
(user and kernel part). Something like this:
#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
session_id 0102030405060708 action drop

Should SW path be also supported? I think that yes, so, this will
need adding handling this new field in flow_dissector. I have read
thread with adding new field to it [1] and my feeling from it is: better
do not add new fields there :) . However, if it is fine to expand
flow_dissector, how to do it in this particular case? Can I check udp
port in flow dissector code and based on that dissect session id from
pfcp header? Won't this lead to a lot of new code for each different
protocols based on well known port numbers?

What about $DEV from tc command? In hw offload for example for VXLAN or
geneve based on this hw knows what type of flow should be offloaded. It
will be great to have the same ability for pfcp (in my case), to allow
adding rule without pfcp specific fileds:
#tc filter add dev $PFCP_DEV protocol ip parent ffff: flower dst_ip $IP
action drop
Or maybe in this kind of flows we should always add in tc flower correct
port number which will tell hw that this flow is pfcp?
#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
enc_dst_port $well_know_pfcp action drop

If creating new netdev (pfcp in this case) is fine solution, how pfcp
driver should look like? Is code for receiving and xmit sufficient? Or
is there need to implement more pfcp features in the new driver? To not
have sth like dummy pfcp driver only to point to it in tc. There was
review with virtual netdev [2] - which drops every packet that returns from
classyfing (I assume not offloaded by hw). Maybe this solution is
better?

I have also seen panda (flower 2) [3]. It isn' available in kernel now.
Do we know timeline for this feature? From review discussion I don't
know if it allow offloading cases like my in hw which wasn't design to
support panda offload.

I feel like I can solve all my concerns using u32 classifier (but I can
be wrong). I thought about creating user space app that will translate
human readable command to u32. Hw will try to offload u32 command if
given flow can be offloaded, if not software path will work as usally. I
have seen that few drivers support u32 offload, but it looks like the
code is from before creation of flower classifier. Do You know if
someone try this combination (user app + u32 + hw offload)?

I am talking about pfcp, but there is few more protocols that hw can
offload, but lack of support in flow dissector is successfully
complicating hw offload.

Thanks for any comments about this topic,
Michal


[1] https://lore.kernel.org/netdev/20210830080800.18591-1-boris.sukholitko@broadcom.com/
[2] https://lore.kernel.org/netdev/20210929094514.15048-1-tparkin@katalix.com/
[3] https://lore.kernel.org/netdev/20210916200041.810-1-felipe@expertise.dev/

Re: hw offload for new protocols

[Jiri Pirko]

Wed, Feb 09, 2022 at 09:45:06AM CET, michal.swiatkowski@linux.intel.com wrote:
>Hi,
>
>I would like to add matching on values from protocol that isn't yet
>supported in kernel, but my hw has abilitty to offload it (session id
>in pfcp for example).
>What is a correct way to implement it in kernel? I was searching on ML
>for threads about that but I didn't find answer to all my concerns.
>
>I assume that for hw offload we should reuse tc flower, which already
>has great ability to offload bunch of widely used protocols. To match on
>my session id value I will for sure have to add another field in tc
>(user and kernel part). Something like this:
>#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
>session_id 0102030405060708 action drop
>
>Should SW path be also supported? I think that yes, so, this will

Yes.


>need adding handling this new field in flow_dissector. I have read

Correct.


>thread with adding new field to it [1] and my feeling from it is: better
>do not add new fields there :) . However, if it is fine to expand
>flow_dissector, how to do it in this particular case? Can I check udp
>port in flow dissector code and based on that dissect session id from
>pfcp header? Won't this lead to a lot of new code for each different
>protocols based on well known port numbers?

I don't think it is good idea to base the flow dissector branch decision
on a well known UDP port.


>
>What about $DEV from tc command? In hw offload for example for VXLAN or
>geneve based on this hw knows what type of flow should be offloaded. It
>will be great to have the same ability for pfcp (in my case), to allow
>adding rule without pfcp specific fileds:
>#tc filter add dev $PFCP_DEV protocol ip parent ffff: flower dst_ip $IP
>action drop

Yes, I agree.


>Or maybe in this kind of flows we should always add in tc flower correct
>port number which will tell hw that this flow is pfcp?
>#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
>enc_dst_port $well_know_pfcp action drop
>
>If creating new netdev (pfcp in this case) is fine solution, how pfcp
>driver should look like? Is code for receiving and xmit sufficient? Or
>is there need to implement more pfcp features in the new driver? To not
>have sth like dummy pfcp driver only to point to it in tc. There was
>review with virtual netdev [2] - which drops every packet that returns from
>classyfing (I assume not offloaded by hw). Maybe this solution is
>better?

Not sure how it fits on PFCP.


>
>I have also seen panda (flower 2) [3]. It isn' available in kernel now.
>Do we know timeline for this feature? From review discussion I don't
>know if it allow offloading cases like my in hw which wasn't design to
>support panda offload.
>
>I feel like I can solve all my concerns using u32 classifier (but I can
>be wrong). I thought about creating user space app that will translate
>human readable command to u32. Hw will try to offload u32 command if
>given flow can be offloaded, if not software path will work as usally. I
>have seen that few drivers support u32 offload, but it looks like the
>code is from before creation of flower classifier. Do You know if
>someone try this combination (user app + u32 + hw offload)?
>
>I am talking about pfcp, but there is few more protocols that hw can
>offload, but lack of support in flow dissector is successfully
>complicating hw offload.
>
>Thanks for any comments about this topic,
>Michal
>
>
>[1] https://lore.kernel.org/netdev/20210830080800.18591-1-boris.sukholitko@broadcom.com/
>[2] https://lore.kernel.org/netdev/20210929094514.15048-1-tparkin@katalix.com/
>[3] https://lore.kernel.org/netdev/20210916200041.810-1-felipe@expertise.dev/

Re: hw offload for new protocols

[Michal Swiatkowski]

On Wed, Feb 09, 2022 at 03:12:27PM +0100, Jiri Pirko wrote:
> Wed, Feb 09, 2022 at 09:45:06AM CET, michal.swiatkowski@linux.intel.com wrote:
> >Hi,
> >
> >I would like to add matching on values from protocol that isn't yet
> >supported in kernel, but my hw has abilitty to offload it (session id
> >in pfcp for example).
> >What is a correct way to implement it in kernel? I was searching on ML
> >for threads about that but I didn't find answer to all my concerns.
> >
> >I assume that for hw offload we should reuse tc flower, which already
> >has great ability to offload bunch of widely used protocols. To match on
> >my session id value I will for sure have to add another field in tc
> >(user and kernel part). Something like this:
> >#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
> >session_id 0102030405060708 action drop
> >
> >Should SW path be also supported? I think that yes, so, this will
> 
> Yes.
> 
> 
> >need adding handling this new field in flow_dissector. I have read
> 
> Correct.
> 
> 
> >thread with adding new field to it [1] and my feeling from it is: better
> >do not add new fields there :) . However, if it is fine to expand
> >flow_dissector, how to do it in this particular case? Can I check udp
> >port in flow dissector code and based on that dissect session id from
> >pfcp header? Won't this lead to a lot of new code for each different
> >protocols based on well known port numbers?
> 
> I don't think it is good idea to base the flow dissector branch decision
> on a well known UDP port.
> 
> 
Ok, will it be better to base it on next header like in VXLAN (and
other tunnels)? But this will need setting tunel info. PFCP isn't really
tunnel. Maybe it can be treat as tunnel without tunneled data?

> >
> >What about $DEV from tc command? In hw offload for example for VXLAN or
> >geneve based on this hw knows what type of flow should be offloaded. It
> >will be great to have the same ability for pfcp (in my case), to allow
> >adding rule without pfcp specific fileds:
> >#tc filter add dev $PFCP_DEV protocol ip parent ffff: flower dst_ip $IP
> >action drop
> 
> Yes, I agree.
> 
> 
> >Or maybe in this kind of flows we should always add in tc flower correct
> >port number which will tell hw that this flow is pfcp?
> >#tc filter add dev $DEV protocol ip parent ffff: flower dst_ip $IP
> >enc_dst_port $well_know_pfcp action drop
> >
> >If creating new netdev (pfcp in this case) is fine solution, how pfcp
> >driver should look like? Is code for receiving and xmit sufficient? Or
> >is there need to implement more pfcp features in the new driver? To not
> >have sth like dummy pfcp driver only to point to it in tc. There was
> >review with virtual netdev [2] - which drops every packet that returns from
> >classyfing (I assume not offloaded by hw). Maybe this solution is
> >better?
> 
> Not sure how it fits on PFCP.
> 
> 
> >
> >I have also seen panda (flower 2) [3]. It isn' available in kernel now.
> >Do we know timeline for this feature? From review discussion I don't
> >know if it allow offloading cases like my in hw which wasn't design to
> >support panda offload.
> >
> >I feel like I can solve all my concerns using u32 classifier (but I can
> >be wrong). I thought about creating user space app that will translate
> >human readable command to u32. Hw will try to offload u32 command if
> >given flow can be offloaded, if not software path will work as usally. I
> >have seen that few drivers support u32 offload, but it looks like the
> >code is from before creation of flower classifier. Do You know if
> >someone try this combination (user app + u32 + hw offload)?
> >
> >I am talking about pfcp, but there is few more protocols that hw can
> >offload, but lack of support in flow dissector is successfully
> >complicating hw offload.
> >
> >Thanks for any comments about this topic,
> >Michal
> >
> >
> >[1] https://lore.kernel.org/netdev/20210830080800.18591-1-boris.sukholitko@broadcom.com/
> >[2] https://lore.kernel.org/netdev/20210929094514.15048-1-tparkin@katalix.com/
> >[3] https://lore.kernel.org/netdev/20210916200041.810-1-felipe@expertise.dev/