XDP redirect - ping fail

XDP redirect - ping fail

[team lnx]

Hello,

I see a ping not working in below topology with xdp redirect


IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4

IFACE2 and IFACE3 are on same host machine

IFACE1 and IFACE2 are  on static ip
IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server

steps
1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
3. Run xdp redirect on IFACE2 and IFACE3
4. start ping from IFACE1 to IFACE4

Are some of the assumptions not right here leading to ping failure ?

Regards,
teamlnx

Re: XDP redirect - ping fail

[Shaw, Jeffrey B]

On 5/4/2023 10:38 AM, team lnx wrote:
> Hello,
> 
> I see a ping not working in below topology with xdp redirect
> 
> 
> IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
> 
> IFACE2 and IFACE3 are on same host machine
> 
> IFACE1 and IFACE2 are  on static ip
> IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
> 
> steps
> 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> 3. Run xdp redirect on IFACE2 and IFACE3
> 4. start ping from IFACE1 to IFACE4

Does IFACE1 have a route to 192.168.2.x and is the next hop resolved? 
You can use `tcpdump` on IFACE1 and IFACE4 to see what packets are sent 
and received.

XDP-REDIRECT cpu affinity

[Sanjay Rao]

I am looking at the implementation of ndo_xdp_xmit() in
mlx5/core/en/xdp.c (most other drivers also have similar
implementations). This function is called for XDP-REDIRECT.

int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame
**frames, u32 flags)
{
   ------
   ------

   sq_num = smp_processor_id();
   if (unlikely(sq_num >= priv->channels.num))
       return -ENXIO;

   --------
}


Doesn't this restrict all ports to be bound to the same CPUs via
smp_affinity? In my setup, one port has CPUs 1-63 and the other has
65-127 for a 63 queue configuration. XDP_REDIRECT will fail because
65-127 will exceed the channel count. Is that by design, or am I
missing something? We can find a similar implementation for the i40e
driver as well.

Regards,
Sanjay

Re: XDP-REDIRECT cpu affinity

[Zvi Effron]

On Thu, May 4, 2023 at 11:28 AM Sanjay Rao <srao@fastly.com> wrote:
>
> I am looking at the implementation of ndo_xdp_xmit() in
> mlx5/core/en/xdp.c (most other drivers also have similar
> implementations). This function is called for XDP-REDIRECT.
>
> int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame
> **frames, u32 flags)
> {
> ------
> ------
>
> sq_num = smp_processor_id();
> if (unlikely(sq_num >= priv->channels.num))
> return -ENXIO;
>
> --------
> }
>
>
> Doesn't this restrict all ports to be bound to the same CPUs via
> smp_affinity? In my setup, one port has CPUs 1-63 and the other has
> 65-127 for a 63 queue configuration. XDP_REDIRECT will fail because
> 65-127 will exceed the channel count. Is that by design, or am I
> missing something? We can find a similar implementation for the i40e
> driver as well.

We had previously brought this up for the i40e driver[0]. My take on the
resulting conversation is that this behavior being a bug, but there is no clear
idea of how to solve it without introducing locks, which would be an
unacceptable performance hit.

>
> Regards,
> Sanjay

--Zvi

[0]: https://lore.kernel.org/xdp-newbies/CAC1LvL1NHj6n+RNYRmja2YDhkcCwREuhjaBz_k255rU1jdO8Sw@mail.gmail.com/

Re: XDP redirect - ping fail

[team lnx]

Yes, IFACE1 has a route(static) to IFACE3 and ping succeeds.
Ping fails to IFACE4 from IFACE1, Do I need NAT and does XDP have support ?


On Thu, May 4, 2023 at 11:05 AM Shaw, Jeffrey B
<jeffrey.b.shaw@intel.com> wrote:
>
> On 5/4/2023 10:38 AM, team lnx wrote:
> > Hello,
> >
> > I see a ping not working in below topology with xdp redirect
> >
> >
> > IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
> >
> > IFACE2 and IFACE3 are on same host machine
> >
> > IFACE1 and IFACE2 are  on static ip
> > IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
> >
> > steps
> > 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> > 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> > 3. Run xdp redirect on IFACE2 and IFACE3
> > 4. start ping from IFACE1 to IFACE4
>
> Does IFACE1 have a route to 192.168.2.x and is the next hop resolved?
> You can use `tcpdump` on IFACE1 and IFACE4 to see what packets are sent
> and received.

Re: XDP redirect - ping fail

[Marco]

Hi Teaminx

Have you already tried to send an icmp packet with zero payload to see
with bpf_printk what happens before redirection?

Il giorno ven 12 mag 2023 alle ore 21:17 team lnx
<teamlnxi8@gmail.com> ha scritto:
>
> Yes, IFACE1 has a route(static) to IFACE3 and ping succeeds.
> Ping fails to IFACE4 from IFACE1, Do I need NAT and does XDP have support ?
>
>
> On Thu, May 4, 2023 at 11:05 AM Shaw, Jeffrey B
> <jeffrey.b.shaw@intel.com> wrote:
> >
> > On 5/4/2023 10:38 AM, team lnx wrote:
> > > Hello,
> > >
> > > I see a ping not working in below topology with xdp redirect
> > >
> > >
> > > IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
> > >
> > > IFACE2 and IFACE3 are on same host machine
> > >
> > > IFACE1 and IFACE2 are  on static ip
> > > IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
> > >
> > > steps
> > > 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> > > 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> > > 3. Run xdp redirect on IFACE2 and IFACE3
> > > 4. start ping from IFACE1 to IFACE4
> >
> > Does IFACE1 have a route to 192.168.2.x and is the next hop resolved?
> > You can use `tcpdump` on IFACE1 and IFACE4 to see what packets are sent
> > and received.



-- 
E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!

Re: XDP redirect - ping fail

[team lnx]

Hello Marco,

I am working on that but I could see a couple of discussions where it
has been mentioned we may have to
enhance xdp to have support NAT.

As an experiment in the same topology, adding masquerade NAT rules (ip
translation) without XDP doesn't have a ping issue.
Does this mean we need to have something like xdp-iptables to add NAT rules ?


On Mon, May 15, 2023 at 4:04 PM team lnx <teamlnxi8@gmail.com> wrote:
>
> Hello Marco,
>
> I am working on that but I could see a couple of discussions where it has been mentioned we may have to
> enhance xdp to have support NAT.
>
> As an experiment in the same topology, adding masquerade NAT rules (ip translation) without XDP doesn't have a ping issue.
> Does this mean we need to have something like xdp-iptables to add NAT rules ?
>
>
>
> On Sat, May 13, 2023 at 12:41 AM Marco <ctxspi@gmail.com> wrote:
>>
>> Hi Teaminx
>>
>> Have you already tried to send an icmp packet with zero payload to see
>> with bpf_printk what happens before redirection?
>>
>> Il giorno ven 12 mag 2023 alle ore 21:17 team lnx
>> <teamlnxi8@gmail.com> ha scritto:
>> >
>> > Yes, IFACE1 has a route(static) to IFACE3 and ping succeeds.
>> > Ping fails to IFACE4 from IFACE1, Do I need NAT and does XDP have support ?
>> >
>> >
>> > On Thu, May 4, 2023 at 11:05 AM Shaw, Jeffrey B
>> > <jeffrey.b.shaw@intel.com> wrote:
>> > >
>> > > On 5/4/2023 10:38 AM, team lnx wrote:
>> > > > Hello,
>> > > >
>> > > > I see a ping not working in below topology with xdp redirect
>> > > >
>> > > >
>> > > > IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
>> > > >
>> > > > IFACE2 and IFACE3 are on same host machine
>> > > >
>> > > > IFACE1 and IFACE2 are  on static ip
>> > > > IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
>> > > >
>> > > > steps
>> > > > 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
>> > > > 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
>> > > > 3. Run xdp redirect on IFACE2 and IFACE3
>> > > > 4. start ping from IFACE1 to IFACE4
>> > >
>> > > Does IFACE1 have a route to 192.168.2.x and is the next hop resolved?
>> > > You can use `tcpdump` on IFACE1 and IFACE4 to see what packets are sent
>> > > and received.
>>
>>
>>
>> --
>> E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!

Re: XDP redirect - ping fail

[Marco]

I have not much experience with dhcp but ensure first that dhcp
packets are not blocked from the xdp program.
So as already mentioned, try pinging with zero payloads and check if
they get redirected.

Il giorno gio 4 mag 2023 alle ore 19:41 team lnx <teamlnxi8@gmail.com>
ha scritto:
>
> Hello,
>
> I see a ping not working in below topology with xdp redirect
>
>
> IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
>
> IFACE2 and IFACE3 are on same host machine
>
> IFACE1 and IFACE2 are  on static ip
> IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
>
> steps
> 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> 3. Run xdp redirect on IFACE2 and IFACE3
> 4. start ping from IFACE1 to IFACE4
>
> Are some of the assumptions not right here leading to ping failure ?
>
> Regards,
> teamlnx



-- 
E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!

Re: XDP redirect - ping fail

[Marco]

Sorry, I answered the dhcp part earlier.
Regarding the Nat, I can tell you that it also works with your XDP
program if properly configured.

Il giorno mar 16 mag 2023 alle ore 10:36 Marco <ctxspi@gmail.com> ha scritto:
>
> I have not much experience with dhcp but ensure first that dhcp
> packets are not blocked from the xdp program.
> So as already mentioned, try pinging with zero payloads and check if
> they get redirected.
>
> Il giorno gio 4 mag 2023 alle ore 19:41 team lnx <teamlnxi8@gmail.com>
> ha scritto:
> >
> > Hello,
> >
> > I see a ping not working in below topology with xdp redirect
> >
> >
> > IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
> >
> > IFACE2 and IFACE3 are on same host machine
> >
> > IFACE1 and IFACE2 are  on static ip
> > IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
> >
> > steps
> > 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> > 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> > 3. Run xdp redirect on IFACE2 and IFACE3
> > 4. start ping from IFACE1 to IFACE4
> >
> > Are some of the assumptions not right here leading to ping failure ?
> >
> > Regards,
> > teamlnx
>
>
>
> --
> E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!



-- 
E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!

Re: XDP redirect - ping fail

[team lnx]

Thank you, I am using "xdp_fwd"  and I believe this alone wouldn't be
sufficient Is that correct ?


On Tue, May 16, 2023 at 12:43 AM Marco <ctxspi@gmail.com> wrote:
>
> Sorry, I answered the dhcp part earlier.
> Regarding the Nat, I can tell you that it also works with your XDP
> program if properly configured.
>
> Il giorno mar 16 mag 2023 alle ore 10:36 Marco <ctxspi@gmail.com> ha scritto:
> >
> > I have not much experience with dhcp but ensure first that dhcp
> > packets are not blocked from the xdp program.
> > So as already mentioned, try pinging with zero payloads and check if
> > they get redirected.
> >
> > Il giorno gio 4 mag 2023 alle ore 19:41 team lnx <teamlnxi8@gmail.com>
> > ha scritto:
> > >
> > > Hello,
> > >
> > > I see a ping not working in below topology with xdp redirect
> > >
> > >
> > > IFACE1 <--------->   IFACE2      IFACE3    <------------------> IFACE4
> > >
> > > IFACE2 and IFACE3 are on same host machine
> > >
> > > IFACE1 and IFACE2 are  on static ip
> > > IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server
> > >
> > > steps
> > > 1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
> > > 2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
> > > 3. Run xdp redirect on IFACE2 and IFACE3
> > > 4. start ping from IFACE1 to IFACE4
> > >
> > > Are some of the assumptions not right here leading to ping failure ?
> > >
> > > Regards,
> > > teamlnx
> >
> >
> >
> > --
> > E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!
>
>
>
> --
> E' meglio coltivare GNU/Linux... tanto Windows si pianta da solo!!

XDP Redirect - Ping fail

[team lnx]

Hello,

I see a ping not working in below topology with xdp redirect

                               --------------------------------------
IFACE1 <--------->  |  IFACE2                  IFACE3 |
<------------------> IFACE4
                              ----------------------------------------

IFACE2 and IFACE3 are on same host machine

IFACE1 and IFACE2 are  on static ip
IFACE3 and IFACE4 are on dynamic ip, IFACE4 runs dhcp server

steps
1. assign static ip between IFACE1 and IFACE2 (ex: 172.168.2.x)
2. dynamic ip between IFACE3 and IFACE4 (ex: 192.168.2.x)
3. Run xdp redirect on IFACE2 and IFACE3
4. start ping from IFACE1 to IFACE4

Are some of the assumptions not right here leading to ping failure ?

Regards,
teamlnx