Using XDP for Cloud VMs by David Ahern

Using XDP for Cloud VMs by David Ahern

[Jesper Dangaard Brouer]

Hi XDP-newbies and cloud-providers,

I want to promote the recent excellent work by David Ahern (maintainer of
several kernel networking components). Ahern explains in detail how to use
XDP for lowering CPU usage (and increasing performance) on the host machine
running the Cloud VMs.  Making it more efficient for a cloud provider.

Ahern is explaining and calling for collaboration at many levels. (Note
[xxx] brackets indicate link below). His [Talk#1] at NetDevConf have not
been released on [YouTube] yet, but [Slides] are available. In [Blogpost#2]
you can see the results for XDP vs OVS, and to understand how CPU-overhead
is measured read [Blogpost#1]. He have even released the [Code] on GitHub,
and encourage people to reproduce his results.

I encourage other cloud providers to collaborate with Ahern. XDP is a core
kernel building block, but we need to create FOSS projects that explore
different use-case (e.g xdp-cloud-provider). XDP is still early days, and as
Ahern already discovered, trying out this Cloud-VMs use-case have identified
several gotchas and missing XDP features that we should work on
adding/fixing in upstream kernels.

 --
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer

Links:

[Talk#1] XDP and the cloud: Using XDP on hosts and VMs
 https://netdevconf.info/0x14/session.html?tutorial-XDP-and-the-cloud

[Slides]
 https://github.com/dsahern/bpf-progs/blob/master/docs/netdev-0x14-XDP-and-the-cloud.pdf
 https://github.com/dsahern/bpf-progs/raw/master/docs/netdev-0x14-XDP-and-the-cloud.pdf

[Code] https://github.com/dsahern/bpf-progs

[Blogpost#2]: XDP vs OVS
 https://people.kernel.org/dsahern/xdp-vs-ovs

[Blogpost#1]: The CPU Cost of Networking on a Host
 https://people.kernel.org/dsahern/the-cpu-cost-of-networking-on-a-host

[Talk#2] XDP Workshop
 https://netdevconf.info/0x14/session.html?workshop-XDP

[NetDevConf]: https://netdevconf.info/0x14/
[YouTube]: https://www.youtube.com/channel/UCribHdOMgiD5R3OUDgx2qTg/videos

Re: Using XDP for Cloud VMs by David Ahern

[David Ahern]

On 8/21/20 3:56 AM, Jesper Dangaard Brouer wrote:
> Hi XDP-newbies and cloud-providers,
> 
> I want to promote the recent excellent work by David Ahern (maintainer of
> several kernel networking components). Ahern explains in detail how to use
> XDP for lowering CPU usage (and increasing performance) on the host machine
> running the Cloud VMs.  Making it more efficient for a cloud provider.
> 
> Ahern is explaining and calling for collaboration at many levels. (Note
> [xxx] brackets indicate link below). His [Talk#1] at NetDevConf have not
> been released on [YouTube] yet, but [Slides] are available. In [Blogpost#2]
> you can see the results for XDP vs OVS, and to understand how CPU-overhead
> is measured read [Blogpost#1]. He have even released the [Code] on GitHub,
> and encourage people to reproduce his results.
> 
> I encourage other cloud providers to collaborate with Ahern. XDP is a core
> kernel building block, but we need to create FOSS projects that explore
> different use-case (e.g xdp-cloud-provider). XDP is still early days, and as
> Ahern already discovered, trying out this Cloud-VMs use-case have identified
> several gotchas and missing XDP features that we should work on
> adding/fixing in upstream kernels.
> 

Hi Jesper:

Thanks for bringing this to xdp-newbies. I would definitely welcome
others to get involved. I had high hopes that this use case would fall
out of zabiplane work, but that project died.

Right now the code is integrated into in my repo for my convenience as I
move between hosts and VMs. If / when others get involved, I am open to
moving the code for the cloud host use case to another repo (under
xdp-project/xdp-cloud-provider?) and setting up lists similar to what
you have for xdp-project as a whole. I also have other code in a private
repository that I can pull over as well — e.g., handling decap of vxlan
packets to get the inner packet to do the forwarding lookup. It is best
used with hardware hints which can mark vxlan packets to limit the overhead.

We really need to get the hardware hints code committed it is key to
making this use case a reality.

Re: Using XDP for Cloud VMs by David Ahern

[Stephen Hemminger]

On Fri, 21 Aug 2020 11:56:14 +0200
Jesper Dangaard Brouer <brouer@redhat.com> wrote:

> is measured read [Blogpost#1]. He have even released the [Code] on GitHub,
> and encourage people to reproduce his results.
> 
> I encourage other cloud providers to collaborate with Ahern. XDP is a core
> kernel building block, but we need to create FOSS projects that explore
> different use-case (e.g xdp-cloud-provider). XDP is still early days, and as
> Ahern already discovered, trying out this Cloud-VMs use-case have identified
> several gotchas and missing XDP features that we should work on
> adding/fixing in upstream kernels.
> 
>  --

Was this normal OVS (kernel based) or OVS-DPDK (userspace).
OVS-DPDK is much faster if you can afford to burn CPU.

Also would be interesting to compare XDP vs VPP but this is very much
an apples to orange kind of exercise.

Re: Using XDP for Cloud VMs by David Ahern

[David Ahern]

On 8/24/20 6:14 PM, Stephen Hemminger wrote:
> On Fri, 21 Aug 2020 11:56:14 +0200
> Jesper Dangaard Brouer <brouer@redhat.com> wrote:
> 
>> is measured read [Blogpost#1]. He have even released the [Code] on GitHub,
>> and encourage people to reproduce his results.
>>
>> I encourage other cloud providers to collaborate with Ahern. XDP is a core
>> kernel building block, but we need to create FOSS projects that explore
>> different use-case (e.g xdp-cloud-provider). XDP is still early days, and as
>> Ahern already discovered, trying out this Cloud-VMs use-case have identified
>> several gotchas and missing XDP features that we should work on
>> adding/fixing in upstream kernels.
>>
>>  --
> 
> Was this normal OVS (kernel based) or OVS-DPDK (userspace).
> OVS-DPDK is much faster if you can afford to burn CPU.

It is not always about fastest; resource consumption on a host matters
as well. The investigation was focused on CPU cycles to process a packet
load. With kernel based OVS, you are sacrificing ~1 core (2 hardware
threads) per ~1M pps.

> 
> Also would be interesting to compare XDP vs VPP but this is very much
> an apples to orange kind of exercise.
> 

Endless interesting comparisons. People with the relevant knowledge and
setups will need to step up and do the tests. The source code and
scripts are available so others can do these comparisons.