Re: [dpdk-dev] [PATCH 5/5] vhost: fix offload flags in Rx path

[David Marchand <david.marchand@redhat.com>]

On Tue, Apr 13, 2021 at 5:27 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> On 4/8/21 8:38 PM, Flavio Leitner wrote:
> > On Thu, Apr 01, 2021 at 11:52:43AM +0200, David Marchand wrote:
> >> The vhost library current configures Tx offloading (PKT_TX_*) on any
> >> packet received from a guest virtio device which asks for some offloading.
> >>
> >> This is problematic, as Tx offloading is something that the application
> >> must ask for: the application needs to configure devices
> >> to support every used offloads (ip, tcp checksumming, tso..), and the
> >> various l2/l3/l4 lengths must be set following any processing that
> >> happened in the application itself.
> >>
> >> On the other hand, the received packets are not marked wrt current
> >> packet l3/l4 checksumming info.
> >>
> >> Copy virtio rx processing to fix those offload flags.
> >>
> >> The vhost example needs a reworking as it was built with the assumption
> >> that mbuf TSO configuration is set up by the vhost library.
> >> This is not done in this patch for now so TSO activation is forcibly
> >> refused.
> >>
> >> Fixes: 859b480d5afd ("vhost: add guest offload setting")
> >
> > There is change that before ECN was ignored and now it is invalid.
> > I think that's the right way to go, but not sure if virtio blocks
> > the negotiation of that feature.
>
> No, I just tested and the feature gets negotiated.

I suppose you tested with testpmd, because I can see ECN is disabled
by default with OVS.


>
> Disabling it in Vhost lib should be avoided to avoid breaking
> live-migration.
>
> It might be safer to revert back to older behavior for it, i.e. just
> ignore the bit. I don't think it is ever set, because otherwise we would
> have had lots of reports since the Vhost log would be flooded with:

-  The VIRTIO_NET_HDR_GSO_ECN bit is supposed to be coupled with TSO bits.
Copying a bit more of this code:
   switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) {
...
   default:
>
> VHOST_LOG_DATA(WARNING,
>         "unsupported gso type %u.\n", hdr->gso_type);

The absence of log does not mean the guest is not sending packets with
VIRTIO_NET_HDR_GSO_ECN set.
Otoh, getting this log instead indicates a bug in the virtio driver
(as we discussed offlist).


- It is not clear to me how deployed the ECN feature is.
I think the Linux kernel won't try to start a TCP connection unless
explicitly configuring it on a socket (but I am a bit lost).

By default, VIRTIO_NET_F_HOST_ECN is announced as supported by vhost-user.
So in theory, a guest virtio netdevice with NETIF_F_TSO_ECN can
transmit packet (with SKB_GSO_TCP_ECN translated to
VIRTIO_NET_HDR_GSO_ECN in virtio_net_hdr_from_skb) to a vhost-user
backend.


- Treating ECN with GSO requires special handling:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0da8537037f337103348f239ad901477e907aa8

I can see some change in the i40e kernel driver at least.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=059dab69652da3525d320d77ac5422ec708ced14
The ixgbe kernel driver is not flagged with NETIF_F_TSO_ECN.

We don't have such a distinction in DPDK: neither a per mbuf flag to
mark packets, nor a device offloading flag/capability.
And the rte_gso library probably does not handle correctly CWR.
About the i40e driver, I can't find the same configuration than the
kernel driver.



- Now, about the next step...

The "good" (I suppose you might disagree here) news, is that this
feature is disabled in OVS:
https://github.com/openvswitch/ovs/blob/master/lib/netdev-dpdk.c#L5162

About handling TSO + ECN, this is a generic problem with the DPDK API
and we have been living for a long time.
I understand passing such packets to hw that does not handle this
correctly breaks the ECN feature not work properly.
But "normal" TSO works.

I agree, we can let such packets be received by vhost like it was done
before my patch.

Investigating the other side (GUEST_ECN + the virtio pmd) could be
worth later, as I think GSO+ECN packets are dropped in the current
code.


-- 
David Marchand