[PATCH net] virtio-net: enable big mode correctly

[PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
large max_mtu. In this case, using small packet mode is not correct
since it may breaks the networking when MTU is grater than
ETH_DATA_LEN.

To have a quick fix, simply enable the big packet mode when
VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.

Reported-by: Eli Cohen <elic@nvidia.com>
Cc: Eli Cohen <elic@nvidia.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/virtio_net.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 7c43bfc1ce44..83ae3ef5eb11 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
 		dev->mtu = mtu;
 		dev->max_mtu = mtu;
 
-		/* TODO: size buffers correctly in this case. */
-		if (dev->mtu > ETH_DATA_LEN)
-			vi->big_packets = true;
 	}
 
+	/* TODO: size buffers correctly in this case. */
+	if (dev->max_mtu > ETH_DATA_LEN)
+		vi->big_packets = true;
+
 	if (vi->any_header_sg)
 		dev->needed_headroom = vi->hdr_len;
 
-- 
2.25.1

Re: [PATCH net] virtio-net: enable big mode correctly

[Michael S. Tsirkin]

On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> large max_mtu. In this case, using small packet mode is not correct
> since it may breaks the networking when MTU is grater than
> ETH_DATA_LEN.
> 
> To have a quick fix, simply enable the big packet mode when
> VIRTIO_NET_F_MTU is not negotiated.

This will slow down dpdk hosts which disable mergeable buffers
and send standard MTU sized packets.

> We can do optimization on top.

I don't think it works like this, increasing mtu
from guest >4k never worked, we can't regress everyone's
performance with a promise to maybe sometime bring it back.

> Reported-by: Eli Cohen <elic@nvidia.com>
> Cc: Eli Cohen <elic@nvidia.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
>
> ---
>  drivers/net/virtio_net.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 7c43bfc1ce44..83ae3ef5eb11 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
>  		dev->mtu = mtu;
>  		dev->max_mtu = mtu;
>  
> -		/* TODO: size buffers correctly in this case. */
> -		if (dev->mtu > ETH_DATA_LEN)
> -			vi->big_packets = true;
>  	}
>  
> +	/* TODO: size buffers correctly in this case. */
> +	if (dev->max_mtu > ETH_DATA_LEN)
> +		vi->big_packets = true;
> +
>  	if (vi->any_header_sg)
>  		dev->needed_headroom = vi->hdr_len;
>  
> -- 
> 2.25.1

Re: [PATCH net] virtio-net: enable big mode correctly

[Eli Cohen]

On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> large max_mtu. In this case, using small packet mode is not correct
> since it may breaks the networking when MTU is grater than
> ETH_DATA_LEN.
> 
> To have a quick fix, simply enable the big packet mode when
> VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> 
> Reported-by: Eli Cohen <elic@nvidia.com>
> Cc: Eli Cohen <elic@nvidia.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  drivers/net/virtio_net.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 7c43bfc1ce44..83ae3ef5eb11 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
>  		dev->mtu = mtu;
>  		dev->max_mtu = mtu;
>  
> -		/* TODO: size buffers correctly in this case. */
> -		if (dev->mtu > ETH_DATA_LEN)
> -			vi->big_packets = true;
>  	}
>  
> +	/* TODO: size buffers correctly in this case. */
> +	if (dev->max_mtu > ETH_DATA_LEN)
> +		vi->big_packets = true;
> +

If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
big_packets to true.


>  	if (vi->any_header_sg)
>  		dev->needed_headroom = vi->hdr_len;
>  
> -- 
> 2.25.1
> 

Re: [PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > large max_mtu. In this case, using small packet mode is not correct
> > since it may breaks the networking when MTU is grater than
> > ETH_DATA_LEN.
> >
> > To have a quick fix, simply enable the big packet mode when
> > VIRTIO_NET_F_MTU is not negotiated.
>
> This will slow down dpdk hosts which disable mergeable buffers
> and send standard MTU sized packets.
>
> > We can do optimization on top.
>
> I don't think it works like this, increasing mtu
> from guest >4k never worked,

Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.

> we can't regress everyone's
> performance with a promise to maybe sometime bring it back.

So consider it never work before I wonder if we can assume a 1500 as
max_mtu value instead of simply using MAX_MTU?

Thanks

>
> > Reported-by: Eli Cohen <elic@nvidia.com>
> > Cc: Eli Cohen <elic@nvidia.com>
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> >
> > ---
> >  drivers/net/virtio_net.c | 7 ++++---
> >  1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> >               dev->mtu = mtu;
> >               dev->max_mtu = mtu;
> >
> > -             /* TODO: size buffers correctly in this case. */
> > -             if (dev->mtu > ETH_DATA_LEN)
> > -                     vi->big_packets = true;
> >       }
> >
> > +     /* TODO: size buffers correctly in this case. */
> > +     if (dev->max_mtu > ETH_DATA_LEN)
> > +             vi->big_packets = true;
> > +
> >       if (vi->any_header_sg)
> >               dev->needed_headroom = vi->hdr_len;
> >
> > --
> > 2.25.1
>

Re: [PATCH net] virtio-net: enable big mode correctly

[Michael S. Tsirkin]

On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > large max_mtu. In this case, using small packet mode is not correct
> > > since it may breaks the networking when MTU is grater than
> > > ETH_DATA_LEN.
> > >
> > > To have a quick fix, simply enable the big packet mode when
> > > VIRTIO_NET_F_MTU is not negotiated.
> >
> > This will slow down dpdk hosts which disable mergeable buffers
> > and send standard MTU sized packets.
> >
> > > We can do optimization on top.
> >
> > I don't think it works like this, increasing mtu
> > from guest >4k never worked,
> 
> Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.

OK, even more so then.

> > we can't regress everyone's
> > performance with a promise to maybe sometime bring it back.
> 
> So consider it never work before I wonder if we can assume a 1500 as
> max_mtu value instead of simply using MAX_MTU?
> 
> Thanks

You want to block guests from setting MTU to a value >GOOD_PACKET_LEN?
Maybe ... it will prevent sending large packets which did work ...
I'd tread carefully here, and I don't think this kind of thing is net
material.

> >
> > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > Cc: Eli Cohen <elic@nvidia.com>
> > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > >
> > > ---
> > >  drivers/net/virtio_net.c | 7 ++++---
> > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > >               dev->mtu = mtu;
> > >               dev->max_mtu = mtu;
> > >
> > > -             /* TODO: size buffers correctly in this case. */
> > > -             if (dev->mtu > ETH_DATA_LEN)
> > > -                     vi->big_packets = true;
> > >       }
> > >
> > > +     /* TODO: size buffers correctly in this case. */
> > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > +             vi->big_packets = true;
> > > +
> > >       if (vi->any_header_sg)
> > >               dev->needed_headroom = vi->hdr_len;
> > >
> > > --
> > > 2.25.1
> >

Re: [PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote:
>
> On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > large max_mtu. In this case, using small packet mode is not correct
> > since it may breaks the networking when MTU is grater than
> > ETH_DATA_LEN.
> >
> > To have a quick fix, simply enable the big packet mode when
> > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> >
> > Reported-by: Eli Cohen <elic@nvidia.com>
> > Cc: Eli Cohen <elic@nvidia.com>
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > ---
> >  drivers/net/virtio_net.c | 7 ++++---
> >  1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> >               dev->mtu = mtu;
> >               dev->max_mtu = mtu;
> >
> > -             /* TODO: size buffers correctly in this case. */
> > -             if (dev->mtu > ETH_DATA_LEN)
> > -                     vi->big_packets = true;
> >       }
> >
> > +     /* TODO: size buffers correctly in this case. */
> > +     if (dev->max_mtu > ETH_DATA_LEN)
> > +             vi->big_packets = true;
> > +
>
> If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
> ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
> big_packets to true.

I may miss something, the dev->max_mtu is just assigned to the mtu
value read from the config space in the code block above  (inside the
feature check of VIRTIO_NET_F_MTU).

Thanks

>
>
> >       if (vi->any_header_sg)
> >               dev->needed_headroom = vi->hdr_len;
> >
> > --
> > 2.25.1
> >
>

Re: [PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote:
> > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > large max_mtu. In this case, using small packet mode is not correct
> > > > since it may breaks the networking when MTU is grater than
> > > > ETH_DATA_LEN.
> > > >
> > > > To have a quick fix, simply enable the big packet mode when
> > > > VIRTIO_NET_F_MTU is not negotiated.
> > >
> > > This will slow down dpdk hosts which disable mergeable buffers
> > > and send standard MTU sized packets.
> > >
> > > > We can do optimization on top.
> > >
> > > I don't think it works like this, increasing mtu
> > > from guest >4k never worked,
> >
> > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.
>
> OK, even more so then.
>
> > > we can't regress everyone's
> > > performance with a promise to maybe sometime bring it back.
> >
> > So consider it never work before I wonder if we can assume a 1500 as
> > max_mtu value instead of simply using MAX_MTU?
> >
> > Thanks
>
> You want to block guests from setting MTU to a value >GOOD_PACKET_LEN?

Yes, or fix the issue to let large packets on RX work (e.g as the TODO
said, size the buffer: for <=4K mtu continue to work as
add_recvbuf_small(), for >= 4K switch to use big).

> Maybe ... it will prevent sending large packets which did work ...

Yes, but it's strange to allow TX but not RX

> I'd tread carefully here, and I don't think this kind of thing is net
> material.

I agree consider it can't be fixed easily.

Thanks

>
> > >
> > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > >
> > > > ---
> > > >  drivers/net/virtio_net.c | 7 ++++---
> > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > >               dev->mtu = mtu;
> > > >               dev->max_mtu = mtu;
> > > >
> > > > -             /* TODO: size buffers correctly in this case. */
> > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > -                     vi->big_packets = true;
> > > >       }
> > > >
> > > > +     /* TODO: size buffers correctly in this case. */
> > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > +             vi->big_packets = true;
> > > > +
> > > >       if (vi->any_header_sg)
> > > >               dev->needed_headroom = vi->hdr_len;
> > > >
> > > > --
> > > > 2.25.1
> > >
>

Re: [PATCH net] virtio-net: enable big mode correctly

[Eli Cohen]

On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > large max_mtu. In this case, using small packet mode is not correct
> > > since it may breaks the networking when MTU is grater than
> > > ETH_DATA_LEN.
> > >
> > > To have a quick fix, simply enable the big packet mode when
> > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> > >
> > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > Cc: Eli Cohen <elic@nvidia.com>
> > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > ---
> > >  drivers/net/virtio_net.c | 7 ++++---
> > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > >               dev->mtu = mtu;
> > >               dev->max_mtu = mtu;
> > >
> > > -             /* TODO: size buffers correctly in this case. */
> > > -             if (dev->mtu > ETH_DATA_LEN)
> > > -                     vi->big_packets = true;
> > >       }
> > >
> > > +     /* TODO: size buffers correctly in this case. */
> > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > +             vi->big_packets = true;
> > > +
> >
> > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
> > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
> > big_packets to true.
> 
> I may miss something, the dev->max_mtu is just assigned to the mtu
> value read from the config space in the code block above  (inside the
> feature check of VIRTIO_NET_F_MTU).

Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case
dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to
true.

> 
> Thanks
> 
> >
> >
> > >       if (vi->any_header_sg)
> > >               dev->needed_headroom = vi->hdr_len;
> > >
> > > --
> > > 2.25.1
> > >
> >
> 

Re: [PATCH net] virtio-net: enable big mode correctly

[Michael S. Tsirkin]

On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote:
> > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > > large max_mtu. In this case, using small packet mode is not correct
> > > > > since it may breaks the networking when MTU is grater than
> > > > > ETH_DATA_LEN.
> > > > >
> > > > > To have a quick fix, simply enable the big packet mode when
> > > > > VIRTIO_NET_F_MTU is not negotiated.
> > > >
> > > > This will slow down dpdk hosts which disable mergeable buffers
> > > > and send standard MTU sized packets.
> > > >
> > > > > We can do optimization on top.
> > > >
> > > > I don't think it works like this, increasing mtu
> > > > from guest >4k never worked,
> > >
> > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.
> >
> > OK, even more so then.
> >
> > > > we can't regress everyone's
> > > > performance with a promise to maybe sometime bring it back.
> > >
> > > So consider it never work before I wonder if we can assume a 1500 as
> > > max_mtu value instead of simply using MAX_MTU?
> > >
> > > Thanks
> >
> > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN?
> 
> Yes, or fix the issue to let large packets on RX work (e.g as the TODO
> said, size the buffer: for <=4K mtu continue to work as
> add_recvbuf_small(), for >= 4K switch to use big).

Right. The difficulty is with changing modes, current code isn't
designed for it.

> > Maybe ... it will prevent sending large packets which did work ...
> 
> Yes, but it's strange to allow TX but not RX
> 
> > I'd tread carefully here, and I don't think this kind of thing is net
> > material.
> 
> I agree consider it can't be fixed easily.
> 
> Thanks
> 
> >
> > > >
> > > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > >
> > > > > ---
> > > > >  drivers/net/virtio_net.c | 7 ++++---
> > > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > > >               dev->mtu = mtu;
> > > > >               dev->max_mtu = mtu;
> > > > >
> > > > > -             /* TODO: size buffers correctly in this case. */
> > > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > > -                     vi->big_packets = true;
> > > > >       }
> > > > >
> > > > > +     /* TODO: size buffers correctly in this case. */
> > > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > > +             vi->big_packets = true;
> > > > > +
> > > > >       if (vi->any_header_sg)
> > > > >               dev->needed_headroom = vi->hdr_len;
> > > > >
> > > > > --
> > > > > 2.25.1
> > > >
> >

Re: [PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote:
>
> On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote:
> > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote:
> > >
> > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > large max_mtu. In this case, using small packet mode is not correct
> > > > since it may breaks the networking when MTU is grater than
> > > > ETH_DATA_LEN.
> > > >
> > > > To have a quick fix, simply enable the big packet mode when
> > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> > > >
> > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > ---
> > > >  drivers/net/virtio_net.c | 7 ++++---
> > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > >               dev->mtu = mtu;
> > > >               dev->max_mtu = mtu;
> > > >
> > > > -             /* TODO: size buffers correctly in this case. */
> > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > -                     vi->big_packets = true;
> > > >       }
> > > >
> > > > +     /* TODO: size buffers correctly in this case. */
> > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > +             vi->big_packets = true;
> > > > +
> > >
> > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
> > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
> > > big_packets to true.
> >
> > I may miss something, the dev->max_mtu is just assigned to the mtu
> > value read from the config space in the code block above  (inside the
> > feature check of VIRTIO_NET_F_MTU).
>
> Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case
> dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to
> true.

I see but in this case, the above assignment:

        /* MTU range: 68 - 65535 */
        dev->min_mtu = MIN_MTU;
        dev->max_mtu = MAX_MTU;

happens after alloc_etherdev_mq() which calls ether_setup(), so we are
probably fine here.

Thanks

>
> >
> > Thanks
> >
> > >
> > >
> > > >       if (vi->any_header_sg)
> > > >               dev->needed_headroom = vi->hdr_len;
> > > >
> > > > --
> > > > 2.25.1
> > > >
> > >
> >
>

Re: [PATCH net] virtio-net: enable big mode correctly

[Jason Wang]

On Thu, Nov 25, 2021 at 3:26 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote:
> > On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote:
> > > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > >
> > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > > > large max_mtu. In this case, using small packet mode is not correct
> > > > > > since it may breaks the networking when MTU is grater than
> > > > > > ETH_DATA_LEN.
> > > > > >
> > > > > > To have a quick fix, simply enable the big packet mode when
> > > > > > VIRTIO_NET_F_MTU is not negotiated.
> > > > >
> > > > > This will slow down dpdk hosts which disable mergeable buffers
> > > > > and send standard MTU sized packets.
> > > > >
> > > > > > We can do optimization on top.
> > > > >
> > > > > I don't think it works like this, increasing mtu
> > > > > from guest >4k never worked,
> > > >
> > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.
> > >
> > > OK, even more so then.
> > >
> > > > > we can't regress everyone's
> > > > > performance with a promise to maybe sometime bring it back.
> > > >
> > > > So consider it never work before I wonder if we can assume a 1500 as
> > > > max_mtu value instead of simply using MAX_MTU?
> > > >
> > > > Thanks
> > >
> > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN?
> >
> > Yes, or fix the issue to let large packets on RX work (e.g as the TODO
> > said, size the buffer: for <=4K mtu continue to work as
> > add_recvbuf_small(), for >= 4K switch to use big).
>
> Right. The difficulty is with changing modes, current code isn't
> designed for it.

I think it might work if we reset the device during the mode change.

Thanks

>
> > > Maybe ... it will prevent sending large packets which did work ...
> >
> > Yes, but it's strange to allow TX but not RX
> >
> > > I'd tread carefully here, and I don't think this kind of thing is net
> > > material.
> >
> > I agree consider it can't be fixed easily.
> >
> > Thanks
> >
> > >
> > > > >
> > > > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > > >
> > > > > > ---
> > > > > >  drivers/net/virtio_net.c | 7 ++++---
> > > > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > > > --- a/drivers/net/virtio_net.c
> > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > > > >               dev->mtu = mtu;
> > > > > >               dev->max_mtu = mtu;
> > > > > >
> > > > > > -             /* TODO: size buffers correctly in this case. */
> > > > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > > > -                     vi->big_packets = true;
> > > > > >       }
> > > > > >
> > > > > > +     /* TODO: size buffers correctly in this case. */
> > > > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > > > +             vi->big_packets = true;
> > > > > > +
> > > > > >       if (vi->any_header_sg)
> > > > > >               dev->needed_headroom = vi->hdr_len;
> > > > > >
> > > > > > --
> > > > > > 2.25.1
> > > > >
> > >
>

Re: [PATCH net] virtio-net: enable big mode correctly

[Michael S. Tsirkin]

On Thu, Nov 25, 2021 at 03:26:22PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote:
> > > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote:
> > > >
> > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > > large max_mtu. In this case, using small packet mode is not correct
> > > > > since it may breaks the networking when MTU is grater than
> > > > > ETH_DATA_LEN.
> > > > >
> > > > > To have a quick fix, simply enable the big packet mode when
> > > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> > > > >
> > > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > > ---
> > > > >  drivers/net/virtio_net.c | 7 ++++---
> > > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > > >               dev->mtu = mtu;
> > > > >               dev->max_mtu = mtu;
> > > > >
> > > > > -             /* TODO: size buffers correctly in this case. */
> > > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > > -                     vi->big_packets = true;
> > > > >       }
> > > > >
> > > > > +     /* TODO: size buffers correctly in this case. */
> > > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > > +             vi->big_packets = true;
> > > > > +
> > > >
> > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
> > > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
> > > > big_packets to true.
> > >
> > > I may miss something, the dev->max_mtu is just assigned to the mtu
> > > value read from the config space in the code block above  (inside the
> > > feature check of VIRTIO_NET_F_MTU).
> >
> > Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case
> > dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to
> > true.
> 
> I see but in this case, the above assignment:
> 
>         /* MTU range: 68 - 65535 */
>         dev->min_mtu = MIN_MTU;
>         dev->max_mtu = MAX_MTU;
> 
> happens after alloc_etherdev_mq() which calls ether_setup(), so we are
> probably fine here.
> 
> Thanks

Actually the issue with VIRTIO_NET_F_MTU is that devices might be
tempted to expose 9k here simply because they support jumbo frames,
if they also don't support mergeable buffers this will force big
packet mode.


> >
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > >       if (vi->any_header_sg)
> > > > >               dev->needed_headroom = vi->hdr_len;
> > > > >
> > > > > --
> > > > > 2.25.1
> > > > >
> > > >
> > >
> >

Re: [PATCH net] virtio-net: enable big mode correctly

[Eli Cohen]

On Thu, Nov 25, 2021 at 03:26:22PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:20 PM Eli Cohen <elic@nvidia.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 03:15:33PM +0800, Jason Wang wrote:
> > > On Thu, Nov 25, 2021 at 3:09 PM Eli Cohen <elic@nvidia.com> wrote:
> > > >
> > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > > large max_mtu. In this case, using small packet mode is not correct
> > > > > since it may breaks the networking when MTU is grater than
> > > > > ETH_DATA_LEN.
> > > > >
> > > > > To have a quick fix, simply enable the big packet mode when
> > > > > VIRTIO_NET_F_MTU is not negotiated. We can do optimization on top.
> > > > >
> > > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > > ---
> > > > >  drivers/net/virtio_net.c | 7 ++++---
> > > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > > >               dev->mtu = mtu;
> > > > >               dev->max_mtu = mtu;
> > > > >
> > > > > -             /* TODO: size buffers correctly in this case. */
> > > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > > -                     vi->big_packets = true;
> > > > >       }
> > > > >
> > > > > +     /* TODO: size buffers correctly in this case. */
> > > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > > +             vi->big_packets = true;
> > > > > +
> > > >
> > > > If VIRTIO_NET_F_MTU is provided, then dev->max_mtu is going to equal
> > > > ETH_DATA_LEN (will be set in ether_setup()) so I don't think it will set
> > > > big_packets to true.
> > >
> > > I may miss something, the dev->max_mtu is just assigned to the mtu
> > > value read from the config space in the code block above  (inside the
> > > feature check of VIRTIO_NET_F_MTU).
> >
> > Sorry, I meant "If VIRTIO_NET_F_MTU is ***NOT*** provided". In that case
> > dev->max_mtu eauals ETH_DATA_LEN so you won't set vi->big_packets to
> > true.
> 
> I see but in this case, the above assignment:
> 
>         /* MTU range: 68 - 65535 */
>         dev->min_mtu = MIN_MTU;
>         dev->max_mtu = MAX_MTU;
> 
> happens after alloc_etherdev_mq() which calls ether_setup(), so we are
> probably fine here.
> 

I see, thanks.

> Thanks
> 
> >
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > >       if (vi->any_header_sg)
> > > > >               dev->needed_headroom = vi->hdr_len;
> > > > >
> > > > > --
> > > > > 2.25.1
> > > > >
> > > >
> > >
> >
> 

Re: [PATCH net] virtio-net: enable big mode correctly

[Michael S. Tsirkin]

On Thu, Nov 25, 2021 at 03:28:31PM +0800, Jason Wang wrote:
> On Thu, Nov 25, 2021 at 3:26 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 03:20:07PM +0800, Jason Wang wrote:
> > > On Thu, Nov 25, 2021 at 3:15 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > On Thu, Nov 25, 2021 at 03:11:58PM +0800, Jason Wang wrote:
> > > > > On Thu, Nov 25, 2021 at 3:00 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Nov 25, 2021 at 02:05:47PM +0800, Jason Wang wrote:
> > > > > > > When VIRTIO_NET_F_MTU feature is not negotiated, we assume a very
> > > > > > > large max_mtu. In this case, using small packet mode is not correct
> > > > > > > since it may breaks the networking when MTU is grater than
> > > > > > > ETH_DATA_LEN.
> > > > > > >
> > > > > > > To have a quick fix, simply enable the big packet mode when
> > > > > > > VIRTIO_NET_F_MTU is not negotiated.
> > > > > >
> > > > > > This will slow down dpdk hosts which disable mergeable buffers
> > > > > > and send standard MTU sized packets.
> > > > > >
> > > > > > > We can do optimization on top.
> > > > > >
> > > > > > I don't think it works like this, increasing mtu
> > > > > > from guest >4k never worked,
> > > > >
> > > > > Looking at add_recvbuf_small() it's actually GOOD_PACKET_LEN if I was not wrong.
> > > >
> > > > OK, even more so then.
> > > >
> > > > > > we can't regress everyone's
> > > > > > performance with a promise to maybe sometime bring it back.
> > > > >
> > > > > So consider it never work before I wonder if we can assume a 1500 as
> > > > > max_mtu value instead of simply using MAX_MTU?
> > > > >
> > > > > Thanks
> > > >
> > > > You want to block guests from setting MTU to a value >GOOD_PACKET_LEN?
> > >
> > > Yes, or fix the issue to let large packets on RX work (e.g as the TODO
> > > said, size the buffer: for <=4K mtu continue to work as
> > > add_recvbuf_small(), for >= 4K switch to use big).
> >
> > Right. The difficulty is with changing modes, current code isn't
> > designed for it.
> 
> I think it might work if we reset the device during the mode change.
> 
> Thanks

For sure. It's hard to do without races though, and we need to
carefully restore all the programming done so far.
Maybe it will be easier if we do something like disable_irq
to reliably suppress interrupts from hardware.

> >
> > > > Maybe ... it will prevent sending large packets which did work ...
> > >
> > > Yes, but it's strange to allow TX but not RX
> > >
> > > > I'd tread carefully here, and I don't think this kind of thing is net
> > > > material.
> > >
> > > I agree consider it can't be fixed easily.
> > >
> > > Thanks
> > >
> > > >
> > > > > >
> > > > > > > Reported-by: Eli Cohen <elic@nvidia.com>
> > > > > > > Cc: Eli Cohen <elic@nvidia.com>
> > > > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > > > >
> > > > > > > ---
> > > > > > >  drivers/net/virtio_net.c | 7 ++++---
> > > > > > >  1 file changed, 4 insertions(+), 3 deletions(-)
> > > > > > >
> > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > > index 7c43bfc1ce44..83ae3ef5eb11 100644
> > > > > > > --- a/drivers/net/virtio_net.c
> > > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > > @@ -3200,11 +3200,12 @@ static int virtnet_probe(struct virtio_device *vdev)
> > > > > > >               dev->mtu = mtu;
> > > > > > >               dev->max_mtu = mtu;
> > > > > > >
> > > > > > > -             /* TODO: size buffers correctly in this case. */
> > > > > > > -             if (dev->mtu > ETH_DATA_LEN)
> > > > > > > -                     vi->big_packets = true;
> > > > > > >       }
> > > > > > >
> > > > > > > +     /* TODO: size buffers correctly in this case. */
> > > > > > > +     if (dev->max_mtu > ETH_DATA_LEN)
> > > > > > > +             vi->big_packets = true;
> > > > > > > +
> > > > > > >       if (vi->any_header_sg)
> > > > > > >               dev->needed_headroom = vi->hdr_len;
> > > > > > >
> > > > > > > --
> > > > > > > 2.25.1
> > > > > >
> > > >
> >