All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-23 13:47 ` Corinna Vinschen
  0 siblings, 0 replies; 10+ messages in thread
From: Corinna Vinschen @ 2024-04-23 13:47 UTC (permalink / raw)
  To: netdev, linux-kernel, intel-wired-lan
  Cc: Nikolay Aleksandrov, Jason Xing, Paolo Abeni, Jakub Kicinski,
	Eric Dumazet, David S . Miller, Tony Nguyen, Jesse Brandeburg

From: Paolo Abeni <pabeni@redhat.com>

Sabrina reports that the igb driver does not cope well with large
MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
corruption on TX.

An easy reproducer is to run ssh to connect to the machine.  With
MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.

The root cause of the issue is that the driver does not take into
account properly the (possibly large) shared info size when selecting
the ring layout, and will try to fit two packets inside the same 4K
page even when the 1st fraglist will trump over the 2nd head.

Address the issue forcing the driver to fit a single packet per page,
leaving there enough room to store the (currently) largest possible
skb_shared_info.

Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
Reported-by: Jan Tluka <jtluka@redhat.com>
Reported-by: Jirka Hladky <jhladky@redhat.com>
Reported-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Corinna Vinschen <vinschen@redhat.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2: fix subject, add a simple reproducer

 drivers/net/ethernet/intel/igb/igb_main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index a3f100769e39..22fb2c322bca 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
 
 #if (PAGE_SIZE < 8192)
 	if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
+	    SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
 	    rd32(E1000_RCTL) & E1000_RCTL_SBP)
 		set_ring_uses_large_buffer(rx_ring);
 #endif
-- 
2.44.0


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [Intel-wired-lan] [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-23 13:47 ` Corinna Vinschen
  0 siblings, 0 replies; 10+ messages in thread
From: Corinna Vinschen @ 2024-04-23 13:47 UTC (permalink / raw)
  To: netdev, linux-kernel, intel-wired-lan
  Cc: Jason Xing, Nikolay Aleksandrov, Eric Dumazet, Tony Nguyen,
	Jakub Kicinski, Paolo Abeni, David S . Miller

From: Paolo Abeni <pabeni@redhat.com>

Sabrina reports that the igb driver does not cope well with large
MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
corruption on TX.

An easy reproducer is to run ssh to connect to the machine.  With
MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.

The root cause of the issue is that the driver does not take into
account properly the (possibly large) shared info size when selecting
the ring layout, and will try to fit two packets inside the same 4K
page even when the 1st fraglist will trump over the 2nd head.

Address the issue forcing the driver to fit a single packet per page,
leaving there enough room to store the (currently) largest possible
skb_shared_info.

Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
Reported-by: Jan Tluka <jtluka@redhat.com>
Reported-by: Jirka Hladky <jhladky@redhat.com>
Reported-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Corinna Vinschen <vinschen@redhat.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2: fix subject, add a simple reproducer

 drivers/net/ethernet/intel/igb/igb_main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index a3f100769e39..22fb2c322bca 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
 
 #if (PAGE_SIZE < 8192)
 	if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
+	    SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
 	    rd32(E1000_RCTL) & E1000_RCTL_SBP)
 		set_ring_uses_large_buffer(rx_ring);
 #endif
-- 
2.44.0


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
  2024-04-23 13:47 ` [Intel-wired-lan] " Corinna Vinschen
@ 2024-04-23 14:10   ` Eric Dumazet
  -1 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2024-04-23 14:10 UTC (permalink / raw)
  To: Corinna Vinschen
  Cc: netdev, linux-kernel, intel-wired-lan, Nikolay Aleksandrov,
	Jason Xing, Paolo Abeni, Jakub Kicinski, David S . Miller,
	Tony Nguyen, Jesse Brandeburg

On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
>
> From: Paolo Abeni <pabeni@redhat.com>
>
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
>
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
>
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
>
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
>
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2: fix subject, add a simple reproducer
>
>  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index a3f100769e39..22fb2c322bca 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
>
>  #if (PAGE_SIZE < 8192)
>         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||

I wonder if adding IGB_SKB_PAD would be needed ?

adapter->max_frame_size does not seem to include it.

I would try using all mtus between 1200 and 1280 to make sure this works.

Thanks !

>             rd32(E1000_RCTL) & E1000_RCTL_SBP)
>                 set_ring_uses_large_buffer(rx_ring);
>  #endif
> --
> 2.44.0
>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Intel-wired-lan] [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-23 14:10   ` Eric Dumazet
  0 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2024-04-23 14:10 UTC (permalink / raw)
  To: Corinna Vinschen
  Cc: Jason Xing, netdev, Nikolay Aleksandrov, linux-kernel,
	Tony Nguyen, intel-wired-lan, Jakub Kicinski, Paolo Abeni,
	David S . Miller

On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
>
> From: Paolo Abeni <pabeni@redhat.com>
>
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
>
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
>
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
>
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
>
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2: fix subject, add a simple reproducer
>
>  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index a3f100769e39..22fb2c322bca 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
>
>  #if (PAGE_SIZE < 8192)
>         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||

I wonder if adding IGB_SKB_PAD would be needed ?

adapter->max_frame_size does not seem to include it.

I would try using all mtus between 1200 and 1280 to make sure this works.

Thanks !

>             rd32(E1000_RCTL) & E1000_RCTL_SBP)
>                 set_ring_uses_large_buffer(rx_ring);
>  #endif
> --
> 2.44.0
>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
  2024-04-23 14:10   ` [Intel-wired-lan] " Eric Dumazet
@ 2024-04-26 14:30     ` Corinna Vinschen
  -1 siblings, 0 replies; 10+ messages in thread
From: Corinna Vinschen @ 2024-04-26 14:30 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: netdev, linux-kernel, intel-wired-lan, Nikolay Aleksandrov,
	Jason Xing, Paolo Abeni, Jakub Kicinski, David S . Miller,
	Tony Nguyen, Jesse Brandeburg

Hi Eric,

On Apr 23 16:10, Eric Dumazet wrote:
> On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
> >
> > From: Paolo Abeni <pabeni@redhat.com>
> >
> > Sabrina reports that the igb driver does not cope well with large
> > MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> > corruption on TX.
> >
> > An easy reproducer is to run ssh to connect to the machine.  With
> > MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> >
> > The root cause of the issue is that the driver does not take into
> > account properly the (possibly large) shared info size when selecting
> > the ring layout, and will try to fit two packets inside the same 4K
> > page even when the 1st fraglist will trump over the 2nd head.
> >
> > Address the issue forcing the driver to fit a single packet per page,
> > leaving there enough room to store the (currently) largest possible
> > skb_shared_info.
> >
> > Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> > Reported-by: Jan Tluka <jtluka@redhat.com>
> > Reported-by: Jirka Hladky <jhladky@redhat.com>
> > Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> > Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> > Tested-by: Corinna Vinschen <vinschen@redhat.com>
> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > ---
> > v2: fix subject, add a simple reproducer
> >
> >  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> > index a3f100769e39..22fb2c322bca 100644
> > --- a/drivers/net/ethernet/intel/igb/igb_main.c
> > +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> > @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
> >
> >  #if (PAGE_SIZE < 8192)
> >         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> > +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
> 
> I wonder if adding IGB_SKB_PAD would be needed ?
> 
> adapter->max_frame_size does not seem to include it.
> 
> I would try using all mtus between 1200 and 1280 to make sure this works.

Erm... did you mean between 1500 and 1580 by any chance?  1200 doesn't
really seem to make sense...

I tested this patch now with mtu 1500, 1540 and 1580 successfully.

Either way, I'm just heading into vacation, so I guess I'll pick this up
again when I'm back, unless Paolo takes another look during my absence.


Thanks,
Corinna


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Intel-wired-lan] [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-26 14:30     ` Corinna Vinschen
  0 siblings, 0 replies; 10+ messages in thread
From: Corinna Vinschen @ 2024-04-26 14:30 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Jason Xing, netdev, Nikolay Aleksandrov, linux-kernel,
	Tony Nguyen, intel-wired-lan, Jakub Kicinski, Paolo Abeni,
	David S . Miller

Hi Eric,

On Apr 23 16:10, Eric Dumazet wrote:
> On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
> >
> > From: Paolo Abeni <pabeni@redhat.com>
> >
> > Sabrina reports that the igb driver does not cope well with large
> > MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> > corruption on TX.
> >
> > An easy reproducer is to run ssh to connect to the machine.  With
> > MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> >
> > The root cause of the issue is that the driver does not take into
> > account properly the (possibly large) shared info size when selecting
> > the ring layout, and will try to fit two packets inside the same 4K
> > page even when the 1st fraglist will trump over the 2nd head.
> >
> > Address the issue forcing the driver to fit a single packet per page,
> > leaving there enough room to store the (currently) largest possible
> > skb_shared_info.
> >
> > Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> > Reported-by: Jan Tluka <jtluka@redhat.com>
> > Reported-by: Jirka Hladky <jhladky@redhat.com>
> > Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> > Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> > Tested-by: Corinna Vinschen <vinschen@redhat.com>
> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > ---
> > v2: fix subject, add a simple reproducer
> >
> >  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> > index a3f100769e39..22fb2c322bca 100644
> > --- a/drivers/net/ethernet/intel/igb/igb_main.c
> > +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> > @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
> >
> >  #if (PAGE_SIZE < 8192)
> >         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> > +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
> 
> I wonder if adding IGB_SKB_PAD would be needed ?
> 
> adapter->max_frame_size does not seem to include it.
> 
> I would try using all mtus between 1200 and 1280 to make sure this works.

Erm... did you mean between 1500 and 1580 by any chance?  1200 doesn't
really seem to make sense...

I tested this patch now with mtu 1500, 1540 and 1580 successfully.

Either way, I'm just heading into vacation, so I guess I'll pick this up
again when I'm back, unless Paolo takes another look during my absence.


Thanks,
Corinna


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Intel-wired-lan] [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
  2024-04-26 14:30     ` [Intel-wired-lan] " Corinna Vinschen
@ 2024-04-26 17:54       ` Eric Dumazet
  -1 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2024-04-26 17:54 UTC (permalink / raw)
  To: Eric Dumazet, netdev, linux-kernel, intel-wired-lan,
	Nikolay Aleksandrov, Jason Xing, Paolo Abeni, Jakub Kicinski,
	David S . Miller, Tony Nguyen, Jesse Brandeburg

On Fri, Apr 26, 2024 at 4:30 PM Corinna Vinschen <vinschen@redhat.com> wrote:
>
> Hi Eric,
>
> On Apr 23 16:10, Eric Dumazet wrote:
> > On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
> > >
> > > From: Paolo Abeni <pabeni@redhat.com>
> > >
> > > Sabrina reports that the igb driver does not cope well with large
> > > MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> > > corruption on TX.
> > >
> > > An easy reproducer is to run ssh to connect to the machine.  With
> > > MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> > >
> > > The root cause of the issue is that the driver does not take into
> > > account properly the (possibly large) shared info size when selecting
> > > the ring layout, and will try to fit two packets inside the same 4K
> > > page even when the 1st fraglist will trump over the 2nd head.
> > >
> > > Address the issue forcing the driver to fit a single packet per page,
> > > leaving there enough room to store the (currently) largest possible
> > > skb_shared_info.
> > >
> > > Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> > > Reported-by: Jan Tluka <jtluka@redhat.com>
> > > Reported-by: Jirka Hladky <jhladky@redhat.com>
> > > Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> > > Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> > > Tested-by: Corinna Vinschen <vinschen@redhat.com>
> > > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > > ---
> > > v2: fix subject, add a simple reproducer
> > >
> > >  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> > > index a3f100769e39..22fb2c322bca 100644
> > > --- a/drivers/net/ethernet/intel/igb/igb_main.c
> > > +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> > > @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
> > >
> > >  #if (PAGE_SIZE < 8192)
> > >         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> > > +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
> >
> > I wonder if adding IGB_SKB_PAD would be needed ?
> >
> > adapter->max_frame_size does not seem to include it.
> >
> > I would try using all mtus between 1200 and 1280 to make sure this works.
>
> Erm... did you mean between 1500 and 1580 by any chance?  1200 doesn't
> really seem to make sense...

No, I meant 1200 to 1280 .  IPv4 should accept these MTU .

1200 + 768 = 1968
1280 + 768 = 2048 (2 KB)

I am worried of some padding that would cross 2048 bytes boundary,
while SKB_HEAD_ALIGN(adapter->max_frame_size) could still be < 2048


>
> I tested this patch now with mtu 1500, 1540 and 1580 successfully.
>
> Either way, I'm just heading into vacation, so I guess I'll pick this up
> again when I'm back, unless Paolo takes another look during my absence.
>

I guess your patch is better than nothing, this can be refined if
necessary later.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-26 17:54       ` Eric Dumazet
  0 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2024-04-26 17:54 UTC (permalink / raw)
  To: Eric Dumazet, netdev, linux-kernel, intel-wired-lan,
	Nikolay Aleksandrov, Jason Xing, Paolo Abeni, Jakub Kicinski,
	David S . Miller, Tony Nguyen, Jesse Brandeburg

On Fri, Apr 26, 2024 at 4:30 PM Corinna Vinschen <vinschen@redhat.com> wrote:
>
> Hi Eric,
>
> On Apr 23 16:10, Eric Dumazet wrote:
> > On Tue, Apr 23, 2024 at 3:47 PM Corinna Vinschen <vinschen@redhat.com> wrote:
> > >
> > > From: Paolo Abeni <pabeni@redhat.com>
> > >
> > > Sabrina reports that the igb driver does not cope well with large
> > > MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> > > corruption on TX.
> > >
> > > An easy reproducer is to run ssh to connect to the machine.  With
> > > MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> > >
> > > The root cause of the issue is that the driver does not take into
> > > account properly the (possibly large) shared info size when selecting
> > > the ring layout, and will try to fit two packets inside the same 4K
> > > page even when the 1st fraglist will trump over the 2nd head.
> > >
> > > Address the issue forcing the driver to fit a single packet per page,
> > > leaving there enough room to store the (currently) largest possible
> > > skb_shared_info.
> > >
> > > Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")
> > > Reported-by: Jan Tluka <jtluka@redhat.com>
> > > Reported-by: Jirka Hladky <jhladky@redhat.com>
> > > Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> > > Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> > > Tested-by: Corinna Vinschen <vinschen@redhat.com>
> > > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > > ---
> > > v2: fix subject, add a simple reproducer
> > >
> > >  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> > > index a3f100769e39..22fb2c322bca 100644
> > > --- a/drivers/net/ethernet/intel/igb/igb_main.c
> > > +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> > > @@ -4833,6 +4833,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
> > >
> > >  #if (PAGE_SIZE < 8192)
> > >         if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
> > > +           SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
> >
> > I wonder if adding IGB_SKB_PAD would be needed ?
> >
> > adapter->max_frame_size does not seem to include it.
> >
> > I would try using all mtus between 1200 and 1280 to make sure this works.
>
> Erm... did you mean between 1500 and 1580 by any chance?  1200 doesn't
> really seem to make sense...

No, I meant 1200 to 1280 .  IPv4 should accept these MTU .

1200 + 768 = 1968
1280 + 768 = 2048 (2 KB)

I am worried of some padding that would cross 2048 bytes boundary,
while SKB_HEAD_ALIGN(adapter->max_frame_size) could still be < 2048


>
> I tested this patch now with mtu 1500, 1540 and 1580 successfully.
>
> Either way, I'm just heading into vacation, so I guess I'll pick this up
> again when I'm back, unless Paolo takes another look during my absence.
>

I guess your patch is better than nothing, this can be refined if
necessary later.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
  2024-04-23 13:47 ` [Intel-wired-lan] " Corinna Vinschen
@ 2024-04-28 13:05   ` Simon Horman
  -1 siblings, 0 replies; 10+ messages in thread
From: Simon Horman @ 2024-04-28 13:05 UTC (permalink / raw)
  To: Corinna Vinschen
  Cc: netdev, linux-kernel, intel-wired-lan, Nikolay Aleksandrov,
	Jason Xing, Paolo Abeni, Jakub Kicinski, Eric Dumazet,
	David S . Miller, Tony Nguyen, Jesse Brandeburg

On Tue, Apr 23, 2024 at 03:47:31PM +0200, Corinna Vinschen wrote:
> From: Paolo Abeni <pabeni@redhat.com>
> 
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
> 
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> 
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
> 
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
> 
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")

nit: The trailing "S" in the subject for the fixes tag seems to have been lost.

Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")

> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>

...

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Intel-wired-lan] [PATCH net v2] igb: cope with large MAX_SKB_FRAGS
@ 2024-04-28 13:05   ` Simon Horman
  0 siblings, 0 replies; 10+ messages in thread
From: Simon Horman @ 2024-04-28 13:05 UTC (permalink / raw)
  To: Corinna Vinschen
  Cc: Jason Xing, Tony Nguyen, netdev, Nikolay Aleksandrov,
	linux-kernel, Eric Dumazet, intel-wired-lan, Jakub Kicinski,
	Paolo Abeni, David S . Miller

On Tue, Apr 23, 2024 at 03:47:31PM +0200, Corinna Vinschen wrote:
> From: Paolo Abeni <pabeni@redhat.com>
> 
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
> 
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> 
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
> 
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
> 
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAG")

nit: The trailing "S" in the subject for the fixes tag seems to have been lost.

Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")

> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>

...

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2024-04-28 13:07 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-23 13:47 [PATCH net v2] igb: cope with large MAX_SKB_FRAGS Corinna Vinschen
2024-04-23 13:47 ` [Intel-wired-lan] " Corinna Vinschen
2024-04-23 14:10 ` Eric Dumazet
2024-04-23 14:10   ` [Intel-wired-lan] " Eric Dumazet
2024-04-26 14:30   ` Corinna Vinschen
2024-04-26 14:30     ` [Intel-wired-lan] " Corinna Vinschen
2024-04-26 17:54     ` Eric Dumazet
2024-04-26 17:54       ` Eric Dumazet
2024-04-28 13:05 ` Simon Horman
2024-04-28 13:05   ` [Intel-wired-lan] " Simon Horman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.