* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-06 17:45 Craig Gallek
2016-07-08 6:19 ` Michael S. Tsirkin
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Craig Gallek @ 2016-07-06 17:45 UTC (permalink / raw)
To: Jason Wang
Cc: mst, netdev, LKML, David Miller, kvm, virtualization,
Eric Dumazet, brouer
On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> Hi all:
>
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
I'm seeing the splat below after this series. I'm still wrapping my
head around this code, but it appears to be happening because the
tun_struct passed into tun_queue_resize is uninitialized.
Specifically, iteration over the disabled list_head fails because prev
= next = NULL. This seems to happen when a startup script on my test
machine changes the queue length. I'll try to figure out what's
happening, but if it's obvious to someone else from the stack, please
let me know.
[ 72.322236] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000010
[ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
[ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
[ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 72.345498] gsmi: Log Shutdown Reason 0x03
[ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
[ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
4.7.0-dbx-DEV #10
[ 72.360253] mlx4_en: eth0: Link Up
[ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
BIOS 2.50.0 01/21/2016
[ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
ffff883f3ef08000
[ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
tun_device_event+0x110/0x340
[ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
[ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
[ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
[ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
[ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
[ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
[ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
knlGS:0000000000000000
[ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
[ 72.455555] Stack:
[ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
ffff887ef9880000
[ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
ffffffff00000000
[ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
ffffffff81cec7a0
[ 72.479559] Call Trace:
[ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
[ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
[ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
[ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
[ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
[ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
[ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
[ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
[ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
[ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
[ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
[ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
[ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
[ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
[ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
[ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
[ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
[ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
[ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
[ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
[ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
ff ff
[ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
[ 72.623883] RSP <ffff883f3ef0bbe8>
[ 72.627327] CR2: 0000000000000010
[ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-07-06 17:45 [PATCH net-next V4 0/6] switch to use tx skb array in tun Craig Gallek
@ 2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
2 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-07-08 6:19 UTC (permalink / raw)
To: Craig Gallek
Cc: Jason Wang, netdev, LKML, David Miller, kvm, virtualization,
Eric Dumazet, brouer
On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
> On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> > Hi all:
> >
> > This series tries to switch to use skb array in tun. This is used to
> > eliminate the spinlock contention between producer and consumer. The
> > conversion was straightforward: just introdce a tx skb array and use
> > it instead of sk_receive_queue.
>
> I'm seeing the splat below after this series. I'm still wrapping my
> head around this code, but it appears to be happening because the
> tun_struct passed into tun_queue_resize is uninitialized.
> Specifically, iteration over the disabled list_head fails because prev
> = next = NULL. This seems to happen when a startup script on my test
> machine changes the queue length. I'll try to figure out what's
> happening, but if it's obvious to someone else from the stack, please
> let me know.
Don't see anything obvious. I'm traveling, will look at it when I'm back
unless it's fixed by then. Jason, any idea?
> [ 72.322236] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000010
> [ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
> [ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
> [ 72.345498] gsmi: Log Shutdown Reason 0x03
> [ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
> ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
> [ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
> 4.7.0-dbx-DEV #10
> [ 72.360253] mlx4_en: eth0: Link Up
> [ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
> BIOS 2.50.0 01/21/2016
> [ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
> ffff883f3ef08000
> [ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
> tun_device_event+0x110/0x340
> [ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
> [ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
> [ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
> [ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
> [ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
> [ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
> [ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
> knlGS:0000000000000000
> [ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
> [ 72.455555] Stack:
> [ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
> ffff887ef9880000
> [ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
> ffffffff00000000
> [ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
> ffffffff81cec7a0
> [ 72.479559] Call Trace:
> [ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
> [ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
> [ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
> [ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
> [ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
> [ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
> [ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
> [ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
> [ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
> [ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
> [ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
> [ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
> [ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
> [ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
> [ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
> [ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
> [ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
> [ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
> [ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
> 00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
> 41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
> ff ff
> [ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.623883] RSP <ffff883f3ef0bbe8>
> [ 72.627327] CR2: 0000000000000010
> [ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-08 6:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-07-08 6:19 UTC (permalink / raw)
To: Craig Gallek
Cc: Eric Dumazet, kvm, netdev, LKML, virtualization, brouer, David Miller
On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
> On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> > Hi all:
> >
> > This series tries to switch to use skb array in tun. This is used to
> > eliminate the spinlock contention between producer and consumer. The
> > conversion was straightforward: just introdce a tx skb array and use
> > it instead of sk_receive_queue.
>
> I'm seeing the splat below after this series. I'm still wrapping my
> head around this code, but it appears to be happening because the
> tun_struct passed into tun_queue_resize is uninitialized.
> Specifically, iteration over the disabled list_head fails because prev
> = next = NULL. This seems to happen when a startup script on my test
> machine changes the queue length. I'll try to figure out what's
> happening, but if it's obvious to someone else from the stack, please
> let me know.
Don't see anything obvious. I'm traveling, will look at it when I'm back
unless it's fixed by then. Jason, any idea?
> [ 72.322236] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000010
> [ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
> [ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
> [ 72.345498] gsmi: Log Shutdown Reason 0x03
> [ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
> ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
> [ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
> 4.7.0-dbx-DEV #10
> [ 72.360253] mlx4_en: eth0: Link Up
> [ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
> BIOS 2.50.0 01/21/2016
> [ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
> ffff883f3ef08000
> [ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
> tun_device_event+0x110/0x340
> [ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
> [ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
> [ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
> [ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
> [ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
> [ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
> [ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
> knlGS:0000000000000000
> [ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
> [ 72.455555] Stack:
> [ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
> ffff887ef9880000
> [ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
> ffffffff00000000
> [ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
> ffffffff81cec7a0
> [ 72.479559] Call Trace:
> [ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
> [ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
> [ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
> [ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
> [ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
> [ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
> [ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
> [ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
> [ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
> [ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
> [ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
> [ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
> [ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
> [ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
> [ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
> [ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
> [ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
> [ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
> [ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
> 00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
> 41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
> ff ff
> [ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.623883] RSP <ffff883f3ef0bbe8>
> [ 72.627327] CR2: 0000000000000010
> [ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-07-06 17:45 [PATCH net-next V4 0/6] switch to use tx skb array in tun Craig Gallek
2016-07-08 6:19 ` Michael S. Tsirkin
@ 2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
2 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-07-08 6:19 UTC (permalink / raw)
To: Craig Gallek
Cc: Jason Wang, netdev, LKML, David Miller, kvm, virtualization,
Eric Dumazet, brouer
On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
> On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> > Hi all:
> >
> > This series tries to switch to use skb array in tun. This is used to
> > eliminate the spinlock contention between producer and consumer. The
> > conversion was straightforward: just introdce a tx skb array and use
> > it instead of sk_receive_queue.
>
> I'm seeing the splat below after this series. I'm still wrapping my
> head around this code, but it appears to be happening because the
> tun_struct passed into tun_queue_resize is uninitialized.
> Specifically, iteration over the disabled list_head fails because prev
> = next = NULL. This seems to happen when a startup script on my test
> machine changes the queue length. I'll try to figure out what's
> happening, but if it's obvious to someone else from the stack, please
> let me know.
Uploading your .config might help BTW.
> [ 72.322236] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000010
> [ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
> [ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
> [ 72.345498] gsmi: Log Shutdown Reason 0x03
> [ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
> ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
> [ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
> 4.7.0-dbx-DEV #10
> [ 72.360253] mlx4_en: eth0: Link Up
> [ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
> BIOS 2.50.0 01/21/2016
> [ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
> ffff883f3ef08000
> [ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
> tun_device_event+0x110/0x340
> [ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
> [ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
> [ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
> [ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
> [ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
> [ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
> [ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
> knlGS:0000000000000000
> [ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
> [ 72.455555] Stack:
> [ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
> ffff887ef9880000
> [ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
> ffffffff00000000
> [ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
> ffffffff81cec7a0
> [ 72.479559] Call Trace:
> [ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
> [ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
> [ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
> [ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
> [ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
> [ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
> [ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
> [ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
> [ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
> [ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
> [ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
> [ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
> [ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
> [ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
> [ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
> [ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
> [ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
> [ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
> [ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
> 00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
> 41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
> ff ff
> [ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.623883] RSP <ffff883f3ef0bbe8>
> [ 72.627327] CR2: 0000000000000010
> [ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-07-06 17:45 [PATCH net-next V4 0/6] switch to use tx skb array in tun Craig Gallek
2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
@ 2016-07-08 6:19 ` Michael S. Tsirkin
2 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-07-08 6:19 UTC (permalink / raw)
To: Craig Gallek
Cc: Eric Dumazet, kvm, netdev, LKML, virtualization, brouer, David Miller
On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
> On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> > Hi all:
> >
> > This series tries to switch to use skb array in tun. This is used to
> > eliminate the spinlock contention between producer and consumer. The
> > conversion was straightforward: just introdce a tx skb array and use
> > it instead of sk_receive_queue.
>
> I'm seeing the splat below after this series. I'm still wrapping my
> head around this code, but it appears to be happening because the
> tun_struct passed into tun_queue_resize is uninitialized.
> Specifically, iteration over the disabled list_head fails because prev
> = next = NULL. This seems to happen when a startup script on my test
> machine changes the queue length. I'll try to figure out what's
> happening, but if it's obvious to someone else from the stack, please
> let me know.
Uploading your .config might help BTW.
> [ 72.322236] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000010
> [ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
> [ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
> [ 72.345498] gsmi: Log Shutdown Reason 0x03
> [ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
> ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
> [ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
> 4.7.0-dbx-DEV #10
> [ 72.360253] mlx4_en: eth0: Link Up
> [ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
> BIOS 2.50.0 01/21/2016
> [ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
> ffff883f3ef08000
> [ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
> tun_device_event+0x110/0x340
> [ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
> [ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
> [ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
> [ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
> [ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
> [ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
> [ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
> knlGS:0000000000000000
> [ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
> [ 72.455555] Stack:
> [ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
> ffff887ef9880000
> [ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
> ffffffff00000000
> [ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
> ffffffff81cec7a0
> [ 72.479559] Call Trace:
> [ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
> [ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
> [ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
> [ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
> [ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
> [ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
> [ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
> [ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
> [ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
> [ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
> [ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
> [ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
> [ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
> [ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
> [ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
> [ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
> [ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
> [ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
> [ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
> [ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
> 00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
> 41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
> ff ff
> [ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
> [ 72.623883] RSP <ffff883f3ef0bbe8>
> [ 72.627327] CR2: 0000000000000010
> [ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-07-08 6:19 ` Michael S. Tsirkin
@ 2016-07-08 9:14 ` Jason Wang
-1 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2016-07-08 9:14 UTC (permalink / raw)
To: Michael S. Tsirkin, Craig Gallek
Cc: netdev, LKML, David Miller, kvm, virtualization, Eric Dumazet, brouer
On 2016年07月08日 14:19, Michael S. Tsirkin wrote:
> On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
>> >On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang<jasowang@redhat.com> wrote:
>>> > >Hi all:
>>> > >
>>> > >This series tries to switch to use skb array in tun. This is used to
>>> > >eliminate the spinlock contention between producer and consumer. The
>>> > >conversion was straightforward: just introdce a tx skb array and use
>>> > >it instead of sk_receive_queue.
>> >
>> >I'm seeing the splat below after this series. I'm still wrapping my
>> >head around this code, but it appears to be happening because the
>> >tun_struct passed into tun_queue_resize is uninitialized.
>> >Specifically, iteration over the disabled list_head fails because prev
>> >= next = NULL. This seems to happen when a startup script on my test
>> >machine changes the queue length. I'll try to figure out what's
>> >happening, but if it's obvious to someone else from the stack, please
>> >let me know.
> Don't see anything obvious. I'm traveling, will look at it when I'm back
> unless it's fixed by then. Jason, any idea?
>
Looks like Craig has posted a fix to this:
http://patchwork.ozlabs.org/patch/645645/
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-08 9:14 ` Jason Wang
0 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2016-07-08 9:14 UTC (permalink / raw)
To: Michael S. Tsirkin, Craig Gallek
Cc: kvm, Eric Dumazet, netdev, LKML, virtualization, brouer, David Miller
On 2016年07月08日 14:19, Michael S. Tsirkin wrote:
> On Wed, Jul 06, 2016 at 01:45:58PM -0400, Craig Gallek wrote:
>> >On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang<jasowang@redhat.com> wrote:
>>> > >Hi all:
>>> > >
>>> > >This series tries to switch to use skb array in tun. This is used to
>>> > >eliminate the spinlock contention between producer and consumer. The
>>> > >conversion was straightforward: just introdce a tx skb array and use
>>> > >it instead of sk_receive_queue.
>> >
>> >I'm seeing the splat below after this series. I'm still wrapping my
>> >head around this code, but it appears to be happening because the
>> >tun_struct passed into tun_queue_resize is uninitialized.
>> >Specifically, iteration over the disabled list_head fails because prev
>> >= next = NULL. This seems to happen when a startup script on my test
>> >machine changes the queue length. I'll try to figure out what's
>> >happening, but if it's obvious to someone else from the stack, please
>> >let me know.
> Don't see anything obvious. I'm traveling, will look at it when I'm back
> unless it's fixed by then. Jason, any idea?
>
Looks like Craig has posted a fix to this:
http://patchwork.ozlabs.org/patch/645645/
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-06 17:45 Craig Gallek
0 siblings, 0 replies; 16+ messages in thread
From: Craig Gallek @ 2016-07-06 17:45 UTC (permalink / raw)
To: Jason Wang
Cc: Eric Dumazet, kvm, mst, netdev, LKML, virtualization, brouer,
David Miller
On Thu, Jun 30, 2016 at 2:45 AM, Jason Wang <jasowang@redhat.com> wrote:
> Hi all:
>
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
I'm seeing the splat below after this series. I'm still wrapping my
head around this code, but it appears to be happening because the
tun_struct passed into tun_queue_resize is uninitialized.
Specifically, iteration over the disabled list_head fails because prev
= next = NULL. This seems to happen when a startup script on my test
machine changes the queue length. I'll try to figure out what's
happening, but if it's obvious to someone else from the stack, please
let me know.
[ 72.322236] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000010
[ 72.329993] IP: [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
[ 72.336032] PGD 7f054f1067 PUD 7ef6f3f067 PMD 0
[ 72.340616] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 72.345498] gsmi: Log Shutdown Reason 0x03
[ 72.349541] Modules linked in: w1_therm wire cdc_acm ehci_pci
ehci_hcd mlx4_en ib_uverbs mlx4_ib ib_core mlx4_core
[ 72.359870] CPU: 12 PID: 7820 Comm: set.ixion-haswe Not tainted
4.7.0-dbx-DEV #10
[ 72.360253] mlx4_en: eth0: Link Up
[ 72.370618] Hardware name: Intel Grantley,Wellsburg/Ixion_IT_15,
BIOS 2.50.0 01/21/2016
[ 72.378525] task: ffff883f2501e8c0 ti: ffff883f3ef08000 task.ti:
ffff883f3ef08000
[ 72.385917] RIP: 0010:[<ffffffff8153c1a0>] [<ffffffff8153c1a0>]
tun_device_event+0x110/0x340
[ 72.394353] RSP: 0018:ffff883f3ef0bbe8 EFLAGS: 00010202
[ 72.399599] RAX: fffffffffffffae8 RBX: ffff887ef9883378 RCX: 0000000000000000
[ 72.406647] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000000
[ 72.413694] RBP: ffff883f3ef0bc58 R08: 0000000000000000 R09: 0000000000000001
[ 72.420742] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000010
[ 72.427789] R13: 0000000000000000 R14: 0000000000000001 R15: ffff883f3ef0bd10
[ 72.434837] FS: 00007fac4e5dd700(0000) GS:ffff883f7f700000(0000)
knlGS:0000000000000000
[ 72.442832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 72.448507] CR2: 0000000000000010 CR3: 0000007ef66ac000 CR4: 00000000001406e0
[ 72.455555] Stack:
[ 72.457541] ffff883f3ef0bc18 0000000000000246 000000000000001e
ffff887ef9880000
[ 72.464880] ffff883f3ef0bd10 0000000000000000 0000000000000000
ffffffff00000000
[ 72.472219] ffff883f3ef0bc58 ffffffff81d24070 00000000fffffff9
ffffffff81cec7a0
[ 72.479559] Call Trace:
[ 72.481986] [<ffffffff810eeb0d>] notifier_call_chain+0x5d/0x80
[ 72.487844] [<ffffffff816365c0>] ? show_tx_maxrate+0x30/0x30
[ 72.493520] [<ffffffff810eeb3e>] __raw_notifier_call_chain+0xe/0x10
[ 72.499801] [<ffffffff810eeb56>] raw_notifier_call_chain+0x16/0x20
[ 72.506001] [<ffffffff8160eb30>] call_netdevice_notifiers_info+0x40/0x70
[ 72.512706] [<ffffffff8160ec36>] call_netdevice_notifiers+0x16/0x20
[ 72.518986] [<ffffffff816365f8>] change_tx_queue_len+0x38/0x80
[ 72.524838] [<ffffffff816381cf>] netdev_store.isra.5+0xbf/0xd0
[ 72.530688] [<ffffffff81638330>] tx_queue_len_store+0x50/0x60
[ 72.536459] [<ffffffff814a6798>] dev_attr_store+0x18/0x30
[ 72.541888] [<ffffffff812ea3ff>] sysfs_kf_write+0x4f/0x70
[ 72.547306] [<ffffffff812e9507>] kernfs_fop_write+0x147/0x1d0
[ 72.553077] [<ffffffff81134a4f>] ? rcu_read_lock_sched_held+0x8f/0xa0
[ 72.559534] [<ffffffff8125a108>] __vfs_write+0x28/0x120
[ 72.564781] [<ffffffff8111b137>] ? percpu_down_read+0x57/0x90
[ 72.570542] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
[ 72.576303] [<ffffffff8125d7d8>] ? __sb_start_write+0xc8/0xe0
[ 72.582063] [<ffffffff8125bd5e>] vfs_write+0xbe/0x1b0
[ 72.587138] [<ffffffff8125c092>] SyS_write+0x52/0xa0
[ 72.592135] [<ffffffff817528a5>] entry_SYSCALL_64_fastpath+0x18/0xa8
[ 72.598497] Code: 45 31 f6 48 8b 93 78 33 00 00 48 81 c3 78 33 00
00 48 39 d3 48 8d 82 e8 fa ff ff 74 25 48 8d b0 40 05 00 00 49 63 d6
41 83 c6 01 <49> 89 34 d4 48 8b 90 18 05 00 00 48 39 d3 48 8d 82 e8 fa
ff ff
[ 72.617767] RIP [<ffffffff8153c1a0>] tun_device_event+0x110/0x340
[ 72.623883] RSP <ffff883f3ef0bbe8>
[ 72.627327] CR2: 0000000000000010
[ 72.630638] ---[ end trace b0c54137cf861b91 ]---
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-06-30 6:45 Jason Wang
@ 2016-07-01 9:40 ` David Miller
2016-06-30 15:45 ` Michael S. Tsirkin
2016-07-01 9:40 ` David Miller
2 siblings, 0 replies; 16+ messages in thread
From: David Miller @ 2016-07-01 9:40 UTC (permalink / raw)
To: jasowang
Cc: mst, netdev, linux-kernel, kvm, virtualization, eric.dumazet, brouer
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 30 Jun 2016 14:45:30 +0800
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
Series applied, thanks Jason.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-01 9:40 ` David Miller
0 siblings, 0 replies; 16+ messages in thread
From: David Miller @ 2016-07-01 9:40 UTC (permalink / raw)
To: jasowang
Cc: kvm, eric.dumazet, mst, netdev, linux-kernel, virtualization, brouer
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 30 Jun 2016 14:45:30 +0800
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
Series applied, thanks Jason.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-06-30 15:45 ` Michael S. Tsirkin
@ 2016-07-01 6:04 ` Jason Wang
0 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2016-07-01 6:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: netdev, linux-kernel, davem, kvm, virtualization, eric.dumazet, brouer
On 2016年06月30日 23:45, Michael S. Tsirkin wrote:
> On Thu, Jun 30, 2016 at 02:45:30PM +0800, Jason Wang wrote:
>> >Hi all:
>> >
>> >This series tries to switch to use skb array in tun. This is used to
>> >eliminate the spinlock contention between producer and consumer. The
>> >conversion was straightforward: just introdce a tx skb array and use
>> >it instead of sk_receive_queue.
>> >
>> >A minor issue is to keep the tx_queue_len behaviour, since tun used to
>> >use it for the length of sk_receive_queue. This is done through:
>> >
>> >- add the ability to resize multiple rings at once to avoid handling
>> > partial resize failure for mutiple rings.
>> >- add the support for zero length ring.
>> >- introduce a notifier which was triggered when tx_queue_len was
>> > changed for a netdev.
>> >- resize all queues during the tx_queue_len changing.
>> >
>> >Tests shows about 15% improvement on guest rx pps:
>> >
>> >Before: ~1300000pps
>> >After : ~1500000pps
> Acked-by: Michael S. Tsirkin<mst@redhat.com>
>
> Acked-from-altitude: 34697 feet.
Wow, thanks a lot!
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-07-01 6:04 ` Jason Wang
0 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2016-07-01 6:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: kvm, eric.dumazet, netdev, linux-kernel, virtualization, brouer, davem
On 2016年06月30日 23:45, Michael S. Tsirkin wrote:
> On Thu, Jun 30, 2016 at 02:45:30PM +0800, Jason Wang wrote:
>> >Hi all:
>> >
>> >This series tries to switch to use skb array in tun. This is used to
>> >eliminate the spinlock contention between producer and consumer. The
>> >conversion was straightforward: just introdce a tx skb array and use
>> >it instead of sk_receive_queue.
>> >
>> >A minor issue is to keep the tx_queue_len behaviour, since tun used to
>> >use it for the length of sk_receive_queue. This is done through:
>> >
>> >- add the ability to resize multiple rings at once to avoid handling
>> > partial resize failure for mutiple rings.
>> >- add the support for zero length ring.
>> >- introduce a notifier which was triggered when tx_queue_len was
>> > changed for a netdev.
>> >- resize all queues during the tx_queue_len changing.
>> >
>> >Tests shows about 15% improvement on guest rx pps:
>> >
>> >Before: ~1300000pps
>> >After : ~1500000pps
> Acked-by: Michael S. Tsirkin<mst@redhat.com>
>
> Acked-from-altitude: 34697 feet.
Wow, thanks a lot!
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-06-30 6:45 Jason Wang
2016-06-30 15:45 ` Michael S. Tsirkin
@ 2016-06-30 15:45 ` Michael S. Tsirkin
2016-07-01 6:04 ` Jason Wang
2016-07-01 9:40 ` David Miller
2 siblings, 1 reply; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-06-30 15:45 UTC (permalink / raw)
To: Jason Wang
Cc: netdev, linux-kernel, davem, kvm, virtualization, eric.dumazet, brouer
On Thu, Jun 30, 2016 at 02:45:30PM +0800, Jason Wang wrote:
> Hi all:
>
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
>
> A minor issue is to keep the tx_queue_len behaviour, since tun used to
> use it for the length of sk_receive_queue. This is done through:
>
> - add the ability to resize multiple rings at once to avoid handling
> partial resize failure for mutiple rings.
> - add the support for zero length ring.
> - introduce a notifier which was triggered when tx_queue_len was
> changed for a netdev.
> - resize all queues during the tx_queue_len changing.
>
> Tests shows about 15% improvement on guest rx pps:
>
> Before: ~1300000pps
> After : ~1500000pps
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-from-altitude: 34697 feet.
> Changes from V3:
> - fix kbuild warnings
> - call NETDEV_CHANGE_TX_QUEUE_LEN on IFLA_TXQLEN
>
> Changes from V2:
> - add multiple rings resizing support for ptr_ring/skb_array
> - add zero length ring support
> - introdce a NETDEV_CHANGE_TX_QUEUE_LEN
> - drop new flags
>
> Changes from V1:
> - switch to use skb array instead of a customized circular buffer
> - add non-blocking support
> - rename .peek to .peek_len
> - drop lockless peeking since test show very minor improvement
>
> Jason Wang (5):
> ptr_ring: support zero length ring
> skb_array: minor tweak
> skb_array: add wrappers for resizing
> net: introduce NETDEV_CHANGE_TX_QUEUE_LEN
> tun: switch to use skb array for tx
>
> Michael S. Tsirkin (1):
> ptr_ring: support resizing multiple queues
>
> drivers/net/tun.c | 138 ++++++++++++++++++++++++++++++++++++---
> drivers/vhost/net.c | 16 ++++-
> include/linux/net.h | 1 +
> include/linux/netdevice.h | 1 +
> include/linux/ptr_ring.h | 77 ++++++++++++++++++----
> include/linux/skb_array.h | 13 +++-
> net/core/net-sysfs.c | 15 ++++-
> net/core/rtnetlink.c | 16 +++--
> tools/virtio/ringtest/ptr_ring.c | 5 ++
> 9 files changed, 255 insertions(+), 27 deletions(-)
>
> --
> 2.7.4
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH net-next V4 0/6] switch to use tx skb array in tun
2016-06-30 6:45 Jason Wang
@ 2016-06-30 15:45 ` Michael S. Tsirkin
2016-06-30 15:45 ` Michael S. Tsirkin
2016-07-01 9:40 ` David Miller
2 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2016-06-30 15:45 UTC (permalink / raw)
To: Jason Wang
Cc: kvm, eric.dumazet, netdev, linux-kernel, virtualization, brouer, davem
On Thu, Jun 30, 2016 at 02:45:30PM +0800, Jason Wang wrote:
> Hi all:
>
> This series tries to switch to use skb array in tun. This is used to
> eliminate the spinlock contention between producer and consumer. The
> conversion was straightforward: just introdce a tx skb array and use
> it instead of sk_receive_queue.
>
> A minor issue is to keep the tx_queue_len behaviour, since tun used to
> use it for the length of sk_receive_queue. This is done through:
>
> - add the ability to resize multiple rings at once to avoid handling
> partial resize failure for mutiple rings.
> - add the support for zero length ring.
> - introduce a notifier which was triggered when tx_queue_len was
> changed for a netdev.
> - resize all queues during the tx_queue_len changing.
>
> Tests shows about 15% improvement on guest rx pps:
>
> Before: ~1300000pps
> After : ~1500000pps
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-from-altitude: 34697 feet.
> Changes from V3:
> - fix kbuild warnings
> - call NETDEV_CHANGE_TX_QUEUE_LEN on IFLA_TXQLEN
>
> Changes from V2:
> - add multiple rings resizing support for ptr_ring/skb_array
> - add zero length ring support
> - introdce a NETDEV_CHANGE_TX_QUEUE_LEN
> - drop new flags
>
> Changes from V1:
> - switch to use skb array instead of a customized circular buffer
> - add non-blocking support
> - rename .peek to .peek_len
> - drop lockless peeking since test show very minor improvement
>
> Jason Wang (5):
> ptr_ring: support zero length ring
> skb_array: minor tweak
> skb_array: add wrappers for resizing
> net: introduce NETDEV_CHANGE_TX_QUEUE_LEN
> tun: switch to use skb array for tx
>
> Michael S. Tsirkin (1):
> ptr_ring: support resizing multiple queues
>
> drivers/net/tun.c | 138 ++++++++++++++++++++++++++++++++++++---
> drivers/vhost/net.c | 16 ++++-
> include/linux/net.h | 1 +
> include/linux/netdevice.h | 1 +
> include/linux/ptr_ring.h | 77 ++++++++++++++++++----
> include/linux/skb_array.h | 13 +++-
> net/core/net-sysfs.c | 15 ++++-
> net/core/rtnetlink.c | 16 +++--
> tools/virtio/ringtest/ptr_ring.c | 5 ++
> 9 files changed, 255 insertions(+), 27 deletions(-)
>
> --
> 2.7.4
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-06-30 6:45 Jason Wang
2016-06-30 15:45 ` Michael S. Tsirkin
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Jason Wang @ 2016-06-30 6:45 UTC (permalink / raw)
To: mst, netdev, linux-kernel, davem
Cc: kvm, virtualization, eric.dumazet, brouer, Jason Wang
Hi all:
This series tries to switch to use skb array in tun. This is used to
eliminate the spinlock contention between producer and consumer. The
conversion was straightforward: just introdce a tx skb array and use
it instead of sk_receive_queue.
A minor issue is to keep the tx_queue_len behaviour, since tun used to
use it for the length of sk_receive_queue. This is done through:
- add the ability to resize multiple rings at once to avoid handling
partial resize failure for mutiple rings.
- add the support for zero length ring.
- introduce a notifier which was triggered when tx_queue_len was
changed for a netdev.
- resize all queues during the tx_queue_len changing.
Tests shows about 15% improvement on guest rx pps:
Before: ~1300000pps
After : ~1500000pps
Changes from V3:
- fix kbuild warnings
- call NETDEV_CHANGE_TX_QUEUE_LEN on IFLA_TXQLEN
Changes from V2:
- add multiple rings resizing support for ptr_ring/skb_array
- add zero length ring support
- introdce a NETDEV_CHANGE_TX_QUEUE_LEN
- drop new flags
Changes from V1:
- switch to use skb array instead of a customized circular buffer
- add non-blocking support
- rename .peek to .peek_len
- drop lockless peeking since test show very minor improvement
Jason Wang (5):
ptr_ring: support zero length ring
skb_array: minor tweak
skb_array: add wrappers for resizing
net: introduce NETDEV_CHANGE_TX_QUEUE_LEN
tun: switch to use skb array for tx
Michael S. Tsirkin (1):
ptr_ring: support resizing multiple queues
drivers/net/tun.c | 138 ++++++++++++++++++++++++++++++++++++---
drivers/vhost/net.c | 16 ++++-
include/linux/net.h | 1 +
include/linux/netdevice.h | 1 +
include/linux/ptr_ring.h | 77 ++++++++++++++++++----
include/linux/skb_array.h | 13 +++-
net/core/net-sysfs.c | 15 ++++-
net/core/rtnetlink.c | 16 +++--
tools/virtio/ringtest/ptr_ring.c | 5 ++
9 files changed, 255 insertions(+), 27 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH net-next V4 0/6] switch to use tx skb array in tun
@ 2016-06-30 6:45 Jason Wang
0 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2016-06-30 6:45 UTC (permalink / raw)
To: mst, netdev, linux-kernel, davem
Cc: brouer, eric.dumazet, kvm, virtualization
Hi all:
This series tries to switch to use skb array in tun. This is used to
eliminate the spinlock contention between producer and consumer. The
conversion was straightforward: just introdce a tx skb array and use
it instead of sk_receive_queue.
A minor issue is to keep the tx_queue_len behaviour, since tun used to
use it for the length of sk_receive_queue. This is done through:
- add the ability to resize multiple rings at once to avoid handling
partial resize failure for mutiple rings.
- add the support for zero length ring.
- introduce a notifier which was triggered when tx_queue_len was
changed for a netdev.
- resize all queues during the tx_queue_len changing.
Tests shows about 15% improvement on guest rx pps:
Before: ~1300000pps
After : ~1500000pps
Changes from V3:
- fix kbuild warnings
- call NETDEV_CHANGE_TX_QUEUE_LEN on IFLA_TXQLEN
Changes from V2:
- add multiple rings resizing support for ptr_ring/skb_array
- add zero length ring support
- introdce a NETDEV_CHANGE_TX_QUEUE_LEN
- drop new flags
Changes from V1:
- switch to use skb array instead of a customized circular buffer
- add non-blocking support
- rename .peek to .peek_len
- drop lockless peeking since test show very minor improvement
Jason Wang (5):
ptr_ring: support zero length ring
skb_array: minor tweak
skb_array: add wrappers for resizing
net: introduce NETDEV_CHANGE_TX_QUEUE_LEN
tun: switch to use skb array for tx
Michael S. Tsirkin (1):
ptr_ring: support resizing multiple queues
drivers/net/tun.c | 138 ++++++++++++++++++++++++++++++++++++---
drivers/vhost/net.c | 16 ++++-
include/linux/net.h | 1 +
include/linux/netdevice.h | 1 +
include/linux/ptr_ring.h | 77 ++++++++++++++++++----
include/linux/skb_array.h | 13 +++-
net/core/net-sysfs.c | 15 ++++-
net/core/rtnetlink.c | 16 +++--
tools/virtio/ringtest/ptr_ring.c | 5 ++
9 files changed, 255 insertions(+), 27 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2016-07-08 9:14 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-06 17:45 [PATCH net-next V4 0/6] switch to use tx skb array in tun Craig Gallek
2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 9:14 ` Jason Wang
2016-07-08 9:14 ` Jason Wang
2016-07-08 6:19 ` Michael S. Tsirkin
2016-07-08 6:19 ` Michael S. Tsirkin
-- strict thread matches above, loose matches on Subject: below --
2016-07-06 17:45 Craig Gallek
2016-06-30 6:45 Jason Wang
2016-06-30 15:45 ` Michael S. Tsirkin
2016-06-30 15:45 ` Michael S. Tsirkin
2016-07-01 6:04 ` Jason Wang
2016-07-01 6:04 ` Jason Wang
2016-07-01 9:40 ` David Miller
2016-07-01 9:40 ` David Miller
2016-06-30 6:45 Jason Wang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.