[PATCH] tun: make tun_build_skb() thread safe

[PATCH] tun: make tun_build_skb() thread safe

[Jason Wang]

From: Eric Dumazet <eric.dumazet@gmail.com>

tun_build_skb() is not thread safe since it uses per queue page frag,
this will break things when multiple threads are sending through same
queue. Switch to use per-thread generator (no lock involved).

Fixes: 66ccbc9c87c2 ("tap: use build_skb() for small packet")
Tested-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/tun.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 5892284..c38cd84 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -175,7 +175,6 @@ struct tun_file {
 	struct list_head next;
 	struct tun_struct *detached;
 	struct skb_array tx_array;
-	struct page_frag alloc_frag;
 };
 
 struct tun_flow_entry {
@@ -578,8 +577,6 @@ static void __tun_detach(struct tun_file *tfile, bool clean)
 		}
 		if (tun)
 			skb_array_cleanup(&tfile->tx_array);
-		if (tfile->alloc_frag.page)
-			put_page(tfile->alloc_frag.page);
 		sock_put(&tfile->sk);
 	}
 }
@@ -1272,7 +1269,7 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
 				     struct virtio_net_hdr *hdr,
 				     int len, int *generic_xdp)
 {
-	struct page_frag *alloc_frag = &tfile->alloc_frag;
+	struct page_frag *alloc_frag = &current->task_frag;
 	struct sk_buff *skb;
 	struct bpf_prog *xdp_prog;
 	int buflen = SKB_DATA_ALIGN(len + TUN_RX_PAD) +
@@ -2580,8 +2577,6 @@ static int tun_chr_open(struct inode *inode, struct file * file)
 	tfile->sk.sk_write_space = tun_sock_write_space;
 	tfile->sk.sk_sndbuf = INT_MAX;
 
-	tfile->alloc_frag.page = NULL;
-
 	file->private_data = tfile;
 	INIT_LIST_HEAD(&tfile->next);
 
-- 
2.7.4

Re: [PATCH] tun: make tun_build_skb() thread safe

[Jason Wang]

On 2017年08月16日 22:14, Jason Wang wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
>
> tun_build_skb() is not thread safe since it uses per queue page frag,
> this will break things when multiple threads are sending through same
> queue. Switch to use per-thread generator (no lock involved).
>
> Fixes: 66ccbc9c87c2 ("tap: use build_skb() for small packet")
> Tested-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>   drivers/net/tun.c | 7 +------

Forget to mention, this is for net-next.

Thanks

Re: [PATCH] tun: make tun_build_skb() thread safe

[Michael S. Tsirkin]

On Wed, Aug 16, 2017 at 10:14:33PM +0800, Jason Wang wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> 
> tun_build_skb() is not thread safe since it uses per queue page frag,
> this will break things when multiple threads are sending through same
> queue. Switch to use per-thread generator (no lock involved).
> 
> Fixes: 66ccbc9c87c2 ("tap: use build_skb() for small packet")
> Tested-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>

Acked-by: Michael S. Tsirkin <mst@redhat.com>

Jason, given the switch to task_frag, would it be worth it to look at
using higher order allocs along the lines of
5640f7685831e088fe6c2e1f863a6805962f8e81 as well?

> ---
>  drivers/net/tun.c | 7 +------
>  1 file changed, 1 insertion(+), 6 deletions(-)
> 
> diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> index 5892284..c38cd84 100644
> --- a/drivers/net/tun.c
> +++ b/drivers/net/tun.c
> @@ -175,7 +175,6 @@ struct tun_file {
>  	struct list_head next;
>  	struct tun_struct *detached;
>  	struct skb_array tx_array;
> -	struct page_frag alloc_frag;
>  };
>  
>  struct tun_flow_entry {
> @@ -578,8 +577,6 @@ static void __tun_detach(struct tun_file *tfile, bool clean)
>  		}
>  		if (tun)
>  			skb_array_cleanup(&tfile->tx_array);
> -		if (tfile->alloc_frag.page)
> -			put_page(tfile->alloc_frag.page);
>  		sock_put(&tfile->sk);
>  	}
>  }
> @@ -1272,7 +1269,7 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
>  				     struct virtio_net_hdr *hdr,
>  				     int len, int *generic_xdp)
>  {
> -	struct page_frag *alloc_frag = &tfile->alloc_frag;
> +	struct page_frag *alloc_frag = &current->task_frag;
>  	struct sk_buff *skb;
>  	struct bpf_prog *xdp_prog;
>  	int buflen = SKB_DATA_ALIGN(len + TUN_RX_PAD) +
> @@ -2580,8 +2577,6 @@ static int tun_chr_open(struct inode *inode, struct file * file)
>  	tfile->sk.sk_write_space = tun_sock_write_space;
>  	tfile->sk.sk_sndbuf = INT_MAX;
>  
> -	tfile->alloc_frag.page = NULL;
> -
>  	file->private_data = tfile;
>  	INIT_LIST_HEAD(&tfile->next);
>  
> -- 
> 2.7.4

Re: [PATCH] tun: make tun_build_skb() thread safe

[David Miller]

From: Jason Wang <jasowang@redhat.com>
Date: Wed, 16 Aug 2017 22:14:33 +0800

> From: Eric Dumazet <eric.dumazet@gmail.com>
> 
> tun_build_skb() is not thread safe since it uses per queue page frag,
> this will break things when multiple threads are sending through same
> queue. Switch to use per-thread generator (no lock involved).
> 
> Fixes: 66ccbc9c87c2 ("tap: use build_skb() for small packet")
> Tested-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>

Applied, thanks.

Re: [PATCH] tun: make tun_build_skb() thread safe

[Jason Wang]

On 2017年08月17日 00:55, Michael S. Tsirkin wrote:
> On Wed, Aug 16, 2017 at 10:14:33PM +0800, Jason Wang wrote:
>> From: Eric Dumazet<eric.dumazet@gmail.com>
>>
>> tun_build_skb() is not thread safe since it uses per queue page frag,
>> this will break things when multiple threads are sending through same
>> queue. Switch to use per-thread generator (no lock involved).
>>
>> Fixes: 66ccbc9c87c2 ("tap: use build_skb() for small packet")
>> Tested-by: Jason Wang<jasowang@redhat.com>
>> Signed-off-by: Eric Dumazet<eric.dumazet@gmail.com>
>> Signed-off-by: Jason Wang<jasowang@redhat.com>
> Acked-by: Michael S. Tsirkin<mst@redhat.com>
>
> Jason, given the switch to task_frag, would it be worth it to look at
> using higher order allocs along the lines of
> 5640f7685831e088fe6c2e1f863a6805962f8e81 as well?
>

I think we've already used high order, don't we?

Thanks