[PATCH][net-next] tun: align write-heavy flow entry members to a cache line

[PATCH][net-next] tun: align write-heavy flow entry members to a cache line

[Li RongQing]

tun flow entry 'updated' fields are written when receive
every packet. Thus if a flow is receiving packets from a
particular flow entry, it'll cause false-sharing with
all the other who has looked it up, so move it in its own
cache line

and update 'queue_index' and 'update' field only when
they are changed to reduce the cache false-sharing.

Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Signed-off-by: Wang Li <wangli39@baidu.com>
Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
 drivers/net/tun.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 835c73f42ae7..d0745dc81976 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -201,7 +201,7 @@ struct tun_flow_entry {
 	u32 rxhash;
 	u32 rps_rxhash;
 	int queue_index;
-	unsigned long updated;
+	unsigned long updated ____cacheline_aligned_in_smp;
 };
 
 #define TUN_NUM_FLOW_ENTRIES 1024
@@ -539,8 +539,10 @@ static void tun_flow_update(struct tun_struct *tun, u32 rxhash,
 	e = tun_flow_find(head, rxhash);
 	if (likely(e)) {
 		/* TODO: keep queueing to old queue until it's empty? */
-		e->queue_index = queue_index;
-		e->updated = jiffies;
+		if (e->queue_index != queue_index)
+			e->queue_index = queue_index;
+		if (e->updated != jiffies)
+			e->updated = jiffies;
 		sock_rps_record_flow_hash(e->rps_rxhash);
 	} else {
 		spin_lock_bh(&tun->lock);
-- 
2.16.2

Re: [PATCH][net-next] tun: align write-heavy flow entry members to a cache line

[Jason Wang]

On 2018/12/6 下午4:08, Li RongQing wrote:
> tun flow entry 'updated' fields are written when receive
> every packet. Thus if a flow is receiving packets from a
> particular flow entry, it'll cause false-sharing with
> all the other who has looked it up, so move it in its own
> cache line
>
> and update 'queue_index' and 'update' field only when
> they are changed to reduce the cache false-sharing.
>
> Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
> Signed-off-by: Wang Li <wangli39@baidu.com>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>
> ---
>   drivers/net/tun.c | 8 +++++---
>   1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> index 835c73f42ae7..d0745dc81976 100644
> --- a/drivers/net/tun.c
> +++ b/drivers/net/tun.c
> @@ -201,7 +201,7 @@ struct tun_flow_entry {
>   	u32 rxhash;
>   	u32 rps_rxhash;
>   	int queue_index;
> -	unsigned long updated;
> +	unsigned long updated ____cacheline_aligned_in_smp;
>   };
>   
>   #define TUN_NUM_FLOW_ENTRIES 1024
> @@ -539,8 +539,10 @@ static void tun_flow_update(struct tun_struct *tun, u32 rxhash,
>   	e = tun_flow_find(head, rxhash);
>   	if (likely(e)) {
>   		/* TODO: keep queueing to old queue until it's empty? */
> -		e->queue_index = queue_index;
> -		e->updated = jiffies;
> +		if (e->queue_index != queue_index)
> +			e->queue_index = queue_index;
> +		if (e->updated != jiffies)
> +			e->updated = jiffies;


Acked-by: Jason Wang <jasowang@redhat.com>

Btw, do you have perf numbers for this?

Thanks


>   		sock_rps_record_flow_hash(e->rps_rxhash);
>   	} else {
>   		spin_lock_bh(&tun->lock);

Re: [PATCH][net-next] tun: align write-heavy flow entry members to a cache line

[David Miller]

From: Li RongQing <lirongqing@baidu.com>
Date: Thu,  6 Dec 2018 16:08:17 +0800

> tun flow entry 'updated' fields are written when receive
> every packet. Thus if a flow is receiving packets from a
> particular flow entry, it'll cause false-sharing with
> all the other who has looked it up, so move it in its own
> cache line
> 
> and update 'queue_index' and 'update' field only when
> they are changed to reduce the cache false-sharing.
> 
> Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
> Signed-off-by: Wang Li <wangli39@baidu.com>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>

Applied.