* [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow
@ 2024-04-18 7:36 Jason Xing
2024-04-18 7:36 ` [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper Jason Xing
` (3 more replies)
0 siblings, 4 replies; 8+ messages in thread
From: Jason Xing @ 2024-04-18 7:36 UTC (permalink / raw)
To: edumazet, kuba, pabeni, davem, horms; +Cc: netdev, kerneljasonxing, Jason Xing
From: Jason Xing <kernelxing@tencent.com>
Since Eric did a more complicated locklessly change to last_qtail
member[1] in struct rps_dev_flow, the left members are easier to change
as the same.
One thing important I would like to share by qooting Eric:
"rflow is located in rxqueue->rps_flow_table, it is thus private to current
thread. Only one cpu can service an RX queue at a time."
So we only pay attention to the reader in the rps_may_expire_flow() and
writer in the set_rps_cpu(). They are in the two different contexts.
[1]:
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=3b4cf29bdab
v3
Link: https://lore.kernel.org/all/20240417062721.45652-1-kerneljasonxing@gmail.com/
1. adjust the protection in a right way (Eric)
v2
1. fix passing wrong type qtail.
Jason Xing (3):
net: rps: protect last_qtail with rps_input_queue_tail_save() helper
net: rps: protect filter locklessly
net: rps: locklessly access rflow->cpu
net/core/dev.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
--
2.37.3
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper
2024-04-18 7:36 [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow Jason Xing
@ 2024-04-18 7:36 ` Jason Xing
2024-04-19 5:57 ` Eric Dumazet
2024-04-18 7:36 ` [PATCH net-next v3 2/3] net: rps: protect filter locklessly Jason Xing
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Jason Xing @ 2024-04-18 7:36 UTC (permalink / raw)
To: edumazet, kuba, pabeni, davem, horms; +Cc: netdev, kerneljasonxing, Jason Xing
From: Jason Xing <kernelxing@tencent.com>
Removing one unnecessary reader protection and add another writer
protection to finish the locklessly proctection job.
Note: the removed READ_ONCE() is not needed because we only have to protect
the locklessly reader in the different context (rps_may_expire_flow()).
Signed-off-by: Jason Xing <kernelxing@tencent.com>
---
net/core/dev.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 854a3a28a8d8..58e0da91bfef 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4501,7 +4501,7 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
struct netdev_rx_queue *rxqueue;
struct rps_dev_flow_table *flow_table;
struct rps_dev_flow *old_rflow;
- u32 flow_id;
+ u32 flow_id, head;
u16 rxq_index;
int rc;
@@ -4529,8 +4529,8 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
old_rflow->filter = RPS_NO_FILTER;
out:
#endif
- rflow->last_qtail =
- READ_ONCE(per_cpu(softnet_data, next_cpu).input_queue_head);
+ head = READ_ONCE(per_cpu(softnet_data, next_cpu).input_queue_head);
+ rps_input_queue_tail_save(&rflow->last_qtail, head);
}
rflow->cpu = next_cpu;
@@ -4613,7 +4613,7 @@ static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb,
if (unlikely(tcpu != next_cpu) &&
(tcpu >= nr_cpu_ids || !cpu_online(tcpu) ||
((int)(READ_ONCE(per_cpu(softnet_data, tcpu).input_queue_head) -
- READ_ONCE(rflow->last_qtail))) >= 0)) {
+ rflow->last_qtail)) >= 0)) {
tcpu = next_cpu;
rflow = set_rps_cpu(dev, skb, rflow, next_cpu);
}
--
2.37.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next v3 2/3] net: rps: protect filter locklessly
2024-04-18 7:36 [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow Jason Xing
2024-04-18 7:36 ` [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper Jason Xing
@ 2024-04-18 7:36 ` Jason Xing
2024-04-19 5:58 ` Eric Dumazet
2024-04-18 7:36 ` [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu Jason Xing
2024-04-19 10:40 ` [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow patchwork-bot+netdevbpf
3 siblings, 1 reply; 8+ messages in thread
From: Jason Xing @ 2024-04-18 7:36 UTC (permalink / raw)
To: edumazet, kuba, pabeni, davem, horms; +Cc: netdev, kerneljasonxing, Jason Xing
From: Jason Xing <kernelxing@tencent.com>
As we can see, rflow->filter can be written/read concurrently, so
lockless access is needed.
Signed-off-by: Jason Xing <kernelxing@tencent.com>
---
net/core/dev.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 58e0da91bfef..ed6efef01582 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4524,9 +4524,9 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
goto out;
old_rflow = rflow;
rflow = &flow_table->flows[flow_id];
- rflow->filter = rc;
- if (old_rflow->filter == rflow->filter)
- old_rflow->filter = RPS_NO_FILTER;
+ WRITE_ONCE(rflow->filter, rc);
+ if (old_rflow->filter == rc)
+ WRITE_ONCE(old_rflow->filter, RPS_NO_FILTER);
out:
#endif
head = READ_ONCE(per_cpu(softnet_data, next_cpu).input_queue_head);
@@ -4666,7 +4666,7 @@ bool rps_may_expire_flow(struct net_device *dev, u16 rxq_index,
if (flow_table && flow_id <= flow_table->mask) {
rflow = &flow_table->flows[flow_id];
cpu = READ_ONCE(rflow->cpu);
- if (rflow->filter == filter_id && cpu < nr_cpu_ids &&
+ if (READ_ONCE(rflow->filter) == filter_id && cpu < nr_cpu_ids &&
((int)(READ_ONCE(per_cpu(softnet_data, cpu).input_queue_head) -
READ_ONCE(rflow->last_qtail)) <
(int)(10 * flow_table->mask)))
--
2.37.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu
2024-04-18 7:36 [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow Jason Xing
2024-04-18 7:36 ` [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper Jason Xing
2024-04-18 7:36 ` [PATCH net-next v3 2/3] net: rps: protect filter locklessly Jason Xing
@ 2024-04-18 7:36 ` Jason Xing
2024-04-19 5:58 ` Eric Dumazet
2024-04-19 10:40 ` [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow patchwork-bot+netdevbpf
3 siblings, 1 reply; 8+ messages in thread
From: Jason Xing @ 2024-04-18 7:36 UTC (permalink / raw)
To: edumazet, kuba, pabeni, davem, horms; +Cc: netdev, kerneljasonxing, Jason Xing
From: Jason Xing <kernelxing@tencent.com>
This is the last member in struct rps_dev_flow which should be
protected locklessly. So finish it.
Signed-off-by: Jason Xing <kernelxing@tencent.com>
---
net/core/dev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index ed6efef01582..8010036c07b6 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4533,7 +4533,7 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
rps_input_queue_tail_save(&rflow->last_qtail, head);
}
- rflow->cpu = next_cpu;
+ WRITE_ONCE(rflow->cpu, next_cpu);
return rflow;
}
--
2.37.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper
2024-04-18 7:36 ` [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper Jason Xing
@ 2024-04-19 5:57 ` Eric Dumazet
0 siblings, 0 replies; 8+ messages in thread
From: Eric Dumazet @ 2024-04-19 5:57 UTC (permalink / raw)
To: Jason Xing; +Cc: kuba, pabeni, davem, horms, netdev, Jason Xing
On Thu, Apr 18, 2024 at 9:36 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> From: Jason Xing <kernelxing@tencent.com>
>
> Removing one unnecessary reader protection and add another writer
> protection to finish the locklessly proctection job.
>
> Note: the removed READ_ONCE() is not needed because we only have to protect
> the locklessly reader in the different context (rps_may_expire_flow()).
>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH net-next v3 2/3] net: rps: protect filter locklessly
2024-04-18 7:36 ` [PATCH net-next v3 2/3] net: rps: protect filter locklessly Jason Xing
@ 2024-04-19 5:58 ` Eric Dumazet
0 siblings, 0 replies; 8+ messages in thread
From: Eric Dumazet @ 2024-04-19 5:58 UTC (permalink / raw)
To: Jason Xing; +Cc: kuba, pabeni, davem, horms, netdev, Jason Xing
On Thu, Apr 18, 2024 at 9:36 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> From: Jason Xing <kernelxing@tencent.com>
>
> As we can see, rflow->filter can be written/read concurrently, so
> lockless access is needed.
>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu
2024-04-18 7:36 ` [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu Jason Xing
@ 2024-04-19 5:58 ` Eric Dumazet
0 siblings, 0 replies; 8+ messages in thread
From: Eric Dumazet @ 2024-04-19 5:58 UTC (permalink / raw)
To: Jason Xing; +Cc: kuba, pabeni, davem, horms, netdev, Jason Xing
On Thu, Apr 18, 2024 at 9:36 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> From: Jason Xing <kernelxing@tencent.com>
>
> This is the last member in struct rps_dev_flow which should be
> protected locklessly. So finish it.
>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow
2024-04-18 7:36 [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow Jason Xing
` (2 preceding siblings ...)
2024-04-18 7:36 ` [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu Jason Xing
@ 2024-04-19 10:40 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 8+ messages in thread
From: patchwork-bot+netdevbpf @ 2024-04-19 10:40 UTC (permalink / raw)
To: Jason Xing; +Cc: edumazet, kuba, pabeni, davem, horms, netdev, kernelxing
Hello:
This series was applied to netdev/net-next.git (main)
by David S. Miller <davem@davemloft.net>:
On Thu, 18 Apr 2024 15:36:00 +0800 you wrote:
> From: Jason Xing <kernelxing@tencent.com>
>
> Since Eric did a more complicated locklessly change to last_qtail
> member[1] in struct rps_dev_flow, the left members are easier to change
> as the same.
>
> One thing important I would like to share by qooting Eric:
> "rflow is located in rxqueue->rps_flow_table, it is thus private to current
> thread. Only one cpu can service an RX queue at a time."
> So we only pay attention to the reader in the rps_may_expire_flow() and
> writer in the set_rps_cpu(). They are in the two different contexts.
>
> [...]
Here is the summary with links:
- [net-next,v3,1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper
https://git.kernel.org/netdev/net-next/c/84b6823cd96b
- [net-next,v3,2/3] net: rps: protect filter locklessly
https://git.kernel.org/netdev/net-next/c/f00bf5dc8320
- [net-next,v3,3/3] net: rps: locklessly access rflow->cpu
https://git.kernel.org/netdev/net-next/c/f7b60cce8470
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-04-19 10:40 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-18 7:36 [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow Jason Xing
2024-04-18 7:36 ` [PATCH net-next v3 1/3] net: rps: protect last_qtail with rps_input_queue_tail_save() helper Jason Xing
2024-04-19 5:57 ` Eric Dumazet
2024-04-18 7:36 ` [PATCH net-next v3 2/3] net: rps: protect filter locklessly Jason Xing
2024-04-19 5:58 ` Eric Dumazet
2024-04-18 7:36 ` [PATCH net-next v3 3/3] net: rps: locklessly access rflow->cpu Jason Xing
2024-04-19 5:58 ` Eric Dumazet
2024-04-19 10:40 ` [PATCH net-next v3 0/3] locklessly protect left members in struct rps_dev_flow patchwork-bot+netdevbpf
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.