* [PATCH net-next 0/6] net: more try_cmpxchg() conversions
@ 2022-11-15 9:10 Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 1/6] net: mm_account_pinned_pages() optimization Eric Dumazet
` (6 more replies)
0 siblings, 7 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:10 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet
Adopt try_cmpxchg() and friends in more places, as this
is preferred nowadays.
Eric Dumazet (6):
net: mm_account_pinned_pages() optimization
ipv6: fib6_new_sernum() optimization
net: net_{enable|disable}_timestamp() optimizations
net: adopt try_cmpxchg() in napi_schedule_prep() and
napi_complete_done()
net: adopt try_cmpxchg() in napi_{enable|disable}()
net: __sock_gen_cookie() cleanup
net/core/dev.c | 42 +++++++++++++++---------------------------
net/core/skbuff.c | 5 ++---
net/core/sock_diag.c | 12 ++++++------
net/ipv6/ip6_fib.c | 7 +++----
4 files changed, 26 insertions(+), 40 deletions(-)
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net-next 1/6] net: mm_account_pinned_pages() optimization
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
@ 2022-11-15 9:10 ` Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization Eric Dumazet
` (5 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:10 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet, Willem de Bruijn
Adopt atomic_long_try_cmpxchg() in mm_account_pinned_pages()
as it is slightly more efficient.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Willem de Bruijn <willemb@google.com>
---
net/core/skbuff.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 90d085290d49d864b8431f99a19dbda867d9c03b..4bf95e36ed162611e8245c64c6caeecf9f60ef5d 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -1267,13 +1267,12 @@ int mm_account_pinned_pages(struct mmpin *mmp, size_t size)
max_pg = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
user = mmp->user ? : current_user();
+ old_pg = atomic_long_read(&user->locked_vm);
do {
- old_pg = atomic_long_read(&user->locked_vm);
new_pg = old_pg + num_pg;
if (new_pg > max_pg)
return -ENOBUFS;
- } while (atomic_long_cmpxchg(&user->locked_vm, old_pg, new_pg) !=
- old_pg);
+ } while (!atomic_long_try_cmpxchg(&user->locked_vm, &old_pg, new_pg));
if (!mmp->user) {
mmp->user = get_uid(user);
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 1/6] net: mm_account_pinned_pages() optimization Eric Dumazet
@ 2022-11-15 9:10 ` Eric Dumazet
2022-11-17 0:05 ` David Ahern
2022-11-15 9:10 ` [PATCH net-next 3/6] net: net_{enable|disable}_timestamp() optimizations Eric Dumazet
` (4 subsequent siblings)
6 siblings, 1 reply; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:10 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet, David Ahern
Adopt atomic_try_cmpxchg() which is slightly more efficient.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: David Ahern <dsahern@kernel.org>
---
net/ipv6/ip6_fib.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 413f66781e50de62d6b20042d84798e7da59165a..2438da5ff6da810d9f612fc66df4d28510f50f10 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -91,13 +91,12 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
static int fib6_new_sernum(struct net *net)
{
- int new, old;
+ int new, old = atomic_read(&net->ipv6.fib6_sernum);
do {
- old = atomic_read(&net->ipv6.fib6_sernum);
new = old < INT_MAX ? old + 1 : 1;
- } while (atomic_cmpxchg(&net->ipv6.fib6_sernum,
- old, new) != old);
+ } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
+
return new;
}
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next 3/6] net: net_{enable|disable}_timestamp() optimizations
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 1/6] net: mm_account_pinned_pages() optimization Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization Eric Dumazet
@ 2022-11-15 9:10 ` Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done() Eric Dumazet
` (3 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:10 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet
Adopting atomic_try_cmpxchg() makes the code cleaner.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/core/dev.c | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 117e830cabb0787ecd3da13bd88f8818fddaddc1..10b56648a9d4a0a709f8e23bb3e114854a4a1b69 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2073,13 +2073,10 @@ static DECLARE_WORK(netstamp_work, netstamp_clear);
void net_enable_timestamp(void)
{
#ifdef CONFIG_JUMP_LABEL
- int wanted;
+ int wanted = atomic_read(&netstamp_wanted);
- while (1) {
- wanted = atomic_read(&netstamp_wanted);
- if (wanted <= 0)
- break;
- if (atomic_cmpxchg(&netstamp_wanted, wanted, wanted + 1) == wanted)
+ while (wanted > 0) {
+ if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted + 1))
return;
}
atomic_inc(&netstamp_needed_deferred);
@@ -2093,13 +2090,10 @@ EXPORT_SYMBOL(net_enable_timestamp);
void net_disable_timestamp(void)
{
#ifdef CONFIG_JUMP_LABEL
- int wanted;
+ int wanted = atomic_read(&netstamp_wanted);
- while (1) {
- wanted = atomic_read(&netstamp_wanted);
- if (wanted <= 1)
- break;
- if (atomic_cmpxchg(&netstamp_wanted, wanted, wanted - 1) == wanted)
+ while (wanted > 1) {
+ if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted - 1))
return;
}
atomic_dec(&netstamp_needed_deferred);
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next 4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done()
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
` (2 preceding siblings ...)
2022-11-15 9:10 ` [PATCH net-next 3/6] net: net_{enable|disable}_timestamp() optimizations Eric Dumazet
@ 2022-11-15 9:10 ` Eric Dumazet
2022-11-15 9:11 ` [PATCH net-next 5/6] net: adopt try_cmpxchg() in napi_{enable|disable}() Eric Dumazet
` (2 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:10 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet
This makes the code slightly more efficient.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/core/dev.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 10b56648a9d4a0a709f8e23bb3e114854a4a1b69..0c12c7ad04f21da05fef4c60ca5570bff48ec491 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5979,10 +5979,9 @@ EXPORT_SYMBOL(__napi_schedule);
*/
bool napi_schedule_prep(struct napi_struct *n)
{
- unsigned long val, new;
+ unsigned long new, val = READ_ONCE(n->state);
do {
- val = READ_ONCE(n->state);
if (unlikely(val & NAPIF_STATE_DISABLE))
return false;
new = val | NAPIF_STATE_SCHED;
@@ -5995,7 +5994,7 @@ bool napi_schedule_prep(struct napi_struct *n)
*/
new |= (val & NAPIF_STATE_SCHED) / NAPIF_STATE_SCHED *
NAPIF_STATE_MISSED;
- } while (cmpxchg(&n->state, val, new) != val);
+ } while (!try_cmpxchg(&n->state, &val, new));
return !(val & NAPIF_STATE_SCHED);
}
@@ -6063,9 +6062,8 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
local_irq_restore(flags);
}
+ val = READ_ONCE(n->state);
do {
- val = READ_ONCE(n->state);
-
WARN_ON_ONCE(!(val & NAPIF_STATE_SCHED));
new = val & ~(NAPIF_STATE_MISSED | NAPIF_STATE_SCHED |
@@ -6078,7 +6076,7 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
*/
new |= (val & NAPIF_STATE_MISSED) / NAPIF_STATE_MISSED *
NAPIF_STATE_SCHED;
- } while (cmpxchg(&n->state, val, new) != val);
+ } while (!try_cmpxchg(&n->state, &val, new));
if (unlikely(val & NAPIF_STATE_MISSED)) {
__napi_schedule(n);
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next 5/6] net: adopt try_cmpxchg() in napi_{enable|disable}()
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
` (3 preceding siblings ...)
2022-11-15 9:10 ` [PATCH net-next 4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done() Eric Dumazet
@ 2022-11-15 9:11 ` Eric Dumazet
2022-11-15 9:11 ` [PATCH net-next 6/6] net: __sock_gen_cookie() cleanup Eric Dumazet
2022-11-16 12:50 ` [PATCH net-next 0/6] net: more try_cmpxchg() conversions patchwork-bot+netdevbpf
6 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:11 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet
This makes code a bit cleaner.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/core/dev.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 0c12c7ad04f21da05fef4c60ca5570bff48ec491..fb943dad96513e0f9eb1c3ce30c5bb7170edea5e 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6397,8 +6397,8 @@ void napi_disable(struct napi_struct *n)
might_sleep();
set_bit(NAPI_STATE_DISABLE, &n->state);
- for ( ; ; ) {
- val = READ_ONCE(n->state);
+ val = READ_ONCE(n->state);
+ do {
if (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) {
usleep_range(20, 200);
continue;
@@ -6406,10 +6406,7 @@ void napi_disable(struct napi_struct *n)
new = val | NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC;
new &= ~(NAPIF_STATE_THREADED | NAPIF_STATE_PREFER_BUSY_POLL);
-
- if (cmpxchg(&n->state, val, new) == val)
- break;
- }
+ } while (!try_cmpxchg(&n->state, &val, new));
hrtimer_cancel(&n->timer);
@@ -6426,16 +6423,15 @@ EXPORT_SYMBOL(napi_disable);
*/
void napi_enable(struct napi_struct *n)
{
- unsigned long val, new;
+ unsigned long new, val = READ_ONCE(n->state);
do {
- val = READ_ONCE(n->state);
BUG_ON(!test_bit(NAPI_STATE_SCHED, &val));
new = val & ~(NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC);
if (n->dev->threaded && n->thread)
new |= NAPIF_STATE_THREADED;
- } while (cmpxchg(&n->state, val, new) != val);
+ } while (!try_cmpxchg(&n->state, &val, new));
}
EXPORT_SYMBOL(napi_enable);
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next 6/6] net: __sock_gen_cookie() cleanup
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
` (4 preceding siblings ...)
2022-11-15 9:11 ` [PATCH net-next 5/6] net: adopt try_cmpxchg() in napi_{enable|disable}() Eric Dumazet
@ 2022-11-15 9:11 ` Eric Dumazet
2022-11-16 12:50 ` [PATCH net-next 0/6] net: more try_cmpxchg() conversions patchwork-bot+netdevbpf
6 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2022-11-15 9:11 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet, Eric Dumazet
Adopt atomic64_try_cmpxchg() and remove the loop,
to make the intent more obvious.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/core/sock_diag.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/net/core/sock_diag.c b/net/core/sock_diag.c
index f7cf74cdd3db1f7f0783db88fc28fd876fbac4b6..b11593cae5a09b15a10d6ba35bccc22263cb8fc8 100644
--- a/net/core/sock_diag.c
+++ b/net/core/sock_diag.c
@@ -25,14 +25,14 @@ DEFINE_COOKIE(sock_cookie);
u64 __sock_gen_cookie(struct sock *sk)
{
- while (1) {
- u64 res = atomic64_read(&sk->sk_cookie);
+ u64 res = atomic64_read(&sk->sk_cookie);
- if (res)
- return res;
- res = gen_cookie_next(&sock_cookie);
- atomic64_cmpxchg(&sk->sk_cookie, 0, res);
+ if (!res) {
+ u64 new = gen_cookie_next(&sock_cookie);
+
+ atomic64_try_cmpxchg(&sk->sk_cookie, &res, new);
}
+ return res;
}
int sock_diag_check_cookie(struct sock *sk, const __u32 *cookie)
--
2.38.1.431.g37b22c650d-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH net-next 0/6] net: more try_cmpxchg() conversions
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
` (5 preceding siblings ...)
2022-11-15 9:11 ` [PATCH net-next 6/6] net: __sock_gen_cookie() cleanup Eric Dumazet
@ 2022-11-16 12:50 ` patchwork-bot+netdevbpf
6 siblings, 0 replies; 9+ messages in thread
From: patchwork-bot+netdevbpf @ 2022-11-16 12:50 UTC (permalink / raw)
To: Eric Dumazet; +Cc: davem, kuba, pabeni, netdev, eric.dumazet
Hello:
This series was applied to netdev/net-next.git (master)
by David S. Miller <davem@davemloft.net>:
On Tue, 15 Nov 2022 09:10:55 +0000 you wrote:
> Adopt try_cmpxchg() and friends in more places, as this
> is preferred nowadays.
>
> Eric Dumazet (6):
> net: mm_account_pinned_pages() optimization
> ipv6: fib6_new_sernum() optimization
> net: net_{enable|disable}_timestamp() optimizations
> net: adopt try_cmpxchg() in napi_schedule_prep() and
> napi_complete_done()
> net: adopt try_cmpxchg() in napi_{enable|disable}()
> net: __sock_gen_cookie() cleanup
>
> [...]
Here is the summary with links:
- [net-next,1/6] net: mm_account_pinned_pages() optimization
https://git.kernel.org/netdev/net-next/c/57fc05e8e82d
- [net-next,2/6] ipv6: fib6_new_sernum() optimization
https://git.kernel.org/netdev/net-next/c/30189806fbb9
- [net-next,3/6] net: net_{enable|disable}_timestamp() optimizations
https://git.kernel.org/netdev/net-next/c/6af645a5b2da
- [net-next,4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done()
https://git.kernel.org/netdev/net-next/c/1462160c7455
- [net-next,5/6] net: adopt try_cmpxchg() in napi_{enable|disable}()
https://git.kernel.org/netdev/net-next/c/4ffa1d1c6842
- [net-next,6/6] net: __sock_gen_cookie() cleanup
https://git.kernel.org/netdev/net-next/c/4ebf802cf1c6
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization
2022-11-15 9:10 ` [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization Eric Dumazet
@ 2022-11-17 0:05 ` David Ahern
0 siblings, 0 replies; 9+ messages in thread
From: David Ahern @ 2022-11-17 0:05 UTC (permalink / raw)
To: Eric Dumazet, David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, eric.dumazet
On 11/15/22 2:10 AM, Eric Dumazet wrote:
> Adopt atomic_try_cmpxchg() which is slightly more efficient.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> Cc: David Ahern <dsahern@kernel.org>
> ---
> net/ipv6/ip6_fib.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
> index 413f66781e50de62d6b20042d84798e7da59165a..2438da5ff6da810d9f612fc66df4d28510f50f10 100644
> --- a/net/ipv6/ip6_fib.c
> +++ b/net/ipv6/ip6_fib.c
> @@ -91,13 +91,12 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
>
> static int fib6_new_sernum(struct net *net)
> {
> - int new, old;
> + int new, old = atomic_read(&net->ipv6.fib6_sernum);
>
> do {
> - old = atomic_read(&net->ipv6.fib6_sernum);
> new = old < INT_MAX ? old + 1 : 1;
> - } while (atomic_cmpxchg(&net->ipv6.fib6_sernum,
> - old, new) != old);
> + } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
> +
> return new;
> }
>
Reviewed-by: David Ahern <dsahern@kernel.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-11-17 0:05 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-15 9:10 [PATCH net-next 0/6] net: more try_cmpxchg() conversions Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 1/6] net: mm_account_pinned_pages() optimization Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization Eric Dumazet
2022-11-17 0:05 ` David Ahern
2022-11-15 9:10 ` [PATCH net-next 3/6] net: net_{enable|disable}_timestamp() optimizations Eric Dumazet
2022-11-15 9:10 ` [PATCH net-next 4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done() Eric Dumazet
2022-11-15 9:11 ` [PATCH net-next 5/6] net: adopt try_cmpxchg() in napi_{enable|disable}() Eric Dumazet
2022-11-15 9:11 ` [PATCH net-next 6/6] net: __sock_gen_cookie() cleanup Eric Dumazet
2022-11-16 12:50 ` [PATCH net-next 0/6] net: more try_cmpxchg() conversions patchwork-bot+netdevbpf
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.