* [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking
@ 2017-01-09 13:20 Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 1/6] xfrm: remove unused function Florian Westphal
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev
xfrm_state_get_afinfo still uses a conditional locking scheme
dating back to when this still used an rwlock:
If return value is NULL, no lock (rcu readlock) was taken,
otherwise, rcu_read_unlock has to be called.
This series moves rcu read lock/unlock responsibility to the callers.
xfrm_state_put_afinfo is removed (it is equivalent
to rcu_read_unlock so thats what will be used instead).
xfrm_state_get_afinfo is renamed to xfrm_state_afinfo_get_rcu()
and is only a rcu dereference wrapper.
In one case, rcu read/unlock can be avoided as we're always
called with rcu read lock held.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 1/6] xfrm: remove unused function
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 2/6] xfrm: avoid rcu sparse warning Florian Westphal
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
Has been ifdef'd out for more than 10 years, remove it.
Signed-off-by: Florian Westphal <fw@strlen.de>
---
net/ipv4/xfrm4_state.c | 8 --------
1 file changed, 8 deletions(-)
diff --git a/net/ipv4/xfrm4_state.c b/net/ipv4/xfrm4_state.c
index 542074c00c78..d6660a8c0ea5 100644
--- a/net/ipv4/xfrm4_state.c
+++ b/net/ipv4/xfrm4_state.c
@@ -90,11 +90,3 @@ void __init xfrm4_state_init(void)
{
xfrm_state_register_afinfo(&xfrm4_state_afinfo);
}
-
-#if 0
-void __exit xfrm4_state_fini(void)
-{
- xfrm_state_unregister_afinfo(&xfrm4_state_afinfo);
-}
-#endif /* 0 */
-
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 2/6] xfrm: avoid rcu sparse warning
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 1/6] xfrm: remove unused function Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 3/6] xfrm: remove xfrm_state_put_afinfo Florian Westphal
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
xfrm/xfrm_state.c:1973:21: error: incompatible types in comparison expression (different address spaces)
Harmless, but lets fix it to reduce the noise.
While at it, get rid of unneeded NULL check, its never hit:
net/ipv4/xfrm4_state.c: xfrm_state_register_afinfo(&xfrm4_state_afinfo);
net/ipv6/xfrm6_state.c: return xfrm_state_register_afinfo(&xfrm6_state_afinfo);
net/ipv6/xfrm6_state.c: xfrm_state_unregister_afinfo(&xfrm6_state_afinfo);
... are the only callsites.
Signed-off-by: Florian Westphal <fw@strlen.de>
---
net/xfrm/xfrm_state.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 64e3c82eedf6..e3c15ad1b246 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -1932,10 +1932,10 @@ EXPORT_SYMBOL(xfrm_unregister_km);
int xfrm_state_register_afinfo(struct xfrm_state_afinfo *afinfo)
{
int err = 0;
- if (unlikely(afinfo == NULL))
- return -EINVAL;
- if (unlikely(afinfo->family >= NPROTO))
+
+ if (WARN_ON(afinfo->family >= NPROTO))
return -EAFNOSUPPORT;
+
spin_lock_bh(&xfrm_state_afinfo_lock);
if (unlikely(xfrm_state_afinfo[afinfo->family] != NULL))
err = -EEXIST;
@@ -1948,14 +1948,14 @@ EXPORT_SYMBOL(xfrm_state_register_afinfo);
int xfrm_state_unregister_afinfo(struct xfrm_state_afinfo *afinfo)
{
- int err = 0;
- if (unlikely(afinfo == NULL))
- return -EINVAL;
- if (unlikely(afinfo->family >= NPROTO))
+ int err = 0, family = afinfo->family;
+
+ if (WARN_ON(family >= NPROTO))
return -EAFNOSUPPORT;
+
spin_lock_bh(&xfrm_state_afinfo_lock);
if (likely(xfrm_state_afinfo[afinfo->family] != NULL)) {
- if (unlikely(xfrm_state_afinfo[afinfo->family] != afinfo))
+ if (rcu_access_pointer(xfrm_state_afinfo[family]) != afinfo)
err = -EINVAL;
else
RCU_INIT_POINTER(xfrm_state_afinfo[afinfo->family], NULL);
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 3/6] xfrm: remove xfrm_state_put_afinfo
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 1/6] xfrm: remove unused function Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 2/6] xfrm: avoid rcu sparse warning Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 4/6] xfrm: add and use xfrm_state_afinfo_get_rcu Florian Westphal
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
commit 44abdc3047aecafc141dfbaf1ed
("xfrm: replace rwlock on xfrm_state_afinfo with rcu") made
xfrm_state_put_afinfo equivalent to rcu_read_unlock.
Use spatch to replace it with direct calls to rcu_read_unlock:
@@
struct xfrm_state_afinfo *a;
@@
- xfrm_state_put_afinfo(a);
+ rcu_read_unlock();
old:
text data bss dec hex filename
22570 72 424 23066 5a1a xfrm_state.o
1612 0 0 1612 64c xfrm_output.o
new:
22554 72 424 23050 5a0a xfrm_state.o
1596 0 0 1596 63c xfrm_output.o
Signed-off-by: Florian Westphal <fw@strlen.de>
---
include/net/xfrm.h | 1 -
net/xfrm/xfrm_output.c | 8 +++-----
net/xfrm/xfrm_state.c | 31 +++++++++++++------------------
3 files changed, 16 insertions(+), 24 deletions(-)
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index 31947b9c21d6..957d0cc30691 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -343,7 +343,6 @@ struct xfrm_state_afinfo {
int xfrm_state_register_afinfo(struct xfrm_state_afinfo *afinfo);
int xfrm_state_unregister_afinfo(struct xfrm_state_afinfo *afinfo);
struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family);
-void xfrm_state_put_afinfo(struct xfrm_state_afinfo *afinfo);
struct xfrm_input_afinfo {
unsigned int family;
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index 637387bbaaea..8ba29fe58352 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -246,10 +246,8 @@ void xfrm_local_error(struct sk_buff *skb, int mtu)
return;
afinfo = xfrm_state_get_afinfo(proto);
- if (!afinfo)
- return;
-
- afinfo->local_error(skb, mtu);
- xfrm_state_put_afinfo(afinfo);
+ if (afinfo)
+ afinfo->local_error(skb, mtu);
+ rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(xfrm_local_error);
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index e3c15ad1b246..8a9ec1b80bf5 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -192,7 +192,7 @@ int xfrm_register_type(const struct xfrm_type *type, unsigned short family)
else
err = -EEXIST;
spin_unlock_bh(&xfrm_type_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_register_type);
@@ -213,7 +213,7 @@ int xfrm_unregister_type(const struct xfrm_type *type, unsigned short family)
else
typemap[type->proto] = NULL;
spin_unlock_bh(&xfrm_type_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_unregister_type);
@@ -235,13 +235,13 @@ static const struct xfrm_type *xfrm_get_type(u8 proto, unsigned short family)
if (unlikely(type && !try_module_get(type->owner)))
type = NULL;
if (!type && !modload_attempted) {
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
request_module("xfrm-type-%d-%d", family, proto);
modload_attempted = 1;
goto retry;
}
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return type;
}
@@ -280,7 +280,7 @@ int xfrm_register_mode(struct xfrm_mode *mode, int family)
out:
spin_unlock_bh(&xfrm_mode_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_register_mode);
@@ -308,7 +308,7 @@ int xfrm_unregister_mode(struct xfrm_mode *mode, int family)
}
spin_unlock_bh(&xfrm_mode_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_unregister_mode);
@@ -331,13 +331,13 @@ static struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
if (unlikely(mode && !try_module_get(mode->owner)))
mode = NULL;
if (!mode && !modload_attempted) {
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
request_module("xfrm-mode-%d-%d", family, encap);
modload_attempted = 1;
goto retry;
}
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return mode;
}
@@ -651,13 +651,13 @@ xfrm_init_tempstate(struct xfrm_state *x, const struct flowi *fl,
afinfo->init_tempsel(&x->sel, fl);
if (family != tmpl->encap_family) {
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
afinfo = xfrm_state_get_afinfo(tmpl->encap_family);
if (!afinfo)
return -1;
}
afinfo->init_temprop(x, tmpl, daddr, saddr);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return 0;
}
@@ -1474,7 +1474,7 @@ xfrm_tmpl_sort(struct xfrm_tmpl **dst, struct xfrm_tmpl **src, int n,
if (afinfo->tmpl_sort)
err = afinfo->tmpl_sort(dst, src, n);
spin_unlock_bh(&net->xfrm.xfrm_state_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_tmpl_sort);
@@ -1494,7 +1494,7 @@ xfrm_state_sort(struct xfrm_state **dst, struct xfrm_state **src, int n,
if (afinfo->state_sort)
err = afinfo->state_sort(dst, src, n);
spin_unlock_bh(&net->xfrm.xfrm_state_lock);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
return err;
}
EXPORT_SYMBOL(xfrm_state_sort);
@@ -1978,11 +1978,6 @@ struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family)
return afinfo;
}
-void xfrm_state_put_afinfo(struct xfrm_state_afinfo *afinfo)
-{
- rcu_read_unlock();
-}
-
/* Temporarily located here until net/xfrm/xfrm_tunnel.c is created */
void xfrm_state_delete_tunnel(struct xfrm_state *x)
{
@@ -2028,7 +2023,7 @@ int __xfrm_init_state(struct xfrm_state *x, bool init_replay)
if (afinfo->init_flags)
err = afinfo->init_flags(x);
- xfrm_state_put_afinfo(afinfo);
+ rcu_read_unlock();
if (err)
goto error;
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 4/6] xfrm: add and use xfrm_state_afinfo_get_rcu
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
` (2 preceding siblings ...)
2017-01-09 13:20 ` [PATCH ipsec-next 3/6] xfrm: remove xfrm_state_put_afinfo Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 5/6] xfrm: state: simplify rcu_read_unlock handling in two spots Florian Westphal
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
xfrm_init_tempstate is always called from within rcu read side section.
We can thus use a simpler function that doesn't call rcu_read_lock
again.
While at it, also make xfrm_init_tempstate return value void, the
return value was never tested.
A followup patch will replace remaining callers of xfrm_state_get_afinfo
with xfrm_state_afinfo_get_rcu variant and then remove the 'old'
get_afinfo interface.
Signed-off-by: Florian Westphal <fw@strlen.de>
---
include/net/xfrm.h | 1 +
net/xfrm/xfrm_state.c | 25 +++++++++++++++----------
2 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index 957d0cc30691..c52197cf51dc 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -343,6 +343,7 @@ struct xfrm_state_afinfo {
int xfrm_state_register_afinfo(struct xfrm_state_afinfo *afinfo);
int xfrm_state_unregister_afinfo(struct xfrm_state_afinfo *afinfo);
struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family);
+struct xfrm_state_afinfo *xfrm_state_afinfo_get_rcu(unsigned int family);
struct xfrm_input_afinfo {
unsigned int family;
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 8a9ec1b80bf5..ee1309e2b8b5 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -639,26 +639,23 @@ void xfrm_sad_getinfo(struct net *net, struct xfrmk_sadinfo *si)
}
EXPORT_SYMBOL(xfrm_sad_getinfo);
-static int
+static void
xfrm_init_tempstate(struct xfrm_state *x, const struct flowi *fl,
const struct xfrm_tmpl *tmpl,
const xfrm_address_t *daddr, const xfrm_address_t *saddr,
unsigned short family)
{
- struct xfrm_state_afinfo *afinfo = xfrm_state_get_afinfo(family);
- if (!afinfo)
- return -1;
- afinfo->init_tempsel(&x->sel, fl);
+ struct xfrm_state_afinfo *afinfo = xfrm_state_afinfo_get_rcu(family);
+
+ if (afinfo)
+ afinfo->init_tempsel(&x->sel, fl);
if (family != tmpl->encap_family) {
- rcu_read_unlock();
- afinfo = xfrm_state_get_afinfo(tmpl->encap_family);
+ afinfo = xfrm_state_afinfo_get_rcu(tmpl->encap_family);
if (!afinfo)
- return -1;
+ return;
}
afinfo->init_temprop(x, tmpl, daddr, saddr);
- rcu_read_unlock();
- return 0;
}
static struct xfrm_state *__xfrm_state_lookup(struct net *net, u32 mark,
@@ -1966,6 +1963,14 @@ int xfrm_state_unregister_afinfo(struct xfrm_state_afinfo *afinfo)
}
EXPORT_SYMBOL(xfrm_state_unregister_afinfo);
+struct xfrm_state_afinfo *xfrm_state_afinfo_get_rcu(unsigned int family)
+{
+ if (unlikely(family >= NPROTO))
+ return NULL;
+
+ return rcu_dereference(xfrm_state_afinfo[family]);
+}
+
struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family)
{
struct xfrm_state_afinfo *afinfo;
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 5/6] xfrm: state: simplify rcu_read_unlock handling in two spots
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
` (3 preceding siblings ...)
2017-01-09 13:20 ` [PATCH ipsec-next 4/6] xfrm: add and use xfrm_state_afinfo_get_rcu Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 6/6] xfrm: remove xfrm_state_get_afinfo Florian Westphal
2017-01-11 12:17 ` [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Steffen Klassert
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
Instead of:
if (foo) {
unlock();
return bar();
}
unlock();
do:
unlock();
if (foo)
return bar();
This is ok because rcu protected structure is only dereferenced before
the conditional.
Signed-off-by: Florian Westphal <fw@strlen.de>
---
net/xfrm/xfrm_state.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index ee1309e2b8b5..0245df063bd7 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -231,17 +231,18 @@ static const struct xfrm_type *xfrm_get_type(u8 proto, unsigned short family)
return NULL;
typemap = afinfo->type_map;
- type = typemap[proto];
+ type = READ_ONCE(typemap[proto]);
if (unlikely(type && !try_module_get(type->owner)))
type = NULL;
+
+ rcu_read_unlock();
+
if (!type && !modload_attempted) {
- rcu_read_unlock();
request_module("xfrm-type-%d-%d", family, proto);
modload_attempted = 1;
goto retry;
}
- rcu_read_unlock();
return type;
}
@@ -327,17 +328,17 @@ static struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
if (unlikely(afinfo == NULL))
return NULL;
- mode = afinfo->mode_map[encap];
+ mode = READ_ONCE(afinfo->mode_map[encap]);
if (unlikely(mode && !try_module_get(mode->owner)))
mode = NULL;
+
+ rcu_read_unlock();
if (!mode && !modload_attempted) {
- rcu_read_unlock();
request_module("xfrm-mode-%d-%d", family, encap);
modload_attempted = 1;
goto retry;
}
- rcu_read_unlock();
return mode;
}
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH ipsec-next 6/6] xfrm: remove xfrm_state_get_afinfo
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
` (4 preceding siblings ...)
2017-01-09 13:20 ` [PATCH ipsec-next 5/6] xfrm: state: simplify rcu_read_unlock handling in two spots Florian Westphal
@ 2017-01-09 13:20 ` Florian Westphal
2017-01-11 12:17 ` [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Steffen Klassert
6 siblings, 0 replies; 8+ messages in thread
From: Florian Westphal @ 2017-01-09 13:20 UTC (permalink / raw)
To: netdev; +Cc: Florian Westphal
Use xfrm_state_afinfo_get_rcu() everywhere, this also moves
rcu read (un)lock responsibility to the callers.
This avoids the conditional locking scheme we currently have.
Signed-off-by: Florian Westphal <fw@strlen.de>
---
include/net/xfrm.h | 1 -
net/xfrm/xfrm_output.c | 3 +-
net/xfrm/xfrm_state.c | 97 +++++++++++++++++++++++++++++++-------------------
3 files changed, 62 insertions(+), 39 deletions(-)
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index c52197cf51dc..d81e153bfeee 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -342,7 +342,6 @@ struct xfrm_state_afinfo {
int xfrm_state_register_afinfo(struct xfrm_state_afinfo *afinfo);
int xfrm_state_unregister_afinfo(struct xfrm_state_afinfo *afinfo);
-struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family);
struct xfrm_state_afinfo *xfrm_state_afinfo_get_rcu(unsigned int family);
struct xfrm_input_afinfo {
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index 8ba29fe58352..351efd207d88 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -245,7 +245,8 @@ void xfrm_local_error(struct sk_buff *skb, int mtu)
else
return;
- afinfo = xfrm_state_get_afinfo(proto);
+ rcu_read_lock();
+ afinfo = xfrm_state_afinfo_get_rcu(proto);
if (afinfo)
afinfo->local_error(skb, mtu);
rcu_read_unlock();
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 0245df063bd7..1a7f0c0ec3bd 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -178,12 +178,17 @@ void km_state_expired(struct xfrm_state *x, int hard, u32 portid);
static DEFINE_SPINLOCK(xfrm_type_lock);
int xfrm_register_type(const struct xfrm_type *type, unsigned short family)
{
- struct xfrm_state_afinfo *afinfo = xfrm_state_get_afinfo(family);
+ struct xfrm_state_afinfo *afinfo;
const struct xfrm_type **typemap;
int err = 0;
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (!afinfo) {
+ rcu_read_unlock();
return -EAFNOSUPPORT;
+ }
typemap = afinfo->type_map;
spin_lock_bh(&xfrm_type_lock);
@@ -199,12 +204,17 @@ EXPORT_SYMBOL(xfrm_register_type);
int xfrm_unregister_type(const struct xfrm_type *type, unsigned short family)
{
- struct xfrm_state_afinfo *afinfo = xfrm_state_get_afinfo(family);
+ struct xfrm_state_afinfo *afinfo;
const struct xfrm_type **typemap;
int err = 0;
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (!afinfo) {
+ rcu_read_unlock();
return -EAFNOSUPPORT;
+ }
typemap = afinfo->type_map;
spin_lock_bh(&xfrm_type_lock);
@@ -226,9 +236,13 @@ static const struct xfrm_type *xfrm_get_type(u8 proto, unsigned short family)
int modload_attempted = 0;
retry:
- afinfo = xfrm_state_get_afinfo(family);
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (unlikely(!afinfo)) {
+ rcu_read_unlock();
return NULL;
+ }
+
typemap = afinfo->type_map;
type = READ_ONCE(typemap[proto]);
@@ -261,9 +275,12 @@ int xfrm_register_mode(struct xfrm_mode *mode, int family)
if (unlikely(mode->encap >= XFRM_MODE_MAX))
return -EINVAL;
- afinfo = xfrm_state_get_afinfo(family);
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (!afinfo) {
+ rcu_read_unlock();
return -EAFNOSUPPORT;
+ }
err = -EEXIST;
modemap = afinfo->mode_map;
@@ -295,9 +312,12 @@ int xfrm_unregister_mode(struct xfrm_mode *mode, int family)
if (unlikely(mode->encap >= XFRM_MODE_MAX))
return -EINVAL;
- afinfo = xfrm_state_get_afinfo(family);
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (!afinfo) {
+ rcu_read_unlock();
return -EAFNOSUPPORT;
+ }
err = -ENOENT;
modemap = afinfo->mode_map;
@@ -322,17 +342,21 @@ static struct xfrm_mode *xfrm_get_mode(unsigned int encap, int family)
if (unlikely(encap >= XFRM_MODE_MAX))
return NULL;
-
retry:
- afinfo = xfrm_state_get_afinfo(family);
- if (unlikely(afinfo == NULL))
+ rcu_read_lock();
+
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (unlikely(!afinfo)) {
+ rcu_read_unlock();
return NULL;
+ }
mode = READ_ONCE(afinfo->mode_map[encap]);
if (unlikely(mode && !try_module_get(mode->owner)))
mode = NULL;
rcu_read_unlock();
+
if (!mode && !modload_attempted) {
request_module("xfrm-mode-%d-%d", family, encap);
modload_attempted = 1;
@@ -1463,15 +1487,21 @@ int
xfrm_tmpl_sort(struct xfrm_tmpl **dst, struct xfrm_tmpl **src, int n,
unsigned short family, struct net *net)
{
- int err = 0;
- struct xfrm_state_afinfo *afinfo = xfrm_state_get_afinfo(family);
+ struct xfrm_state_afinfo *afinfo;
+ int err = -EAFNOSUPPORT;
+
+ rcu_read_lock();
+
+ afinfo = xfrm_state_afinfo_get_rcu(family);
if (!afinfo)
- return -EAFNOSUPPORT;
+ goto error;
+ err = 0;
spin_lock_bh(&net->xfrm.xfrm_state_lock); /*FIXME*/
if (afinfo->tmpl_sort)
err = afinfo->tmpl_sort(dst, src, n);
spin_unlock_bh(&net->xfrm.xfrm_state_lock);
+error:
rcu_read_unlock();
return err;
}
@@ -1481,17 +1511,20 @@ int
xfrm_state_sort(struct xfrm_state **dst, struct xfrm_state **src, int n,
unsigned short family)
{
- int err = 0;
- struct xfrm_state_afinfo *afinfo = xfrm_state_get_afinfo(family);
+ struct xfrm_state_afinfo *afinfo;
struct net *net = xs_net(*src);
+ int err = -EAFNOSUPPORT;
+ afinfo = xfrm_state_afinfo_get_rcu(family);
if (!afinfo)
- return -EAFNOSUPPORT;
+ goto error;
+ err = 0;
spin_lock_bh(&net->xfrm.xfrm_state_lock);
if (afinfo->state_sort)
err = afinfo->state_sort(dst, src, n);
spin_unlock_bh(&net->xfrm.xfrm_state_lock);
+error:
rcu_read_unlock();
return err;
}
@@ -1972,18 +2005,6 @@ struct xfrm_state_afinfo *xfrm_state_afinfo_get_rcu(unsigned int family)
return rcu_dereference(xfrm_state_afinfo[family]);
}
-struct xfrm_state_afinfo *xfrm_state_get_afinfo(unsigned int family)
-{
- struct xfrm_state_afinfo *afinfo;
- if (unlikely(family >= NPROTO))
- return NULL;
- rcu_read_lock();
- afinfo = rcu_dereference(xfrm_state_afinfo[family]);
- if (unlikely(!afinfo))
- rcu_read_unlock();
- return afinfo;
-}
-
/* Temporarily located here until net/xfrm/xfrm_tunnel.c is created */
void xfrm_state_delete_tunnel(struct xfrm_state *x)
{
@@ -2018,14 +2039,16 @@ int __xfrm_init_state(struct xfrm_state *x, bool init_replay)
struct xfrm_state_afinfo *afinfo;
struct xfrm_mode *inner_mode;
int family = x->props.family;
- int err;
+ int err = 0;
- err = -EAFNOSUPPORT;
- afinfo = xfrm_state_get_afinfo(family);
- if (!afinfo)
- goto error;
+ rcu_read_lock();
+
+ afinfo = xfrm_state_afinfo_get_rcu(family);
+ if (!afinfo) {
+ rcu_read_unlock();
+ return -EAFNOSUPPORT;
+ }
- err = 0;
if (afinfo->init_flags)
err = afinfo->init_flags(x);
--
2.7.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
` (5 preceding siblings ...)
2017-01-09 13:20 ` [PATCH ipsec-next 6/6] xfrm: remove xfrm_state_get_afinfo Florian Westphal
@ 2017-01-11 12:17 ` Steffen Klassert
6 siblings, 0 replies; 8+ messages in thread
From: Steffen Klassert @ 2017-01-11 12:17 UTC (permalink / raw)
To: Florian Westphal; +Cc: netdev
On Mon, Jan 09, 2017 at 02:20:44PM +0100, Florian Westphal wrote:
> xfrm_state_get_afinfo still uses a conditional locking scheme
> dating back to when this still used an rwlock:
>
> If return value is NULL, no lock (rcu readlock) was taken,
> otherwise, rcu_read_unlock has to be called.
>
> This series moves rcu read lock/unlock responsibility to the callers.
>
> xfrm_state_put_afinfo is removed (it is equivalent
> to rcu_read_unlock so thats what will be used instead).
>
> xfrm_state_get_afinfo is renamed to xfrm_state_afinfo_get_rcu()
> and is only a rcu dereference wrapper.
>
> In one case, rcu read/unlock can be avoided as we're always
> called with rcu read lock held.
All applied to ipsec-next, thanks a lot!
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-01-11 12:17 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-09 13:20 [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 1/6] xfrm: remove unused function Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 2/6] xfrm: avoid rcu sparse warning Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 3/6] xfrm: remove xfrm_state_put_afinfo Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 4/6] xfrm: add and use xfrm_state_afinfo_get_rcu Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 5/6] xfrm: state: simplify rcu_read_unlock handling in two spots Florian Westphal
2017-01-09 13:20 ` [PATCH ipsec-next 6/6] xfrm: remove xfrm_state_get_afinfo Florian Westphal
2017-01-11 12:17 ` [PATCH ipsec-next 0/6] xfrm: remove xfrm_state_get_afinfo conditional rcu locking Steffen Klassert
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).