Re: [PATCH v2] netlabel: fix RCU annotation for IPv4 options on socket creation

[Paul Moore <paul@paul-moore.com>]

On Apr 24, 2024 Davide Caratti <dcaratti@redhat.com> wrote:
> 
> Xiumei reports the following splat when netlabel and TCP socket are used:
> 
>  =============================
>  WARNING: suspicious RCU usage
>  6.9.0-rc2+ #637 Not tainted
>  -----------------------------
>  net/ipv4/cipso_ipv4.c:1880 suspicious rcu_dereference_protected() usage!
> 
>  other info that might help us debug this:
> 
>  rcu_scheduler_active = 2, debug_locks = 1
>  1 lock held by ncat/23333:
>   #0: ffffffff906030c0 (rcu_read_lock){....}-{1:2}, at: netlbl_sock_setattr+0x25/0x1b0
> 
>  stack backtrace:
>  CPU: 11 PID: 23333 Comm: ncat Kdump: loaded Not tainted 6.9.0-rc2+ #637
>  Hardware name: Supermicro SYS-6027R-72RF/X9DRH-7TF/7F/iTF/iF, BIOS 3.0  07/26/2013
>  Call Trace:
>   <TASK>
>   dump_stack_lvl+0xa9/0xc0
>   lockdep_rcu_suspicious+0x117/0x190
>   cipso_v4_sock_setattr+0x1ab/0x1b0
>   netlbl_sock_setattr+0x13e/0x1b0
>   selinux_netlbl_socket_post_create+0x3f/0x80
>   selinux_socket_post_create+0x1a0/0x460
>   security_socket_post_create+0x42/0x60
>   __sock_create+0x342/0x3a0
>   __sys_socket_create.part.22+0x42/0x70
>   __sys_socket+0x37/0xb0
>   __x64_sys_socket+0x16/0x20
>   do_syscall_64+0x96/0x180
>   ? do_user_addr_fault+0x68d/0xa30
>   ? exc_page_fault+0x171/0x280
>   ? asm_exc_page_fault+0x22/0x30
>   entry_SYSCALL_64_after_hwframe+0x71/0x79
>  RIP: 0033:0x7fbc0ca3fc1b
>  Code: 73 01 c3 48 8b 0d 05 f2 1b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 f1 1b 00 f7 d8 64 89 01 48
>  RSP: 002b:00007fff18635208 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
>  RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fbc0ca3fc1b
>  RDX: 0000000000000006 RSI: 0000000000000001 RDI: 0000000000000002
>  RBP: 000055d24f80f8a0 R08: 0000000000000003 R09: 0000000000000001
> 
> R10: 0000000000020000 R11: 0000000000000246 R12: 000055d24f80f8a0
>  R13: 0000000000000000 R14: 000055d24f80fb88 R15: 0000000000000000
>   </TASK>
> 
> The current implementation of cipso_v4_sock_setattr() replaces IP options
> under the assumption that the caller holds the socket lock; however, such
> assumption is not true, nor needed, in selinux_socket_post_create() hook.
> 
> Let all callers of cipso_v4_sock_setattr() specify the "socket lock held"
> condition, except selinux_socket_post_create() _ where such condition can
> safely be set as true even without holding the socket lock.
> While at it: use rcu_replace_pointer() instead of open coding, and remove
> useless NULL check of 'old' before kfree_rcu(old, ...).
> 
> v2:
>  - pass lockdep_sock_is_held() through a boolean variable in the stack
>    (thanks Eric Dumazet, Paul Moore, Casey Schaufler)
>  - use rcu_replace_pointer() instead of rcu_dereference_protected() +
>    rcu_assign_pointer()
>  - remove NULL check of 'old' before kfree_rcu()
> 
> Fixes: f6d8bd051c39 ("inet: add RCU protection to inet->opt")
> Reported-by: Xiumei Mu <xmu@redhat.com>
> Signed-off-by: Davide Caratti <dcaratti@redhat.com>
> ---
>  include/net/cipso_ipv4.h     |  6 ++++--
>  include/net/netlabel.h       |  6 ++++--
>  net/ipv4/cipso_ipv4.c        | 13 ++++++-------
>  net/netlabel/netlabel_kapi.c |  9 ++++++---
>  security/selinux/netlabel.c  |  5 ++++-
>  security/smack/smack_lsm.c   |  3 ++-
>  6 files changed, 26 insertions(+), 16 deletions(-)

...

> diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c
> index 8b17d83e5fde..c4ac704cbcc2 100644
> --- a/net/ipv4/cipso_ipv4.c
> +++ b/net/ipv4/cipso_ipv4.c
> @@ -1815,6 +1815,7 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len,
>   * @sk: the socket
>   * @doi_def: the CIPSO DOI to use
>   * @secattr: the specific security attributes of the socket
> + * @slock_held: true if caller holds the socket lock
>   *
>   * Description:
>   * Set the CIPSO option on the given socket using the DOI definition and
> @@ -1826,7 +1827,8 @@ static int cipso_v4_genopt(unsigned char *buf, u32 buf_len,
>   */
>  int cipso_v4_sock_setattr(struct sock *sk,
>  			  const struct cipso_v4_doi *doi_def,
> -			  const struct netlbl_lsm_secattr *secattr)
> +			  const struct netlbl_lsm_secattr *secattr,
> +			  bool slock_held)

This is a nitpicky bikeshedding remark, but "slock_held" sounds really
awkward to me, something like "sk_locked" sounds much better.

>  {
>  	int ret_val = -EPERM;
>  	unsigned char *buf = NULL;
> @@ -1876,18 +1878,15 @@ int cipso_v4_sock_setattr(struct sock *sk,
>  
>  	sk_inet = inet_sk(sk);
>  
> -	old = rcu_dereference_protected(sk_inet->inet_opt,
> -					lockdep_sock_is_held(sk));
> +	old = rcu_replace_pointer(sk_inet->inet_opt, opt, slock_held);
>  	if (inet_test_bit(IS_ICSK, sk)) {
>  		sk_conn = inet_csk(sk);
>  		if (old)
>  			sk_conn->icsk_ext_hdr_len -= old->opt.optlen;
> -		sk_conn->icsk_ext_hdr_len += opt->opt.optlen;
> +		sk_conn->icsk_ext_hdr_len += opt_len;
>  		sk_conn->icsk_sync_mss(sk, sk_conn->icsk_pmtu_cookie);
>  	}
> -	rcu_assign_pointer(sk_inet->inet_opt, opt);
> -	if (old)
> -		kfree_rcu(old, rcu);
> +	kfree_rcu(old, rcu);

Thanks for sticking with this and posting a v2.

These changes look okay to me, but considering the 'Fixes:' tag and the
RCU splat it is reasonable to expect that this is going to be backported
to the various stable trees.  With that in mind, I think we should try
to keep the immediate fix as simple as possible, saving the other
changes for a separate patch.  This means sticking with
rcu_dereference_protected() and omitting the opt_len optimization; both
can be done in a second patch without the 'Fixes:' marking.

Unless I missing something and those changes are somehow part of the
fix?

--
paul-moore.com