bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support
@ 2022-11-22 19:53 Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu"))) Yonghong Song
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Yonghong Song @ 2022-11-22 19:53 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau

Currently, without rcu attribute info in BTF, the verifier treats
rcu tagged pointer as a normal pointer. This might be a problem
for sleepable program where rcu_read_lock()/unlock() is not available.
For example, for a sleepable fentry program, if rcu protected memory
access is interleaved with a sleepable helper/kfunc, it is possible
the memory access after the sleepable helper/kfunc might be invalid
since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection,
it is possible that the rcu protected object might be release
in the middle of bpf program execution which may cause incorrect
result.

To prevent above cases, enable btf_type_tag("rcu") attributes,
introduce new bpf_rcu_read_lock/unlock() kfuncs and add verifier support.

In the rest of patch set, Patch 1 enabled btf_type_tag for __rcu
attribute. Patche 2 added might_sleep in bpf_func_proto. Patch 3 added new
bpf_rcu_read_lock/unlock() kfuncs and verifier support.
Patch 4 added some tests for these two new kfuncs.

Changelogs:
  v7 -> v8:
    . add might_sleep in bpf_func_proto so we can easily identify whether
      a helper is sleepable or not.
    . do not enforce rcu rules for sleepable, e.g., rcu dereference must
      be in a bpf_rcu_read_lock region. This is to keep old code working
      fine.
    . Mark 'b' in 'b = a->b' (b is tagged with __rcu) as MEM_RCU only if
      'b = a->b' in rcu read region and 'a' is trusted. This adds safety
      guarantee for 'b' inside the rcu read region.
  v6 -> v7:
    . rebase on top of bpf-next.
    . remove the patch which enables sleepable program using
      cgrp_local_storage map. This is orthogonal to this patch set
      and will be addressed separately.
    . mark the rcu pointer dereference result as UNTRUSTED if inside
      a bpf_rcu_read_lock() region.
  v5 -> v6:
    . fix selftest prog miss_unlock which tested nested locking.
    . add comments in selftest prog cgrp_succ to explain how to handle
      nested memory access after rcu memory load.
  v4 -> v5:
    . add new test to aarch64 deny list.
  v3 -> v4:
    . fix selftest failures when built with gcc. gcc doesn't support
      btf_type_tag yet and some tests relies on that. skip these
      tests if vmlinux BTF does not have btf_type_tag("rcu").
  v2 -> v3:
    . went back to MEM_RCU approach with invalidate rcu ptr registers
      at bpf_rcu_read_unlock() place.
    . remove KF_RCU_LOCK/UNLOCK flag and compare btf_id at verification
      time instead.
  v1 -> v2:
    . use kfunc instead of helper for bpf_rcu_read_lock/unlock.
    . not use MEM_RCU bpf_type_flag, instead use active_rcu_lock
      in reg state to identify rcu ptr's.
    . Add more self tests.
    . add new test to s390x deny list.

Yonghong Song (4):
  compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
  bpf: Introduce might_sleep field in bpf_func_proto
  bpf: Add kfunc bpf_rcu_read_lock/unlock()
  selftests/bpf: Add tests for bpf_rcu_read_lock()

 include/linux/bpf.h                           |   4 +
 include/linux/bpf_verifier.h                  |   4 +-
 include/linux/compiler_types.h                |   3 +-
 kernel/bpf/bpf_lsm.c                          |   6 +-
 kernel/bpf/btf.c                              |   3 +
 kernel/bpf/helpers.c                          |  14 +
 kernel/bpf/verifier.c                         | 123 ++++++-
 kernel/trace/bpf_trace.c                      |   4 +-
 tools/testing/selftests/bpf/DENYLIST.aarch64  |   1 +
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/rcu_read_lock.c  | 138 ++++++++
 .../selftests/bpf/progs/rcu_read_lock.c       | 306 ++++++++++++++++++
 12 files changed, 591 insertions(+), 16 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c

-- 
2.30.2


^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH bpf-next v8 1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
  2022-11-22 19:53 [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support Yonghong Song
@ 2022-11-22 19:53 ` Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 2/4] bpf: Introduce might_sleep field in bpf_func_proto Yonghong Song
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Yonghong Song @ 2022-11-22 19:53 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, KP Singh

Currently, without rcu attribute info in BTF, the verifier treats
rcu tagged pointer as a normal pointer. This might be a problem
for sleepable program where rcu_read_lock()/unlock() is not available.
For example, for a sleepable fentry program, if rcu protected memory
access is interleaved with a sleepable helper/kfunc, it is possible
the memory access after the sleepable helper/kfunc might be invalid
since the object might have been freed then. To prevent such cases,
introducing rcu tagging for memory accesses in verifier can help
to reject such programs.

To enable rcu tagging in BTF, during kernel compilation,
define __rcu as attribute btf_type_tag("rcu") so __rcu information can
be preserved in dwarf and btf, and later can be used for bpf prog verification.

Acked-by: KP Singh <kpsingh@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
---
 include/linux/compiler_types.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index eb0466236661..7c1afe0f4129 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -49,7 +49,8 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 # endif
 # define __iomem
 # define __percpu	BTF_TYPE_TAG(percpu)
-# define __rcu
+# define __rcu		BTF_TYPE_TAG(rcu)
+
 # define __chk_user_ptr(x)	(void)0
 # define __chk_io_ptr(x)	(void)0
 /* context/locking */
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH bpf-next v8 2/4] bpf: Introduce might_sleep field in bpf_func_proto
  2022-11-22 19:53 [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu"))) Yonghong Song
@ 2022-11-22 19:53 ` Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock() Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock() Yonghong Song
  3 siblings, 0 replies; 14+ messages in thread
From: Yonghong Song @ 2022-11-22 19:53 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau

Introduce bpf_func_proto->might_sleep to indicate a particular helper
might sleep. This will make later check whether a helper might be
sleepable or not easier.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 include/linux/bpf.h      | 1 +
 kernel/bpf/bpf_lsm.c     | 6 ++++--
 kernel/bpf/helpers.c     | 2 ++
 kernel/bpf/verifier.c    | 5 +++++
 kernel/trace/bpf_trace.c | 4 ++--
 5 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index c9eafa67f2a2..43fd7eeeeabb 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -682,6 +682,7 @@ struct bpf_func_proto {
 	u64 (*func)(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
 	bool gpl_only;
 	bool pkt_access;
+	bool might_sleep;
 	enum bpf_return_type ret_type;
 	union {
 		struct {
diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c
index d6c9b3705f24..ae0267f150b5 100644
--- a/kernel/bpf/bpf_lsm.c
+++ b/kernel/bpf/bpf_lsm.c
@@ -151,6 +151,7 @@ BTF_ID_LIST_SINGLE(bpf_ima_inode_hash_btf_ids, struct, inode)
 static const struct bpf_func_proto bpf_ima_inode_hash_proto = {
 	.func		= bpf_ima_inode_hash,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_BTF_ID,
 	.arg1_btf_id	= &bpf_ima_inode_hash_btf_ids[0],
@@ -169,6 +170,7 @@ BTF_ID_LIST_SINGLE(bpf_ima_file_hash_btf_ids, struct, file)
 static const struct bpf_func_proto bpf_ima_file_hash_proto = {
 	.func		= bpf_ima_file_hash,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_BTF_ID,
 	.arg1_btf_id	= &bpf_ima_file_hash_btf_ids[0],
@@ -221,9 +223,9 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	case BPF_FUNC_bprm_opts_set:
 		return &bpf_bprm_opts_set_proto;
 	case BPF_FUNC_ima_inode_hash:
-		return prog->aux->sleepable ? &bpf_ima_inode_hash_proto : NULL;
+		return &bpf_ima_inode_hash_proto;
 	case BPF_FUNC_ima_file_hash:
-		return prog->aux->sleepable ? &bpf_ima_file_hash_proto : NULL;
+		return &bpf_ima_file_hash_proto;
 	case BPF_FUNC_get_attach_cookie:
 		return bpf_prog_has_trampoline(prog) ? &bpf_get_attach_cookie_proto : NULL;
 #ifdef CONFIG_NET
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 9e8a5557ea8d..c3b798a1e3e9 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -660,6 +660,7 @@ BPF_CALL_3(bpf_copy_from_user, void *, dst, u32, size,
 const struct bpf_func_proto bpf_copy_from_user_proto = {
 	.func		= bpf_copy_from_user,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_UNINIT_MEM,
 	.arg2_type	= ARG_CONST_SIZE_OR_ZERO,
@@ -690,6 +691,7 @@ BPF_CALL_5(bpf_copy_from_user_task, void *, dst, u32, size,
 const struct bpf_func_proto bpf_copy_from_user_task_proto = {
 	.func		= bpf_copy_from_user_task,
 	.gpl_only	= true,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_UNINIT_MEM,
 	.arg2_type	= ARG_CONST_SIZE_OR_ZERO,
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9528a066cfa5..068cc885903c 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -7516,6 +7516,11 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 		return -EINVAL;
 	}
 
+	if (!env->prog->aux->sleepable && fn->might_sleep) {
+		verbose(env, "helper call might sleep in a non-sleepable prog\n");
+		return -EINVAL;
+	}
+
 	/* With LD_ABS/IND some JITs save/restore skb from r1. */
 	changes_data = bpf_helper_changes_pkt_data(fn->func);
 	if (changes_data && fn->arg1_type != ARG_PTR_TO_CTX) {
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 5b9008bc597b..3bbd3f0c810c 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -1485,9 +1485,9 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	case BPF_FUNC_get_task_stack:
 		return &bpf_get_task_stack_proto;
 	case BPF_FUNC_copy_from_user:
-		return prog->aux->sleepable ? &bpf_copy_from_user_proto : NULL;
+		return &bpf_copy_from_user_proto;
 	case BPF_FUNC_copy_from_user_task:
-		return prog->aux->sleepable ? &bpf_copy_from_user_task_proto : NULL;
+		return &bpf_copy_from_user_task_proto;
 	case BPF_FUNC_snprintf_btf:
 		return &bpf_snprintf_btf_proto;
 	case BPF_FUNC_per_cpu_ptr:
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
  2022-11-22 19:53 [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu"))) Yonghong Song
  2022-11-22 19:53 ` [PATCH bpf-next v8 2/4] bpf: Introduce might_sleep field in bpf_func_proto Yonghong Song
@ 2022-11-22 19:53 ` Yonghong Song
  2022-11-23  0:24   ` Martin KaFai Lau
  2022-11-22 19:53 ` [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock() Yonghong Song
  3 siblings, 1 reply; 14+ messages in thread
From: Yonghong Song @ 2022-11-22 19:53 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau

Add two kfunc's bpf_rcu_read_lock() and bpf_rcu_read_unlock(). These two kfunc's
can be used for all program types. The following is an example about how
rcu pointer are used w.r.t. bpf_rcu_read_lock()/bpf_rcu_read_unlock().

  struct task_struct {
    ...
    struct task_struct              *last_wakee;
    struct task_struct __rcu        *real_parent;
    ...
  };

Let us say prog does 'task = bpf_get_current_task_btf()' to get a
'task' pointer. The basic rules are:
  - 'real_parent = task->real_parent' should be inside bpf_rcu_read_lock
    region.  this is to simulate rcu_dereference() operation. The
    'real_parent' is marked as MEM_RCU only if (1). task->real_parent is
    inside bpf_rcu_read_lock region, and (2). task is a trusted ptr,
    marked with PTR_TRUSTED or ref_obj_id != 0. In other words,
    MEM_RCU marked ptr can be 'trusted' inside the bpf_rcu_read_lock region.
  - 'last_wakee = real_parent->last_wakee' should be inside bpf_rcu_read_lock
    region since it tries to access rcu protected memory.
  - the ptr 'last_wakee' will be marked as PTR_UNTRUSTED since in general
    it is not clear whether the object pointed by 'last_wakee' is valid or
    not even inside bpf_rcu_read_lock region.

To prevent rcu pointer leaks outside the rcu read lock region.
The verifier will clear all rcu pointer register state to unknown, i.e.,
scalar_value, at bpf_rcu_read_unlock() kfunc call site,
so later dereference becomes impossible.

The current implementation does not support nested rcu read lock
region in the prog.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 include/linux/bpf.h          |   3 +
 include/linux/bpf_verifier.h |   4 +-
 kernel/bpf/btf.c             |   3 +
 kernel/bpf/helpers.c         |  12 ++++
 kernel/bpf/verifier.c        | 118 ++++++++++++++++++++++++++++++++---
 5 files changed, 129 insertions(+), 11 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 43fd7eeeeabb..c6aa6912ea16 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -572,6 +572,9 @@ enum bpf_type_flag {
 	 */
 	PTR_TRUSTED		= BIT(12 + BPF_BASE_TYPE_BITS),
 
+	/* MEM is tagged with rcu and memory access needs rcu_read_lock protection. */
+	MEM_RCU			= BIT(13 + BPF_BASE_TYPE_BITS),
+
 	__BPF_TYPE_FLAG_MAX,
 	__BPF_TYPE_LAST_FLAG	= __BPF_TYPE_FLAG_MAX - 1,
 };
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 545152ac136c..1f3ce54e50ed 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -344,6 +344,7 @@ struct bpf_verifier_state {
 		u32 id;
 	} active_lock;
 	bool speculative;
+	bool active_rcu_lock;
 
 	/* first and last insn idx of this verifier state */
 	u32 first_insn_idx;
@@ -445,6 +446,7 @@ struct bpf_insn_aux_data {
 	u32 seen; /* this insn was processed by the verifier at env->pass_cnt */
 	bool sanitize_stack_spill; /* subject to Spectre v4 sanitation */
 	bool zext_dst; /* this insn zero extends dst reg */
+	bool storage_get_func_atomic; /* bpf_*_storage_get() with atomic memory alloc */
 	u8 alu_state; /* used in combination with alu_limit */
 
 	/* below fields are initialized once */
@@ -680,7 +682,7 @@ static inline bool bpf_prog_check_recur(const struct bpf_prog *prog)
 	}
 }
 
-#define BPF_REG_TRUSTED_MODIFIERS (MEM_ALLOC | PTR_TRUSTED)
+#define BPF_REG_TRUSTED_MODIFIERS (MEM_ALLOC | MEM_RCU | PTR_TRUSTED)
 
 static inline bool bpf_type_has_unsafe_modifiers(u32 type)
 {
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 1a59cc7ad730..68df0df27302 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6237,6 +6237,9 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 				/* check __percpu tag */
 				if (strcmp(tag_value, "percpu") == 0)
 					tmp_flag = MEM_PERCPU;
+				/* check __rcu tag */
+				if (strcmp(tag_value, "rcu") == 0)
+					tmp_flag = MEM_RCU;
 			}
 
 			stype = btf_type_skip_modifiers(btf, mtype->type, &id);
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index c3b798a1e3e9..3c5b41604c7b 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -1891,6 +1891,16 @@ void *bpf_rdonly_cast(void *obj__ign, u32 btf_id__k)
 	return obj__ign;
 }
 
+void bpf_rcu_read_lock(void)
+{
+	rcu_read_lock();
+}
+
+void bpf_rcu_read_unlock(void)
+{
+	rcu_read_unlock();
+}
+
 __diag_pop();
 
 BTF_SET8_START(generic_btf_ids)
@@ -1921,6 +1931,8 @@ BTF_ID(func, bpf_task_release)
 BTF_SET8_START(common_btf_ids)
 BTF_ID_FLAGS(func, bpf_cast_to_kern_ctx)
 BTF_ID_FLAGS(func, bpf_rdonly_cast)
+BTF_ID_FLAGS(func, bpf_rcu_read_lock)
+BTF_ID_FLAGS(func, bpf_rcu_read_unlock)
 BTF_SET8_END(common_btf_ids)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 068cc885903c..f6105723d6ca 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -527,6 +527,14 @@ static bool is_callback_calling_function(enum bpf_func_id func_id)
 	       func_id == BPF_FUNC_user_ringbuf_drain;
 }
 
+static bool is_storage_get_function(enum bpf_func_id func_id)
+{
+	return func_id == BPF_FUNC_sk_storage_get ||
+	       func_id == BPF_FUNC_inode_storage_get ||
+	       func_id == BPF_FUNC_task_storage_get ||
+	       func_id == BPF_FUNC_cgrp_storage_get;
+}
+
 static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id,
 					const struct bpf_map *map)
 {
@@ -589,11 +597,12 @@ static const char *reg_type_str(struct bpf_verifier_env *env,
 			strncpy(postfix, "_or_null", 16);
 	}
 
-	snprintf(prefix, sizeof(prefix), "%s%s%s%s%s%s",
+	snprintf(prefix, sizeof(prefix), "%s%s%s%s%s%s%s",
 		 type & MEM_RDONLY ? "rdonly_" : "",
 		 type & MEM_RINGBUF ? "ringbuf_" : "",
 		 type & MEM_USER ? "user_" : "",
 		 type & MEM_PERCPU ? "percpu_" : "",
+		 type & MEM_RCU ? "rcu_" : "",
 		 type & PTR_UNTRUSTED ? "untrusted_" : "",
 		 type & PTR_TRUSTED ? "trusted_" : ""
 	);
@@ -1220,6 +1229,7 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state,
 		dst_state->frame[i] = NULL;
 	}
 	dst_state->speculative = src->speculative;
+	dst_state->active_rcu_lock = src->active_rcu_lock;
 	dst_state->curframe = src->curframe;
 	dst_state->active_lock.ptr = src->active_lock.ptr;
 	dst_state->active_lock.id = src->active_lock.id;
@@ -4737,9 +4747,30 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,
 	if (type_flag(reg->type) & PTR_UNTRUSTED)
 		flag |= PTR_UNTRUSTED;
 
-	/* Any pointer obtained from walking a trusted pointer is no longer trusted. */
+	/* By default any pointer obtained from walking a trusted pointer is
+	 * no longer trusted except the rcu case below.
+	 */
 	flag &= ~PTR_TRUSTED;
 
+	if (flag & MEM_RCU) {
+		/* Mark value register as MEM_RCU only if it is protected by
+		 * bpf_rcu_read_lock() and the ptr reg is trusted (PTR_TRUSTED or
+		 * ref_obj_id != 0). MEM_RCU itself can already indicate
+		 * trustedness inside the rcu read lock region. But Mark it
+		 * as PTR_TRUSTED as well similar to MEM_ALLOC.
+		 */
+		if (!env->cur_state->active_rcu_lock ||
+		    (!(reg->type & PTR_TRUSTED) && !reg->ref_obj_id))
+			flag &= ~MEM_RCU;
+		else
+			flag |= PTR_TRUSTED;
+	} else if (reg->type & MEM_RCU) {
+		/* ptr (reg) is marked as MEM_RCU, but value reg is not marked as MEM_RCU.
+		 * Mark the value reg as PTR_UNTRUSTED conservatively.
+		 */
+		flag |= PTR_UNTRUSTED;
+	}
+
 	if (atype == BPF_READ && value_regno >= 0)
 		mark_btf_ld_reg(env, regs, value_regno, ret, reg->btf, btf_id, flag);
 
@@ -5897,6 +5928,7 @@ static const struct bpf_reg_types btf_ptr_types = {
 	.types = {
 		PTR_TO_BTF_ID,
 		PTR_TO_BTF_ID | PTR_TRUSTED,
+		PTR_TO_BTF_ID | MEM_RCU | PTR_TRUSTED,
 	},
 };
 static const struct bpf_reg_types percpu_btf_ptr_types = {
@@ -6075,6 +6107,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env,
 	case PTR_TO_BTF_ID:
 	case PTR_TO_BTF_ID | MEM_ALLOC:
 	case PTR_TO_BTF_ID | PTR_TRUSTED:
+	case PTR_TO_BTF_ID | MEM_RCU | PTR_TRUSTED:
 	case PTR_TO_BTF_ID | MEM_ALLOC | PTR_TRUSTED:
 		/* When referenced PTR_TO_BTF_ID is passed to release function,
 		 * it's fixed offset must be 0.	In the other cases, fixed offset
@@ -7539,6 +7572,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 		return err;
 	}
 
+	if (env->cur_state->active_rcu_lock) {
+		if (fn->might_sleep) {
+			verbose(env, "sleepable helper %s#%din rcu_read_lock region\n",
+				func_id_name(func_id), func_id);
+			return -EINVAL;
+		}
+
+		if (env->prog->aux->sleepable && is_storage_get_function(func_id))
+			env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+	}
+
 	meta.func_id = func_id;
 	/* check args */
 	for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) {
@@ -8163,6 +8207,8 @@ enum special_kfunc_type {
 	KF_bpf_list_pop_back,
 	KF_bpf_cast_to_kern_ctx,
 	KF_bpf_rdonly_cast,
+	KF_bpf_rcu_read_lock,
+	KF_bpf_rcu_read_unlock,
 };
 
 BTF_SET_START(special_kfunc_set)
@@ -8185,6 +8231,18 @@ BTF_ID(func, bpf_list_pop_front)
 BTF_ID(func, bpf_list_pop_back)
 BTF_ID(func, bpf_cast_to_kern_ctx)
 BTF_ID(func, bpf_rdonly_cast)
+BTF_ID(func, bpf_rcu_read_lock)
+BTF_ID(func, bpf_rcu_read_unlock)
+
+static bool is_kfunc_bpf_rcu_read_lock(struct bpf_kfunc_call_arg_meta *meta)
+{
+	return meta->func_id == special_kfunc_list[KF_bpf_rcu_read_lock];
+}
+
+static bool is_kfunc_bpf_rcu_read_unlock(struct bpf_kfunc_call_arg_meta *meta)
+{
+	return meta->func_id == special_kfunc_list[KF_bpf_rcu_read_unlock];
+}
 
 static enum kfunc_ptr_arg_type
 get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
@@ -8817,6 +8875,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	const struct btf_type *t, *func, *func_proto, *ptr_type;
 	struct bpf_reg_state *regs = cur_regs(env);
 	const char *func_name, *ptr_type_name;
+	bool sleepable, rcu_lock, rcu_unlock;
 	struct bpf_kfunc_call_arg_meta meta;
 	u32 i, nargs, func_id, ptr_type_id;
 	int err, insn_idx = *insn_idx_p;
@@ -8858,11 +8917,38 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		return -EACCES;
 	}
 
-	if (is_kfunc_sleepable(&meta) && !env->prog->aux->sleepable) {
+	sleepable = is_kfunc_sleepable(&meta);
+	if (sleepable && !env->prog->aux->sleepable) {
 		verbose(env, "program must be sleepable to call sleepable kfunc %s\n", func_name);
 		return -EACCES;
 	}
 
+	rcu_lock = is_kfunc_bpf_rcu_read_lock(&meta);
+	rcu_unlock = is_kfunc_bpf_rcu_read_unlock(&meta);
+	if (env->cur_state->active_rcu_lock) {
+		struct bpf_func_state *state;
+		struct bpf_reg_state *reg;
+
+		if (rcu_lock) {
+			verbose(env, "nested rcu read lock (kernel function %s)\n", func_name);
+			return -EINVAL;
+		} else if (rcu_unlock) {
+			bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({
+				if (reg->type & MEM_RCU)
+					__mark_reg_unknown(env, reg);
+			}));
+			env->cur_state->active_rcu_lock = false;
+		} else if (sleepable) {
+			verbose(env, "kernel func %s is sleepable within rcu_read_lock region\n", func_name);
+			return -EACCES;
+		}
+	} else if (rcu_lock) {
+		env->cur_state->active_rcu_lock = true;
+	} else if (rcu_unlock) {
+		verbose(env, "unmatched rcu read unlock (kernel function %s)\n", func_name);
+		return -EINVAL;
+	}
+
 	/* Check the arguments */
 	err = check_kfunc_args(env, &meta);
 	if (err < 0)
@@ -11754,6 +11840,11 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		return -EINVAL;
 	}
 
+	if (env->prog->aux->sleepable && env->cur_state->active_rcu_lock) {
+		verbose(env, "BPF_LD_[ABS|IND] cannot be used inside bpf_rcu_read_lock-ed region\n");
+		return -EINVAL;
+	}
+
 	if (regs[ctx_reg].type != PTR_TO_CTX) {
 		verbose(env,
 			"at the time of BPF_LD_ABS|IND R6 != pointer to skb\n");
@@ -13019,6 +13110,9 @@ static bool states_equal(struct bpf_verifier_env *env,
 	    old->active_lock.id != cur->active_lock.id)
 		return false;
 
+	if (old->active_rcu_lock != cur->active_rcu_lock)
+		return false;
+
 	/* for states to be equal callsites have to be the same
 	 * and all frame states need to be equivalent
 	 */
@@ -13706,6 +13800,11 @@ static int do_check(struct bpf_verifier_env *env)
 					return -EINVAL;
 				}
 
+				if (env->cur_state->active_rcu_lock) {
+					verbose(env, "bpf_rcu_read_unlock is missing\n");
+					return -EINVAL;
+				}
+
 				/* We must do check_reference_leak here before
 				 * prepare_func_exit to handle the case when
 				 * state->curframe > 0, it may be a callback
@@ -14802,6 +14901,7 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		case PTR_TO_BTF_ID:
 		case PTR_TO_BTF_ID | PTR_UNTRUSTED:
 		case PTR_TO_BTF_ID | PTR_TRUSTED:
+		case PTR_TO_BTF_ID | MEM_RCU | PTR_TRUSTED:
 		/* PTR_TO_BTF_ID | MEM_ALLOC always has a valid lifetime, unlike
 		 * PTR_TO_BTF_ID, and an active ref_obj_id, but the same cannot
 		 * be said once it is marked PTR_UNTRUSTED, hence we must handle
@@ -15494,14 +15594,12 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
 			goto patch_call_imm;
 		}
 
-		if (insn->imm == BPF_FUNC_task_storage_get ||
-		    insn->imm == BPF_FUNC_sk_storage_get ||
-		    insn->imm == BPF_FUNC_inode_storage_get ||
-		    insn->imm == BPF_FUNC_cgrp_storage_get) {
-			if (env->prog->aux->sleepable)
-				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL);
-			else
+		if (is_storage_get_function(insn->imm)) {
+			if (!env->prog->aux->sleepable ||
+			    env->insn_aux_data[i + delta].storage_get_func_atomic)
 				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
+			else
+				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL);
 			insn_buf[1] = *insn;
 			cnt = 2;
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
  2022-11-22 19:53 [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support Yonghong Song
                   ` (2 preceding siblings ...)
  2022-11-22 19:53 ` [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock() Yonghong Song
@ 2022-11-22 19:53 ` Yonghong Song
  2022-11-23  0:56   ` Martin KaFai Lau
  3 siblings, 1 reply; 14+ messages in thread
From: Yonghong Song @ 2022-11-22 19:53 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau

Add a few positive/negative tests to test bpf_rcu_read_lock()
and its corresponding verifier support. The new test will fail
on s390x and aarch64, so an entry is added to each of their
respective deny lists.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 tools/testing/selftests/bpf/DENYLIST.aarch64  |   1 +
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/rcu_read_lock.c  | 138 ++++++++
 .../selftests/bpf/progs/rcu_read_lock.c       | 306 ++++++++++++++++++
 4 files changed, 446 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c

diff --git a/tools/testing/selftests/bpf/DENYLIST.aarch64 b/tools/testing/selftests/bpf/DENYLIST.aarch64
index affc5aebbf0f..8e77515d56f6 100644
--- a/tools/testing/selftests/bpf/DENYLIST.aarch64
+++ b/tools/testing/selftests/bpf/DENYLIST.aarch64
@@ -45,6 +45,7 @@ modify_return                                    # modify_return__attach failed
 module_attach                                    # skel_attach skeleton attach failed: -524
 mptcp/base                                       # run_test mptcp unexpected error: -524 (errno 524)
 netcnt                                           # packets unexpected packets: actual 10001 != expected 10000
+rcu_read_lock                                    # failed to attach: ERROR: strerror_r(-524)=22
 recursion                                        # skel_attach unexpected error: -524 (errno 524)
 ringbuf                                          # skel_attach skeleton attachment failed: -1
 setget_sockopt                                   # attach_cgroup unexpected error: -524
diff --git a/tools/testing/selftests/bpf/DENYLIST.s390x b/tools/testing/selftests/bpf/DENYLIST.s390x
index 12cf2159975e..77aabc4dc64c 100644
--- a/tools/testing/selftests/bpf/DENYLIST.s390x
+++ b/tools/testing/selftests/bpf/DENYLIST.s390x
@@ -42,6 +42,7 @@ module_attach                            # skel_attach skeleton attach failed: -
 mptcp
 netcnt                                   # failed to load BPF skeleton 'netcnt_prog': -7                               (?)
 probe_user                               # check_kprobe_res wrong kprobe res from probe read                           (?)
+rcu_read_lock                            # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3                (?)
 recursion                                # skel_attach unexpected error: -524                                          (trampoline)
 ringbuf                                  # skel_load skeleton load failed                                              (?)
 select_reuseport                         # intermittently fails on new s390x setup
diff --git a/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c b/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
new file mode 100644
index 000000000000..b0cb8b1389cb
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates.*/
+
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <sys/syscall.h>
+#include <sys/types.h>
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include "rcu_read_lock.skel.h"
+#include "cgroup_helpers.h"
+
+static unsigned long long cgroup_id;
+
+static void test_local_storage(void)
+{
+	struct rcu_read_lock *skel;
+	int err;
+
+	skel = rcu_read_lock__open();
+	if (!ASSERT_OK_PTR(skel, "skel_open"))
+		return;
+
+	skel->bss->target_pid = syscall(SYS_gettid);
+
+	bpf_program__set_autoload(skel->progs.get_cgroup_id, true);
+	bpf_program__set_autoload(skel->progs.task_succ, true);
+	bpf_program__set_autoload(skel->progs.task_acquire, true);
+	bpf_program__set_autoload(skel->progs.no_lock, true);
+	bpf_program__set_autoload(skel->progs.two_regions, true);
+	bpf_program__set_autoload(skel->progs.non_sleepable_1, true);
+	bpf_program__set_autoload(skel->progs.non_sleepable_2, true);
+	err = rcu_read_lock__load(skel);
+	if (!ASSERT_OK(err, "skel_load"))
+		goto done;
+
+	err = rcu_read_lock__attach(skel);
+	if (!ASSERT_OK(err, "skel_attach"))
+		goto done;
+
+	syscall(SYS_getpgid);
+
+	ASSERT_EQ(skel->bss->task_storage_val, 2, "task_storage_val");
+	ASSERT_EQ(skel->bss->cgroup_id, cgroup_id, "cgroup_id");
+done:
+	rcu_read_lock__destroy(skel);
+}
+
+static const char * const inproper_region_tests[] = {
+	"miss_lock",
+	"miss_unlock",
+	"non_sleepable_rcu_mismatch",
+	"inproper_sleepable_helper",
+	"inproper_sleepable_kfunc",
+	"nested_rcu_region",
+};
+
+static void test_inproper_region(void)
+{
+	struct rcu_read_lock *skel;
+	struct bpf_program *prog;
+	int i, err;
+
+	for (i = 0; i < ARRAY_SIZE(inproper_region_tests); i++) {
+		skel = rcu_read_lock__open();
+		if (!ASSERT_OK_PTR(skel, "skel_open"))
+			return;
+
+		prog = bpf_object__find_program_by_name(skel->obj, inproper_region_tests[i]);
+		if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name"))
+			goto out;
+		bpf_program__set_autoload(prog, true);
+		err = rcu_read_lock__load(skel);
+		ASSERT_ERR(err, "skel_load");
+out:
+		rcu_read_lock__destroy(skel);
+	}
+}
+
+static const char * const rcuptr_misuse_tests[] = {
+	"task_untrusted_ptr",
+	"task_incorrect_rcu_region1",
+	"task_incorrect_rcu_region2",
+	"cross_rcu_region",
+};
+
+static void test_rcuptr_misuse(void)
+{
+	struct rcu_read_lock *skel;
+	struct bpf_program *prog;
+	struct btf *vmlinux_btf;
+	int i, err, type_id;
+
+	vmlinux_btf = btf__load_vmlinux_btf();
+	if (!ASSERT_OK_PTR(vmlinux_btf, "could not load vmlinux BTF"))
+		return;
+
+	/* skip the test if btf_type_tag("rcu") is not present in vmlinux */
+	type_id = btf__find_by_name_kind(vmlinux_btf, "rcu", BTF_KIND_TYPE_TAG);
+	if (type_id < 0) {
+		test__skip();
+		return;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(rcuptr_misuse_tests); i++) {
+		skel = rcu_read_lock__open();
+		if (!ASSERT_OK_PTR(skel, "skel_open"))
+			return;
+
+		prog = bpf_object__find_program_by_name(skel->obj, rcuptr_misuse_tests[i]);
+		if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name"))
+			goto out;
+		bpf_program__set_autoload(prog, true);
+		err = rcu_read_lock__load(skel);
+		ASSERT_ERR(err, "skel_load");
+out:
+		rcu_read_lock__destroy(skel);
+	}
+}
+
+void test_rcu_read_lock(void)
+{
+	int cgroup_fd;
+
+	cgroup_fd = test__join_cgroup("/rcu_read_lock");
+	if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup /rcu_read_lock"))
+		return;
+
+	cgroup_id = get_cgroup_id("/rcu_read_lock");
+	if (test__start_subtest("local_storage"))
+		test_local_storage();
+	if (test__start_subtest("negative_tests_inproper_region"))
+		test_inproper_region();
+	if (test__start_subtest("negative_tests_rcuptr_misuse"))
+		test_rcuptr_misuse();
+
+	close(cgroup_fd);
+}
diff --git a/tools/testing/selftests/bpf/progs/rcu_read_lock.c b/tools/testing/selftests/bpf/progs/rcu_read_lock.c
new file mode 100644
index 000000000000..cf470a095ef3
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/rcu_read_lock.c
@@ -0,0 +1,306 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_tracing_net.h"
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct {
+	__uint(type, BPF_MAP_TYPE_TASK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, long);
+} map_a SEC(".maps");
+
+__u32 user_data, key_serial, target_pid = 0;
+__u64 flags, task_storage_val, cgroup_id;
+
+struct bpf_key *bpf_lookup_user_key(__u32 serial, __u64 flags) __ksym;
+void bpf_key_put(struct bpf_key *key) __ksym;
+void bpf_rcu_read_lock(void) __ksym;
+void bpf_rcu_read_unlock(void) __ksym;
+struct task_struct *bpf_task_acquire(struct task_struct *p) __ksym;
+void bpf_task_release(struct task_struct *p) __ksym;
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int get_cgroup_id(void *ctx)
+{
+	struct task_struct *task;
+
+	task = bpf_get_current_task_btf();
+	if (task->pid != target_pid)
+		return 0;
+
+	/* simulate bpf_get_current_cgroup_id() helper */
+	bpf_rcu_read_lock();
+	cgroup_id = task->cgroups->dfl_cgrp->kn->id;
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int task_succ(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+	long init_val = 2;
+	long *ptr;
+
+	task = bpf_get_current_task_btf();
+	if (task->pid != target_pid)
+		return 0;
+
+	bpf_rcu_read_lock();
+	/* region including helper using rcu ptr real_parent */
+	real_parent = task->real_parent;
+	ptr = bpf_task_storage_get(&map_a, real_parent, &init_val,
+				   BPF_LOCAL_STORAGE_GET_F_CREATE);
+	if (!ptr)
+		goto out;
+	ptr = bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	if (!ptr)
+		goto out;
+	task_storage_val = *ptr;
+out:
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
+int task_acquire(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	/* acquire a reference which can be used outside rcu read lock region */
+	real_parent = bpf_task_acquire(real_parent);
+	bpf_rcu_read_unlock();
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_task_release(real_parent);
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
+int no_lock(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	/* no bpf_rcu_read_lock(), old code still works */
+	task = bpf_get_current_task_btf();
+	real_parent = task->real_parent;
+	bpf_printk("pid %u\n", real_parent->pid);
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
+int two_regions(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	/* two regions */
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	bpf_rcu_read_unlock();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry/" SYS_PREFIX "sys_getpgid")
+int non_sleepable_1(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry/" SYS_PREFIX "sys_getpgid")
+int non_sleepable_2(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	bpf_rcu_read_lock();
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_unlock();
+
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int miss_lock(void *ctx)
+{
+	struct task_struct *task;
+	struct css_set *cgroups;
+	struct cgroup *dfl_cgrp;
+
+	/* missing bpf_rcu_read_lock() */
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	(void)bpf_task_storage_get(&map_a, task, 0, 0);
+	bpf_rcu_read_unlock();
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int miss_unlock(void *ctx)
+{
+	struct task_struct *task;
+	struct css_set *cgroups;
+	struct cgroup *dfl_cgrp;
+
+	/* missing bpf_rcu_read_unlock() */
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	(void)bpf_task_storage_get(&map_a, task, 0, 0);
+	return 0;
+}
+
+SEC("?fentry/" SYS_PREFIX "sys_getpgid")
+int non_sleepable_rcu_mismatch(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	task = bpf_get_current_task_btf();
+	/* non-sleepable: missing bpf_rcu_read_unlock() in one path */
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	if (real_parent)
+		bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int inproper_sleepable_helper(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+	struct pt_regs *regs;
+	__u32 value = 0;
+	void *ptr;
+
+	task = bpf_get_current_task_btf();
+	/* sleepable helper in rcu read lock region */
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	regs = (struct pt_regs *)bpf_task_pt_regs(real_parent);
+	if (!regs) {
+		bpf_rcu_read_unlock();
+		return 0;
+	}
+
+	ptr = (void *)PT_REGS_IP(regs);
+	(void)bpf_copy_from_user_task(&value, sizeof(uint32_t), ptr, task, 0);
+	user_data = value;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?lsm.s/bpf")
+int BPF_PROG(inproper_sleepable_kfunc, int cmd, union bpf_attr *attr, unsigned int size)
+{
+	struct bpf_key *bkey;
+
+	/* sleepable kfunc in rcu read lock region */
+	bpf_rcu_read_lock();
+	bkey = bpf_lookup_user_key(key_serial, flags);
+	bpf_rcu_read_unlock();
+	if (!bkey)
+		return -1;
+	bpf_key_put(bkey);
+
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
+int nested_rcu_region(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	/* nested rcu read lock regions */
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int task_untrusted_ptr(void *ctx)
+{
+	struct task_struct *task, *last_wakee;
+
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	/* the pointer last_wakee marked as untrusted */
+	last_wakee = task->real_parent->last_wakee;
+	(void)bpf_task_storage_get(&map_a, last_wakee, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int task_incorrect_rcu_region1(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	bpf_rcu_read_unlock();
+	/* helper use of rcu ptr outside the rcu read lock region */
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+int task_incorrect_rcu_region2(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	task = bpf_get_current_task_btf();
+	/* missing bpf_rcu_read_unlock() in one path */
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	if (real_parent)
+		bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
+int cross_rcu_region(void *ctx)
+{
+	struct task_struct *task, *real_parent;
+
+	/* rcu ptr define/use in different regions */
+	task = bpf_get_current_task_btf();
+	bpf_rcu_read_lock();
+	real_parent = task->real_parent;
+	bpf_rcu_read_unlock();
+	bpf_rcu_read_lock();
+	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
  2022-11-22 19:53 ` [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock() Yonghong Song
@ 2022-11-23  0:24   ` Martin KaFai Lau
  2022-11-23  1:06     ` Yonghong Song
  0 siblings, 1 reply; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23  0:24 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf

On 11/22/22 11:53 AM, Yonghong Song wrote:
> +	if (flag & MEM_RCU) {
> +		/* Mark value register as MEM_RCU only if it is protected by
> +		 * bpf_rcu_read_lock() and the ptr reg is trusted (PTR_TRUSTED or
> +		 * ref_obj_id != 0). MEM_RCU itself can already indicate
> +		 * trustedness inside the rcu read lock region. But Mark it
> +		 * as PTR_TRUSTED as well similar to MEM_ALLOC.
> +		 */
> +		if (!env->cur_state->active_rcu_lock ||
> +		    (!(reg->type & PTR_TRUSTED) && !reg->ref_obj_id))

Can is_trusted_reg() be reused or MEM_ALLOC is not applicable here?

> +			flag &= ~MEM_RCU;
> +		else
> +			flag |= PTR_TRUSTED;
> +	} else if (reg->type & MEM_RCU) {
> +		/* ptr (reg) is marked as MEM_RCU, but value reg is not marked as MEM_RCU.
> +		 * Mark the value reg as PTR_UNTRUSTED conservatively.
> +		 */
> +		flag |= PTR_UNTRUSTED;

Should PTR_UNTRUSTED tagging be limited to ret == PTR_TO_BTF_ID instead of 
tagging SCALAR also?

[ ... ]

> @@ -11754,6 +11840,11 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)
>   		return -EINVAL;
>   	}
>   
> +	if (env->prog->aux->sleepable && env->cur_state->active_rcu_lock) {

I don't know the details about ld_abs :).  Why sleepable check is needed here?

Others lgtm.

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
  2022-11-22 19:53 ` [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock() Yonghong Song
@ 2022-11-23  0:56   ` Martin KaFai Lau
  2022-11-23  1:13     ` Yonghong Song
  0 siblings, 1 reply; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23  0:56 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf

On 11/22/22 11:53 AM, Yonghong Song wrote:
> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
> +int task_acquire(void *ctx)
> +{
> +	struct task_struct *task, *real_parent;
> +
> +	task = bpf_get_current_task_btf();
> +	bpf_rcu_read_lock();
> +	real_parent = task->real_parent;
> +	/* acquire a reference which can be used outside rcu read lock region */
> +	real_parent = bpf_task_acquire(real_parent);
Does the bpf_task_acquire() kfunc need a change to do refcount_inc_not_zero() 
and KF_RET_NULL?

Also, some more 'skip' checks in prog_tests/rcu_read_lock.c is needed for gcc. 
This is failing in gcc CI:

https://github.com/kernel-patches/bpf/actions/runs/3527747280/jobs/5917628248#step:6:5624

   ; bpf_rcu_read_lock();
   2: (85) call bpf_rcu_read_lock#26650
   ; real_parent = task->real_parent;
   3: (79) r1 = *(u64 *)(r6 +1416)       ; R1_w=ptr_task_struct(off=0,imm=0) 
R6_w=trusted_ptr_task_struct(off=0,imm=0)
   ; real_parent = bpf_task_acquire(real_parent);
   4: (85) call bpf_task_acquire#26666
   R1 must be referenced or trusted	
   processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 
peak_states 0 mark_read 0
   -- END PROG LOAD LOG --
   libbpf: prog 'task_acquire': failed to load: -22

> +	bpf_rcu_read_unlock();
> +	(void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
> +	bpf_task_release(real_parent);
> +	return 0;
> +}
> +
> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
> +int no_lock(void *ctx)
> +{
> +	struct task_struct *task, *real_parent;
> +
> +	/* no bpf_rcu_read_lock(), old code still works */
> +	task = bpf_get_current_task_btf();
> +	real_parent = task->real_parent;
> +	bpf_printk("pid %u\n", real_parent->pid);

nit. Can bpf_printk be avoided here?

Others lgtm.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
  2022-11-23  0:24   ` Martin KaFai Lau
@ 2022-11-23  1:06     ` Yonghong Song
  2022-11-23  1:19       ` Martin KaFai Lau
  0 siblings, 1 reply; 14+ messages in thread
From: Yonghong Song @ 2022-11-23  1:06 UTC (permalink / raw)
  To: Martin KaFai Lau, Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf



On 11/22/22 4:24 PM, Martin KaFai Lau wrote:
> On 11/22/22 11:53 AM, Yonghong Song wrote:
>> +    if (flag & MEM_RCU) {
>> +        /* Mark value register as MEM_RCU only if it is protected by
>> +         * bpf_rcu_read_lock() and the ptr reg is trusted 
>> (PTR_TRUSTED or
>> +         * ref_obj_id != 0). MEM_RCU itself can already indicate
>> +         * trustedness inside the rcu read lock region. But Mark it
>> +         * as PTR_TRUSTED as well similar to MEM_ALLOC.
>> +         */
>> +        if (!env->cur_state->active_rcu_lock ||
>> +            (!(reg->type & PTR_TRUSTED) && !reg->ref_obj_id))
> 
> Can is_trusted_reg() be reused or MEM_ALLOC is not applicable here?

Currently MEM_ALLOC is returned by bpf_obj_new() which takes prog btf.
currently MEM_RCU is only enabled with btf_struct_access() for
vmlinux btf. So if MEM_ALLOC is in reg->type, then MEM_RCU
should not appear.

But I guess we could still use is_trusted_reg() here since it
does cover the use case here.

> 
>> +            flag &= ~MEM_RCU;
>> +        else
>> +            flag |= PTR_TRUSTED;
>> +    } else if (reg->type & MEM_RCU) {
>> +        /* ptr (reg) is marked as MEM_RCU, but value reg is not 
>> marked as MEM_RCU.
>> +         * Mark the value reg as PTR_UNTRUSTED conservatively.
>> +         */
>> +        flag |= PTR_UNTRUSTED;
> 
> Should PTR_UNTRUSTED tagging be limited to ret == PTR_TO_BTF_ID instead 
> of tagging SCALAR also?

We should be okay here. flag is a local variable. It is used in
below function when reg_type is not SCALAR_VALUE.

static void mark_btf_ld_reg(struct bpf_verifier_env *env,
                             struct bpf_reg_state *regs, u32 regno,
                             enum bpf_reg_type reg_type,
                             struct btf *btf, u32 btf_id,
                             enum bpf_type_flag flag)
{
         if (reg_type == SCALAR_VALUE) {
                 mark_reg_unknown(env, regs, regno);
                 return;
         }
         mark_reg_known_zero(env, regs, regno);
         regs[regno].type = PTR_TO_BTF_ID | flag;
         regs[regno].btf = btf;
         regs[regno].btf_id = btf_id;
}

> 
> [ ... ]
> 
>> @@ -11754,6 +11840,11 @@ static int check_ld_abs(struct 
>> bpf_verifier_env *env, struct bpf_insn *insn)
>>           return -EINVAL;
>>       }
>> +    if (env->prog->aux->sleepable && env->cur_state->active_rcu_lock) {
> 
> I don't know the details about ld_abs :).  Why sleepable check is needed 
> here?

Do we still care about ld_abs??

Actually I added this since spin_lock excludes this. But taking a deep
look at the function convert_bpf_ld_abs, it is converted to a bunch of
bpf insns and bpf_skb_load_helper_{8,16,32}() which eventually calls
skb_copy_bits(). Checking skb_copy_bits(), it seems the function is not 
sleepable. Will remove this in the next revision.

> 
> Others lgtm.

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
  2022-11-23  0:56   ` Martin KaFai Lau
@ 2022-11-23  1:13     ` Yonghong Song
  2022-11-23  1:39       ` Martin KaFai Lau
  0 siblings, 1 reply; 14+ messages in thread
From: Yonghong Song @ 2022-11-23  1:13 UTC (permalink / raw)
  To: Martin KaFai Lau, Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf



On 11/22/22 4:56 PM, Martin KaFai Lau wrote:
> On 11/22/22 11:53 AM, Yonghong Song wrote:
>> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
>> +int task_acquire(void *ctx)
>> +{
>> +    struct task_struct *task, *real_parent;
>> +
>> +    task = bpf_get_current_task_btf();
>> +    bpf_rcu_read_lock();
>> +    real_parent = task->real_parent;
>> +    /* acquire a reference which can be used outside rcu read lock 
>> region */
>> +    real_parent = bpf_task_acquire(real_parent);
> Does the bpf_task_acquire() kfunc need a change to do 
> refcount_inc_not_zero() and KF_RET_NULL?

We have this definition in kernel:
BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)

So the argument is trusted args so, either marked as 
PTR_TRUSTED/MEM_ALLOC or have a reference acquired already, so
I guess we should be fine here.

> 
> Also, some more 'skip' checks in prog_tests/rcu_read_lock.c is needed 
> for gcc. This is failing in gcc CI:
> 
> https://github.com/kernel-patches/bpf/actions/runs/3527747280/jobs/5917628248#step:6:5624
> 
>    ; bpf_rcu_read_lock();
>    2: (85) call bpf_rcu_read_lock#26650
>    ; real_parent = task->real_parent;
>    3: (79) r1 = *(u64 *)(r6 +1416)       ; 
> R1_w=ptr_task_struct(off=0,imm=0) R6_w=trusted_ptr_task_struct(off=0,imm=0)
>    ; real_parent = bpf_task_acquire(real_parent);
>    4: (85) call bpf_task_acquire#26666
>    R1 must be referenced or trusted
>    processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 
> 0 peak_states 0 mark_read 0
>    -- END PROG LOAD LOG --
>    libbpf: prog 'task_acquire': failed to load: -22

Yes, we should skip this for gcc compiled kernel since rcu tag is not
available.

> 
>> +    bpf_rcu_read_unlock();
>> +    (void)bpf_task_storage_get(&map_a, real_parent, 0, 0);
>> +    bpf_task_release(real_parent);
>> +    return 0;
>> +}
>> +
>> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
>> +int no_lock(void *ctx)
>> +{
>> +    struct task_struct *task, *real_parent;
>> +
>> +    /* no bpf_rcu_read_lock(), old code still works */
>> +    task = bpf_get_current_task_btf();
>> +    real_parent = task->real_parent;
>> +    bpf_printk("pid %u\n", real_parent->pid);
> 
> nit. Can bpf_printk be avoided here?

I could add a target_pid comparison to prevent the issue. But
will follow your suggestion to use a different function instead
of bpf_printk.

> 
> Others lgtm.
> 

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
  2022-11-23  1:06     ` Yonghong Song
@ 2022-11-23  1:19       ` Martin KaFai Lau
  2022-11-23  1:24         ` Yonghong Song
  0 siblings, 1 reply; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23  1:19 UTC (permalink / raw)
  To: Yonghong Song, Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf

On 11/22/22 5:06 PM, Yonghong Song wrote:
> We should be okay here. flag is a local variable. It is used in
> below function when reg_type is not SCALAR_VALUE.
> 
> static void mark_btf_ld_reg(struct bpf_verifier_env *env,
>                              struct bpf_reg_state *regs, u32 regno,
>                              enum bpf_reg_type reg_type,
>                              struct btf *btf, u32 btf_id,
>                              enum bpf_type_flag flag)
> {
>          if (reg_type == SCALAR_VALUE) {
>                  mark_reg_unknown(env, regs, regno);

Ah, got it.

>>> @@ -11754,6 +11840,11 @@ static int check_ld_abs(struct bpf_verifier_env 
>>> *env, struct bpf_insn *insn)
>>>           return -EINVAL;
>>>       }
>>> +    if (env->prog->aux->sleepable && env->cur_state->active_rcu_lock) {
>>
>> I don't know the details about ld_abs :).  Why sleepable check is needed here?
> 
> Do we still care about ld_abs??
> 
> Actually I added this since spin_lock excludes this. But taking a deep

 From looking at check_ld_abs() again, I just noticed this comment:

         /* Disallow usage of BPF_LD_[ABS|IND] with reference tracking, as
          * gen_ld_abs() may terminate the program at runtime, leading to
          * reference leak.
          */

I think active_rcu_lock should be tested.  My question was more on why the 
env->prog->aux->sleepable test is also needed.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
  2022-11-23  1:19       ` Martin KaFai Lau
@ 2022-11-23  1:24         ` Yonghong Song
  0 siblings, 0 replies; 14+ messages in thread
From: Yonghong Song @ 2022-11-23  1:24 UTC (permalink / raw)
  To: Martin KaFai Lau, Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf



On 11/22/22 5:19 PM, Martin KaFai Lau wrote:
> On 11/22/22 5:06 PM, Yonghong Song wrote:
>> We should be okay here. flag is a local variable. It is used in
>> below function when reg_type is not SCALAR_VALUE.
>>
>> static void mark_btf_ld_reg(struct bpf_verifier_env *env,
>>                              struct bpf_reg_state *regs, u32 regno,
>>                              enum bpf_reg_type reg_type,
>>                              struct btf *btf, u32 btf_id,
>>                              enum bpf_type_flag flag)
>> {
>>          if (reg_type == SCALAR_VALUE) {
>>                  mark_reg_unknown(env, regs, regno);
> 
> Ah, got it.
> 
>>>> @@ -11754,6 +11840,11 @@ static int check_ld_abs(struct 
>>>> bpf_verifier_env *env, struct bpf_insn *insn)
>>>>           return -EINVAL;
>>>>       }
>>>> +    if (env->prog->aux->sleepable && 
>>>> env->cur_state->active_rcu_lock) {
>>>
>>> I don't know the details about ld_abs :).  Why sleepable check is 
>>> needed here?
>>
>> Do we still care about ld_abs??
>>
>> Actually I added this since spin_lock excludes this. But taking a deep
> 
>  From looking at check_ld_abs() again, I just noticed this comment:
> 
>          /* Disallow usage of BPF_LD_[ABS|IND] with reference tracking, as
>           * gen_ld_abs() may terminate the program at runtime, leading to
>           * reference leak.
>           */
> 
> I think active_rcu_lock should be tested.  My question was more on why 
> the env->prog->aux->sleepable test is also needed.

Will remove env->prog->aux->sleepable in the next version. It is a 
leftover missed with v8 not to focus on sleepable aspect of the lock.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
  2022-11-23  1:13     ` Yonghong Song
@ 2022-11-23  1:39       ` Martin KaFai Lau
  2022-11-23  1:52         ` Martin KaFai Lau
  0 siblings, 1 reply; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23  1:39 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf

On 11/22/22 5:13 PM, Yonghong Song wrote:
> 
> 
> On 11/22/22 4:56 PM, Martin KaFai Lau wrote:
>> On 11/22/22 11:53 AM, Yonghong Song wrote:
>>> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
>>> +int task_acquire(void *ctx)
>>> +{
>>> +    struct task_struct *task, *real_parent;
>>> +
>>> +    task = bpf_get_current_task_btf();
>>> +    bpf_rcu_read_lock();
>>> +    real_parent = task->real_parent;
>>> +    /* acquire a reference which can be used outside rcu read lock region */
>>> +    real_parent = bpf_task_acquire(real_parent);
>> Does the bpf_task_acquire() kfunc need a change to do refcount_inc_not_zero() 
>> and KF_RET_NULL?
> 
> We have this definition in kernel:
> BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
> 
> So the argument is trusted args so, either marked as PTR_TRUSTED/MEM_ALLOC or 
> have a reference acquired already, so
> I guess we should be fine here.


The verifier part is fine on {KF_TRUSTED_ARGS, PTR_TRUSTED}.

iiuc, PTR_TRUSTED means the kfunc can safely dereference the pointer because the 
ptr has not been freed yet but does not mean its refcnt > 0 and not on its way 
to be freed after the rcu gp.

If real_parent's refcnt is 0 here, bpf_task_acquire() will resurrect a task 
which is on its way to be freed and the task can be stored in a map, so a UAF.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
  2022-11-23  1:39       ` Martin KaFai Lau
@ 2022-11-23  1:52         ` Martin KaFai Lau
       [not found]           ` <SN6PR1501MB20649F820B6FFE58166817E5CA0C9@SN6PR1501MB2064.namprd15.prod.outlook.com>
  0 siblings, 1 reply; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23  1:52 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	kernel-team, Martin KaFai Lau, bpf

On 11/22/22 5:39 PM, Martin KaFai Lau wrote:
> On 11/22/22 5:13 PM, Yonghong Song wrote:
>>
>>
>> On 11/22/22 4:56 PM, Martin KaFai Lau wrote:
>>> On 11/22/22 11:53 AM, Yonghong Song wrote:
>>>> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
>>>> +int task_acquire(void *ctx)
>>>> +{
>>>> +    struct task_struct *task, *real_parent;
>>>> +
>>>> +    task = bpf_get_current_task_btf();
>>>> +    bpf_rcu_read_lock();
>>>> +    real_parent = task->real_parent;
>>>> +    /* acquire a reference which can be used outside rcu read lock region */
>>>> +    real_parent = bpf_task_acquire(real_parent);
>>> Does the bpf_task_acquire() kfunc need a change to do refcount_inc_not_zero() 
>>> and KF_RET_NULL?
>>
>> We have this definition in kernel:
>> BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
>>
>> So the argument is trusted args so, either marked as PTR_TRUSTED/MEM_ALLOC or 
>> have a reference acquired already, so
>> I guess we should be fine here.
> 
> 
> The verifier part is fine on {KF_TRUSTED_ARGS, PTR_TRUSTED}.
> 
> iiuc, PTR_TRUSTED means the kfunc can safely dereference the pointer because the 
> ptr has not been freed yet but does not mean its refcnt > 0 and not on its way 
> to be freed after the rcu gp.
> 
> If real_parent's refcnt is 0 here, bpf_task_acquire() will resurrect a task 
> which is on its way to be freed and the task can be stored in a map, so a UAF.


This could be addressed as a follow up though since it is not specific to this set.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
       [not found]           ` <SN6PR1501MB20649F820B6FFE58166817E5CA0C9@SN6PR1501MB2064.namprd15.prod.outlook.com>
@ 2022-11-23 23:13             ` Martin KaFai Lau
  0 siblings, 0 replies; 14+ messages in thread
From: Martin KaFai Lau @ 2022-11-23 23:13 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	Kernel Team, Martin KaFai Lau, bpf

On 11/22/22 9:29 PM, Yonghong Song wrote:
> 
> 
> From: Martin KaFai Lau <martin.lau@linux.dev>
> Date: Tuesday, November 22, 2022 at 5:53 PM
> To: Yonghong Song <yhs@meta.com>
> Cc: Alexei Starovoitov <ast@kernel.org>, Andrii Nakryiko <andrii@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, Kernel Team <kernel-team@meta.com>, Martin KaFai Lau <martin.lau@kernel.org>, bpf@vger.kernel.org <bpf@vger.kernel.org>
> Subject: Re: [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
> On 11/22/22 5:39 PM, Martin KaFai Lau wrote:
>> On 11/22/22 5:13 PM, Yonghong Song wrote:
>>>
>>>
>>> On 11/22/22 4:56 PM, Martin KaFai Lau wrote:
>>>> On 11/22/22 11:53 AM, Yonghong Song wrote:
>>>>> +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep")
>>>>> +int task_acquire(void *ctx)
>>>>> +{
>>>>> +    struct task_struct *task, *real_parent;
>>>>> +
>>>>> +    task = bpf_get_current_task_btf();
>>>>> +    bpf_rcu_read_lock();
>>>>> +    real_parent = task->real_parent;
>>>>> +    /* acquire a reference which can be used outside rcu read lock region */
>>>>> +    real_parent = bpf_task_acquire(real_parent);
>>>> Does the bpf_task_acquire() kfunc need a change to do refcount_inc_not_zero()
>>>> and KF_RET_NULL?
>>>
>>> We have this definition in kernel:
>>> BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
>>>
>>> So the argument is trusted args so, either marked as PTR_TRUSTED/MEM_ALLOC or
>>> have a reference acquired already, so
>>> I guess we should be fine here.
>>
>>
>> The verifier part is fine on {KF_TRUSTED_ARGS, PTR_TRUSTED}.
>>
>> iiuc, PTR_TRUSTED means the kfunc can safely dereference the pointer because the
>> ptr has not been freed yet but does not mean its refcnt > 0 and not on its way
>> to be freed after the rcu gp.
>>
>> If real_parent's refcnt is 0 here, bpf_task_acquire() will resurrect a task
>> which is on its way to be freed and the task can be stored in a map, so a UAF.
> I see. Maybe we need strong trusted vs. weak trusted variants. Strong trusted means refcnt > 0 and weak means no guarantee? Or we consider everything as week and tries to grab a reference anyway? In most if not all cases, ‘current’ should represent a strong trusted btf_id I guess.

yeah, "current" task here is fine.  current->real_parent is questionable.

imo, I think this check may be better done in runtime. The bpf_*_acquire() kfunc 
should always do refcount_inc_not_zero() + KF_RET_NULL. Otherwise, it may end up 
requiring to tag which ctx has a zero/non-zero refcnt.  eg. the 
security_sk_alloc() hook, the sk's refcnt is 0 and later the kernel does a 
refcount_set(&sk->sk_refcnt, 1).

> 
>> This could be addressed as a follow up though since it is not specific to this set.
> Right, we have the same potential problem for both task and cgroup acquire functions.
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2022-11-23 23:14 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-22 19:53 [PATCH bpf-next v8 0/4] bpf: Add bpf_rcu_read_lock() support Yonghong Song
2022-11-22 19:53 ` [PATCH bpf-next v8 1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu"))) Yonghong Song
2022-11-22 19:53 ` [PATCH bpf-next v8 2/4] bpf: Introduce might_sleep field in bpf_func_proto Yonghong Song
2022-11-22 19:53 ` [PATCH bpf-next v8 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock() Yonghong Song
2022-11-23  0:24   ` Martin KaFai Lau
2022-11-23  1:06     ` Yonghong Song
2022-11-23  1:19       ` Martin KaFai Lau
2022-11-23  1:24         ` Yonghong Song
2022-11-22 19:53 ` [PATCH bpf-next v8 4/4] selftests/bpf: Add tests for bpf_rcu_read_lock() Yonghong Song
2022-11-23  0:56   ` Martin KaFai Lau
2022-11-23  1:13     ` Yonghong Song
2022-11-23  1:39       ` Martin KaFai Lau
2022-11-23  1:52         ` Martin KaFai Lau
     [not found]           ` <SN6PR1501MB20649F820B6FFE58166817E5CA0C9@SN6PR1501MB2064.namprd15.prod.outlook.com>
2022-11-23 23:13             ` Martin KaFai Lau

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).