* [PATCH bpf-next 0/9] introduce bpf_spin_lock
@ 2019-01-16 5:08 Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
` (6 more replies)
0 siblings, 7 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
Many algorithms need to read and modify several variables atomically.
Until now it was hard to impossible to implement such algorithms in BPF.
Hence introduce support for bpf_spin_lock.
The api consists of 'struct bpf_spin_lock' that should be placed
inside hash/array/cgroup_local_storage element
and bpf_spin_lock/unlock() helper function.
Example:
struct hash_elem {
int cnt;
struct bpf_spin_lock lock;
};
struct hash_elem * val = bpf_map_lookup_elem(&hash_map, &key);
if (val) {
bpf_spin_lock(&val->lock);
val->cnt++;
bpf_spin_unlock(&val->lock);
}
and BPF_F_LOCK flag for lookup/update bpf syscall commands that
allows user space to read/write map elements under lock.
Together these primitives allow race free access to map elements
from bpf programs and from user space.
Key restriction: root only.
Key requirement: maps must be annotated with BTF.
This concept was discussed at Linux Plumbers Conference 2018.
Thank you everyone who participated and helped to iron out details
of api and implementation.
Patch 1: bpf_spin_lock support in the verifier, BTF, hash, array.
Patch 2: bpf_spin_lock in cgroup local storage.
Patches 3,4,5: tests
Patch 6: BPF_F_LOCK flag to lookup/update
Patches 7,8,9: tests
Alexei Starovoitov (9):
bpf: introduce bpf_spin_lock
bpf: add support for bpf_spin_lock to cgroup local storage
tools/bpf: sync include/uapi/linux/bpf.h
selftests/bpf: add bpf_spin_lock tests
selftests/bpf: add bpf_spin_lock C test
bpf: introduce BPF_F_LOCK flag
tools/bpf: sync uapi/bpf.h
libbpf: introduce bpf_map_lookup_elem_flags()
selftests/bpf: test for BPF_F_LOCK
include/linux/bpf.h | 39 +-
include/linux/bpf_verifier.h | 1 +
include/linux/btf.h | 1 +
include/uapi/linux/bpf.h | 8 +-
kernel/bpf/arraymap.c | 23 +-
kernel/bpf/btf.c | 37 ++
kernel/bpf/core.c | 2 +
kernel/bpf/hashtab.c | 48 ++-
kernel/bpf/helpers.c | 53 +++
kernel/bpf/local_storage.c | 16 +-
kernel/bpf/syscall.c | 48 ++-
kernel/bpf/verifier.c | 139 +++++-
net/core/filter.c | 16 +-
tools/include/uapi/linux/bpf.h | 8 +-
tools/lib/bpf/bpf.c | 13 +
tools/lib/bpf/bpf.h | 2 +
tools/lib/bpf/libbpf.map | 4 +
tools/lib/bpf/tags | 254 +++++++++++
tools/testing/selftests/bpf/Makefile | 2 +-
tools/testing/selftests/bpf/bpf_helpers.h | 4 +
tools/testing/selftests/bpf/test_map_lock.c | 66 +++
tools/testing/selftests/bpf/test_progs.c | 117 ++++-
tools/testing/selftests/bpf/test_spin_lock.c | 108 +++++
tools/testing/selftests/bpf/test_verifier.c | 429 ++++++++++++++++++-
24 files changed, 1411 insertions(+), 27 deletions(-)
create mode 100644 tools/lib/bpf/tags
create mode 100644 tools/testing/selftests/bpf/test_map_lock.c
create mode 100644 tools/testing/selftests/bpf/test_spin_lock.c
--
2.17.1
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 22:48 ` Daniel Borkmann
2019-01-17 0:16 ` Martin Lau
2019-01-16 5:08 ` [PATCH bpf-next 2/9] bpf: add support for bpf_spin_lock to cgroup local storage Alexei Starovoitov
` (5 subsequent siblings)
6 siblings, 2 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
Introduce 'struct bpf_spin_lock' and bpf_spin_lock/unlock() helpers to let
bpf program serialize access to other variables.
Example:
struct hash_elem {
int cnt;
struct bpf_spin_lock lock;
};
struct hash_elem * val = bpf_map_lookup_elem(&hash_map, &key);
if (val) {
bpf_spin_lock(&val->lock);
val->cnt++;
bpf_spin_unlock(&val->lock);
}
Restrictions and safety checks:
- bpf_spin_lock is only allowed inside HASH and ARRAY maps.
- BTF description of the map is mandatory for safety analysis.
- bpf program can take one bpf_spin_lock at a time, since two or more can
cause dead locks.
- only one 'struct bpf_spin_lock' is allowed per map element.
It drastically simplifies implementation yet allows bpf program to use
any number of bpf_spin_locks.
- when bpf_spin_lock is taken the calls (either bpf2bpf or helpers) are not allowed.
- bpf program must bpf_spin_unlock() before return.
- bpf program can access 'struct bpf_spin_lock' only via
bpf_spin_lock()/bpf_spin_unlock() helpers.
- load/store into 'struct bpf_spin_lock lock;' field is not allowed.
- to use bpf_spin_lock() helper the BTF description of map value must be
a struct and have 'struct bpf_spin_lock anyname;' field at the top level.
Nested lock inside another struct is not allowed.
- syscall map_lookup doesn't copy bpf_spin_lock field to user space.
- syscall map_update and program map_update do not update bpf_spin_lock field.
- bpf_spin_lock cannot be on the stack or inside networking packet.
bpf_spin_lock can only be inside HASH or ARRAY map value.
- bpf_spin_lock is available to root only and to all program types.
Implementation details:
- on !SMP bpf_spin_lock() becomes nop
- presence of bpf_spin_lock inside map value could have been indicated via
extra flag during map_create, but specifying it via BTF is cleaner.
It provides introspection for map key/value and reduces user coding mistakes.
Next steps:
- allow bpf_spin_lock in other map types (like cgroup local storage)
- introduce BPF_F_LOCK flag for bpf_map_update() syscall and helper
to request kernel to grab bpf_spin_lock before rewriting the value.
That will serialize access to map elements.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
include/linux/bpf.h | 37 +++++++++-
include/linux/bpf_verifier.h | 1 +
include/linux/btf.h | 1 +
include/uapi/linux/bpf.h | 7 +-
kernel/bpf/arraymap.c | 7 +-
kernel/bpf/btf.c | 37 ++++++++++
kernel/bpf/core.c | 2 +
kernel/bpf/hashtab.c | 6 +-
kernel/bpf/helpers.c | 35 +++++++++
kernel/bpf/syscall.c | 21 +++++-
kernel/bpf/verifier.c | 137 ++++++++++++++++++++++++++++++++++-
net/core/filter.c | 16 +++-
12 files changed, 293 insertions(+), 14 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e734f163bd0b..5ffa32ea7673 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -72,14 +72,15 @@ struct bpf_map {
u32 value_size;
u32 max_entries;
u32 map_flags;
- u32 pages;
+ int spin_lock_off; /* >=0 valid offset, <0 error */
u32 id;
int numa_node;
u32 btf_key_type_id;
u32 btf_value_type_id;
struct btf *btf;
+ u32 pages;
bool unpriv_array;
- /* 55 bytes hole */
+ /* 51 bytes hole */
/* The 3rd and 4th cacheline with misc members to avoid false sharing
* particularly with refcounting.
@@ -91,6 +92,34 @@ struct bpf_map {
char name[BPF_OBJ_NAME_LEN];
};
+static inline bool map_value_has_spin_lock(const struct bpf_map *map)
+{
+ return map->spin_lock_off >= 0;
+}
+
+static inline void check_and_init_map_lock(struct bpf_map *map, void *dst)
+{
+ if (likely(!map_value_has_spin_lock(map)))
+ return;
+ *(struct bpf_spin_lock *)(dst + map->spin_lock_off) =
+ (struct bpf_spin_lock){};
+}
+
+/* copy everything but bpf_spin_lock */
+static inline void copy_map_value(struct bpf_map *map, void *dst, void *src)
+{
+ if (unlikely(map_value_has_spin_lock(map))) {
+ u32 off = map->spin_lock_off;
+
+ memcpy(dst, src, off);
+ memcpy(dst + off + sizeof(struct bpf_spin_lock),
+ src + off + sizeof(struct bpf_spin_lock),
+ map->value_size - off - sizeof(struct bpf_spin_lock));
+ } else {
+ memcpy(dst, src, map->value_size);
+ }
+}
+
struct bpf_offload_dev;
struct bpf_offloaded_map;
@@ -162,6 +191,7 @@ enum bpf_arg_type {
ARG_PTR_TO_CTX, /* pointer to context */
ARG_ANYTHING, /* any (initialized) argument is ok */
ARG_PTR_TO_SOCKET, /* pointer to bpf_sock */
+ ARG_PTR_TO_SPIN_LOCK, /* pointer to bpf_spin_lock */
};
/* type of values returned from helper functions */
@@ -869,7 +899,8 @@ extern const struct bpf_func_proto bpf_msg_redirect_hash_proto;
extern const struct bpf_func_proto bpf_msg_redirect_map_proto;
extern const struct bpf_func_proto bpf_sk_redirect_hash_proto;
extern const struct bpf_func_proto bpf_sk_redirect_map_proto;
-
+extern const struct bpf_func_proto bpf_spin_lock_proto;
+extern const struct bpf_func_proto bpf_spin_unlock_proto;
extern const struct bpf_func_proto bpf_get_local_storage_proto;
/* Shared helpers among cBPF and eBPF. */
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 573cca00a0e6..ff2ff2d9e810 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -148,6 +148,7 @@ struct bpf_verifier_state {
/* call stack tracking */
struct bpf_func_state *frame[MAX_CALL_FRAMES];
u32 curframe;
+ u32 active_spin_lock;
bool speculative;
};
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 12502e25e767..455d31b55828 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -50,6 +50,7 @@ u32 btf_id(const struct btf *btf);
bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
const struct btf_member *m,
u32 expected_offset, u32 expected_size);
+int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t);
#ifdef CONFIG_BPF_SYSCALL
const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 91c43884f295..30f9dfd40f13 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -2421,7 +2421,9 @@ union bpf_attr {
FN(map_peek_elem), \
FN(msg_push_data), \
FN(msg_pop_data), \
- FN(rc_pointer_rel),
+ FN(rc_pointer_rel), \
+ FN(spin_lock), \
+ FN(spin_unlock),
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
* function eBPF program intends to call
@@ -3054,4 +3056,7 @@ struct bpf_line_info {
__u32 line_col;
};
+struct bpf_spin_lock {
+ __u32 val;
+};
#endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c
index 25632a75d630..d6d979910a2a 100644
--- a/kernel/bpf/arraymap.c
+++ b/kernel/bpf/arraymap.c
@@ -270,9 +270,10 @@ static int array_map_update_elem(struct bpf_map *map, void *key, void *value,
memcpy(this_cpu_ptr(array->pptrs[index & array->index_mask]),
value, map->value_size);
else
- memcpy(array->value +
- array->elem_size * (index & array->index_mask),
- value, map->value_size);
+ copy_map_value(map,
+ array->value +
+ array->elem_size * (index & array->index_mask),
+ value);
return 0;
}
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index a2f53642592b..ed5ae2b1f035 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -1979,6 +1979,43 @@ static void btf_struct_log(struct btf_verifier_env *env,
btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
}
+/* find 'struct bpf_spin_lock' in map value.
+ * return >= 0 offset if found
+ * and < 0 in case of error
+ */
+int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t)
+{
+ const struct btf_member *member;
+ u32 i, off = -ENOENT;
+
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_STRUCT)
+ return -EINVAL;
+
+ for_each_member(i, t, member) {
+ const struct btf_type *member_type = btf_type_by_id(btf,
+ member->type);
+ if (!btf_type_is_struct(member_type))
+ continue;
+ if (member_type->size != sizeof(struct bpf_spin_lock))
+ continue;
+ if (strcmp(__btf_name_by_offset(btf, member_type->name_off),
+ "bpf_spin_lock"))
+ continue;
+ if (off != -ENOENT)
+ /* only one 'struct bpf_spin_lock' is allowed */
+ return -E2BIG;
+ off = btf_member_bit_offset(t, member);
+ if (off % 8)
+ /* valid C code cannot generate such BTF */
+ return -EINVAL;
+ off /= 8;
+ if (off % __alignof__(struct bpf_spin_lock))
+ /* valid struct bpf_spin_lock will be 4 byte aligned */
+ return -EINVAL;
+ }
+ return off;
+}
+
static void btf_struct_seq_show(const struct btf *btf, const struct btf_type *t,
u32 type_id, void *data, u8 bits_offset,
struct seq_file *m)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index f908b9356025..497d0c4c123c 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2036,6 +2036,8 @@ const struct bpf_func_proto bpf_map_delete_elem_proto __weak;
const struct bpf_func_proto bpf_map_push_elem_proto __weak;
const struct bpf_func_proto bpf_map_pop_elem_proto __weak;
const struct bpf_func_proto bpf_map_peek_elem_proto __weak;
+const struct bpf_func_proto bpf_spin_lock_proto __weak;
+const struct bpf_func_proto bpf_spin_unlock_proto __weak;
const struct bpf_func_proto bpf_get_prandom_u32_proto __weak;
const struct bpf_func_proto bpf_get_smp_processor_id_proto __weak;
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 4b7c76765d9d..48a41bf65e1b 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -770,6 +770,8 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
l_new = ERR_PTR(-ENOMEM);
goto dec_count;
}
+ check_and_init_map_lock(&htab->map,
+ l_new->key + round_up(key_size, 8));
}
memcpy(l_new->key, key, key_size);
@@ -792,7 +794,9 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
if (!prealloc)
htab_elem_set_ptr(l_new, key_size, pptr);
} else {
- memcpy(l_new->key + round_up(key_size, 8), value, size);
+ copy_map_value(&htab->map,
+ l_new->key + round_up(key_size, 8),
+ value);
}
l_new->hash = hash;
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index a74972b07e74..591fdedae7bf 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -221,6 +221,41 @@ const struct bpf_func_proto bpf_get_current_comm_proto = {
.arg2_type = ARG_CONST_SIZE,
};
+BPF_CALL_1(bpf_spin_lock, struct bpf_spin_lock *, lock)
+{
+#if defined(CONFIG_SMP)
+ struct qspinlock *qlock = (void *)lock;
+
+ BUILD_BUG_ON(sizeof(*qlock) != sizeof(*lock));
+ queued_spin_lock(qlock);
+#endif
+ return 0;
+}
+
+const struct bpf_func_proto bpf_spin_lock_proto = {
+ .func = bpf_spin_lock,
+ .gpl_only = false,
+ .ret_type = RET_VOID,
+ .arg1_type = ARG_PTR_TO_SPIN_LOCK,
+};
+
+BPF_CALL_1(bpf_spin_unlock, struct bpf_spin_lock *, lock)
+{
+#if defined(CONFIG_SMP)
+ struct qspinlock *qlock = (void *)lock;
+
+ queued_spin_unlock(qlock);
+#endif
+ return 0;
+}
+
+const struct bpf_func_proto bpf_spin_unlock_proto = {
+ .func = bpf_spin_unlock,
+ .gpl_only = false,
+ .ret_type = RET_VOID,
+ .arg1_type = ARG_PTR_TO_SPIN_LOCK,
+};
+
#ifdef CONFIG_CGROUPS
BPF_CALL_0(bpf_get_current_cgroup_id)
{
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index b155cd17c1bd..ebf0a673cb83 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -463,7 +463,7 @@ int map_check_no_btf(const struct bpf_map *map,
return -ENOTSUPP;
}
-static int map_check_btf(const struct bpf_map *map, const struct btf *btf,
+static int map_check_btf(struct bpf_map *map, const struct btf *btf,
u32 btf_key_id, u32 btf_value_id)
{
const struct btf_type *key_type, *value_type;
@@ -478,6 +478,21 @@ static int map_check_btf(const struct bpf_map *map, const struct btf *btf,
if (!value_type || value_size != map->value_size)
return -EINVAL;
+ map->spin_lock_off = btf_find_spin_lock(btf, value_type);
+
+ if (map_value_has_spin_lock(map)) {
+ if (map->map_type != BPF_MAP_TYPE_HASH &&
+ map->map_type != BPF_MAP_TYPE_ARRAY)
+ return -ENOTSUPP;
+ if (map->spin_lock_off + sizeof(struct bpf_spin_lock) >
+ map->value_size) {
+ WARN_ONCE(1,
+ "verifier bug spin_lock_off %d value_size %d\n",
+ map->spin_lock_off, map->value_size);
+ return -EFAULT;
+ }
+ }
+
if (map->ops->map_check_btf)
ret = map->ops->map_check_btf(map, btf, key_type, value_type);
@@ -542,6 +557,8 @@ static int map_create(union bpf_attr *attr)
map->btf = btf;
map->btf_key_type_id = attr->btf_key_type_id;
map->btf_value_type_id = attr->btf_value_type_id;
+ } else {
+ map->spin_lock_off = -EINVAL;
}
err = security_bpf_map_alloc(map);
@@ -740,7 +757,7 @@ static int map_lookup_elem(union bpf_attr *attr)
err = -ENOENT;
} else {
err = 0;
- memcpy(value, ptr, value_size);
+ copy_map_value(map, value, ptr);
}
rcu_read_unlock();
}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 56674a7c3778..0f3d1fb30d7a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -213,6 +213,7 @@ struct bpf_call_arg_meta {
s64 msize_smax_value;
u64 msize_umax_value;
int ptr_id;
+ int func_id;
};
static DEFINE_MUTEX(bpf_verifier_lock);
@@ -351,6 +352,12 @@ static bool reg_is_refcounted(const struct bpf_reg_state *reg)
return type_is_refcounted(reg->type);
}
+static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg)
+{
+ return reg->type == PTR_TO_MAP_VALUE &&
+ map_value_has_spin_lock(reg->map_ptr);
+}
+
static bool reg_is_refcounted_or_null(const struct bpf_reg_state *reg)
{
return type_is_refcounted_or_null(reg->type);
@@ -712,6 +719,7 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state,
}
dst_state->speculative = src->speculative;
dst_state->curframe = src->curframe;
+ dst_state->active_spin_lock = src->active_spin_lock;
for (i = 0; i <= src->curframe; i++) {
dst = dst_state->frame[i];
if (!dst) {
@@ -1483,6 +1491,21 @@ static int check_map_access(struct bpf_verifier_env *env, u32 regno,
if (err)
verbose(env, "R%d max value is outside of the array range\n",
regno);
+
+ if (map_value_has_spin_lock(reg->map_ptr)) {
+ u32 lock = reg->map_ptr->spin_lock_off;
+
+ /* if any part of struct bpf_spin_lock can be touched by
+ * load/store reject this program
+ */
+ if ((reg->smin_value + off <= lock &&
+ lock < reg->umax_value + off + size) ||
+ (reg->smin_value + off < lock + sizeof(struct bpf_spin_lock) &&
+ lock + sizeof(struct bpf_spin_lock) <= reg->umax_value + off + size)) {
+ verbose(env, "bpf_spin_lock cannot be accessed directly by load/store\n");
+ return -EACCES;
+ }
+ }
return err;
}
@@ -2192,6 +2215,91 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
}
}
+/* Implementation details:
+ * bpf_map_lookup returns PTR_TO_MAP_VALUE_OR_NULL
+ * Two bpf_map_lookups (even with the same key) will have different reg->id.
+ * For traditional PTR_TO_MAP_VALUE the verifier clears reg->id after
+ * value_or_null->value transition, since the verifier only cares about
+ * the range of access to valid map value pointer and doesn't care about actual
+ * address of the map element.
+ * For maps with 'struct bpf_spin_lock' inside map value the verifier keeps
+ * reg->id > 0 after value_or_null->value transition. By doing so
+ * two bpf_map_lookups will be considered two different pointers that
+ * point to different bpf_spin_locks.
+ * The verifier allows taking only one bpf_spin_lock at a time to avoid
+ * dead-locks.
+ * Since only one bpf_spin_lock is allowed the checks are simpler than
+ * reg_is_refcounted() logic. The verifier needs to remember only
+ * one spin_lock instead of array of acquired_refs.
+ * cur_state->active_spin_lock remembers which map value element got locked
+ * and clears it after bpf_spin_unlock.
+ */
+static int process_spin_lock(struct bpf_verifier_env *env, int regno,
+ bool is_lock)
+{
+ struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno];
+ struct bpf_verifier_state *cur = env->cur_state;
+ bool is_const = tnum_is_const(reg->var_off);
+ struct bpf_map *map = reg->map_ptr;
+ u64 val = reg->var_off.value;
+
+ if (reg->type != PTR_TO_MAP_VALUE) {
+ verbose(env, "R%d is not a pointer to map_value\n", regno);
+ return -EINVAL;
+ }
+ if (!is_const) {
+ verbose(env,
+ "R%d doesn't have constant offset. bpf_spin_lock has to be at the constant offset\n",
+ regno);
+ return -EINVAL;
+ }
+ if (!map->btf) {
+ verbose(env,
+ "map '%s' has to have BTF in order to use bpf_spin_lock\n",
+ map->name);
+ return -EINVAL;
+ }
+ if (!map_value_has_spin_lock(map)) {
+ if (map->spin_lock_off == -E2BIG)
+ verbose(env,
+ "map '%s' has more than one 'struct bpf_spin_lock'\n",
+ map->name);
+ else if (map->spin_lock_off == -ENOENT)
+ verbose(env,
+ "map '%s' doesn't have 'struct bpf_spin_lock'\n",
+ map->name);
+ else
+ verbose(env,
+ "map '%s' is not a struct type or bpf_spin_lock is mangled\n",
+ map->name);
+ return -EINVAL;
+ }
+ if (map->spin_lock_off != val + reg->off) {
+ verbose(env, "off %lld doesn't point to 'struct bpf_spin_lock'\n",
+ val + reg->off);
+ return -EINVAL;
+ }
+ if (is_lock) {
+ if (cur->active_spin_lock) {
+ verbose(env,
+ "Locking two bpf_spin_locks are not allowed\n");
+ return -EINVAL;
+ }
+ cur->active_spin_lock = reg->id;
+ } else {
+ if (!cur->active_spin_lock) {
+ verbose(env, "bpf_spin_unlock without taking a lock\n");
+ return -EINVAL;
+ }
+ if (cur->active_spin_lock != reg->id) {
+ verbose(env, "bpf_spin_unlock of different lock\n");
+ return -EINVAL;
+ }
+ cur->active_spin_lock = 0;
+ }
+ return 0;
+}
+
static bool arg_type_is_mem_ptr(enum bpf_arg_type type)
{
return type == ARG_PTR_TO_MEM ||
@@ -2268,6 +2376,17 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno,
return -EFAULT;
}
meta->ptr_id = reg->id;
+ } else if (arg_type == ARG_PTR_TO_SPIN_LOCK) {
+ if (meta->func_id == BPF_FUNC_spin_lock) {
+ if (process_spin_lock(env, regno, true))
+ return -EACCES;
+ } else if (meta->func_id == BPF_FUNC_spin_unlock) {
+ if (process_spin_lock(env, regno, false))
+ return -EACCES;
+ } else {
+ verbose(env, "verifier internal error\n");
+ return -EFAULT;
+ }
} else if (arg_type_is_mem_ptr(arg_type)) {
expected_type = PTR_TO_STACK;
/* One exception here. In case function allows for NULL to be
@@ -2887,6 +3006,7 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn
return err;
}
+ meta.func_id = func_id;
/* check args */
err = check_func_arg(env, BPF_REG_1, fn->arg1_type, &meta);
if (err)
@@ -4344,7 +4464,8 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state,
} else if (reg->type == PTR_TO_SOCKET_OR_NULL) {
reg->type = PTR_TO_SOCKET;
}
- if (is_null || !reg_is_refcounted(reg)) {
+ if (is_null || !(reg_is_refcounted(reg) ||
+ reg_may_point_to_spin_lock(reg))) {
/* We don't need id from this point onwards anymore,
* thus we should better reset it, so that state
* pruning has chances to take effect.
@@ -5651,6 +5772,9 @@ static bool states_equal(struct bpf_verifier_env *env,
if (old->speculative && !cur->speculative)
return false;
+ if (old->active_spin_lock != cur->active_spin_lock)
+ return false;
+
/* for states to be equal callsites have to be the same
* and all frame states need to be equivalent
*/
@@ -6068,6 +6192,12 @@ static int do_check(struct bpf_verifier_env *env)
return -EINVAL;
}
+ if (env->cur_state->active_spin_lock &&
+ (insn->src_reg == BPF_PSEUDO_CALL ||
+ insn->imm != BPF_FUNC_spin_unlock)) {
+ verbose(env, "function calls are not allowed while holding a lock\n");
+ return -EINVAL;
+ }
if (insn->src_reg == BPF_PSEUDO_CALL)
err = check_func_call(env, insn, &env->insn_idx);
else
@@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
return -EINVAL;
}
+ if (env->cur_state->active_spin_lock) {
+ verbose(env, "bpf_spin_unlock is missing\n");
+ return -EINVAL;
+ }
+
if (state->curframe) {
/* exit from nested function */
env->prev_insn_idx = env->insn_idx;
diff --git a/net/core/filter.c b/net/core/filter.c
index 2b3b436ef545..24a5d874d156 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5306,10 +5306,20 @@ bpf_base_func_proto(enum bpf_func_id func_id)
return &bpf_tail_call_proto;
case BPF_FUNC_ktime_get_ns:
return &bpf_ktime_get_ns_proto;
+ default:
+ break;
+ }
+
+ if (!capable(CAP_SYS_ADMIN))
+ return NULL;
+
+ switch (func_id) {
+ case BPF_FUNC_spin_lock:
+ return &bpf_spin_lock_proto;
+ case BPF_FUNC_spin_unlock:
+ return &bpf_spin_unlock_proto;
case BPF_FUNC_trace_printk:
- if (capable(CAP_SYS_ADMIN))
- return bpf_get_trace_printk_proto();
- /* else: fall through */
+ return bpf_get_trace_printk_proto();
default:
return NULL;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 2/9] bpf: add support for bpf_spin_lock to cgroup local storage
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 3/9] tools/bpf: sync include/uapi/linux/bpf.h Alexei Starovoitov
` (4 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
Allow 'struct bpf_spin_lock' to reside inside cgroup local storage.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
kernel/bpf/local_storage.c | 2 ++
kernel/bpf/syscall.c | 3 ++-
kernel/bpf/verifier.c | 2 ++
3 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
index 07a34ef562a0..0295427f06e2 100644
--- a/kernel/bpf/local_storage.c
+++ b/kernel/bpf/local_storage.c
@@ -147,6 +147,7 @@ static int cgroup_storage_update_elem(struct bpf_map *map, void *_key,
return -ENOMEM;
memcpy(&new->data[0], value, map->value_size);
+ check_and_init_map_lock(map, new->data);
new = xchg(&storage->buf, new);
kfree_rcu(new, rcu);
@@ -483,6 +484,7 @@ struct bpf_cgroup_storage *bpf_cgroup_storage_alloc(struct bpf_prog *prog,
storage->buf = kmalloc_node(size, flags, map->numa_node);
if (!storage->buf)
goto enomem;
+ check_and_init_map_lock(map, storage->buf->data);
} else {
storage->percpu_buf = __alloc_percpu_gfp(size, 8, flags);
if (!storage->percpu_buf)
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index ebf0a673cb83..b29e6dc44650 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -482,7 +482,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
if (map_value_has_spin_lock(map)) {
if (map->map_type != BPF_MAP_TYPE_HASH &&
- map->map_type != BPF_MAP_TYPE_ARRAY)
+ map->map_type != BPF_MAP_TYPE_ARRAY &&
+ map->map_type != BPF_MAP_TYPE_CGROUP_STORAGE)
return -ENOTSUPP;
if (map->spin_lock_off + sizeof(struct bpf_spin_lock) >
map->value_size) {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 0f3d1fb30d7a..cade790c3c05 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3089,6 +3089,8 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn
regs[BPF_REG_0].map_ptr = meta.map_ptr;
if (fn->ret_type == RET_PTR_TO_MAP_VALUE) {
regs[BPF_REG_0].type = PTR_TO_MAP_VALUE;
+ if (map_value_has_spin_lock(meta.map_ptr))
+ regs[BPF_REG_0].id = ++env->id_gen;
} else {
regs[BPF_REG_0].type = PTR_TO_MAP_VALUE_OR_NULL;
regs[BPF_REG_0].id = ++env->id_gen;
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 3/9] tools/bpf: sync include/uapi/linux/bpf.h
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 2/9] bpf: add support for bpf_spin_lock to cgroup local storage Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 4/9] selftests/bpf: add bpf_spin_lock tests Alexei Starovoitov
` (3 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
sync bpf.h
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/include/uapi/linux/bpf.h | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 91c43884f295..30f9dfd40f13 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -2421,7 +2421,9 @@ union bpf_attr {
FN(map_peek_elem), \
FN(msg_push_data), \
FN(msg_pop_data), \
- FN(rc_pointer_rel),
+ FN(rc_pointer_rel), \
+ FN(spin_lock), \
+ FN(spin_unlock),
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
* function eBPF program intends to call
@@ -3054,4 +3056,7 @@ struct bpf_line_info {
__u32 line_col;
};
+struct bpf_spin_lock {
+ __u32 val;
+};
#endif /* _UAPI__LINUX_BPF_H__ */
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 4/9] selftests/bpf: add bpf_spin_lock tests
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
` (2 preceding siblings ...)
2019-01-16 5:08 ` [PATCH bpf-next 3/9] tools/bpf: sync include/uapi/linux/bpf.h Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 5/9] selftests/bpf: add bpf_spin_lock C test Alexei Starovoitov
` (2 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
add bpf_spin_lock tests to test_verifier.c that don't require
latest llvm with BTF support
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/test_verifier.c | 429 +++++++++++++++++++-
1 file changed, 428 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index 2fd90d456892..7cd1e07423d5 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -32,6 +32,7 @@
#include <linux/bpf_perf_event.h>
#include <linux/bpf.h>
#include <linux/if_ether.h>
+#include <linux/btf.h>
#include <bpf/bpf.h>
@@ -49,7 +50,7 @@
#define MAX_INSNS BPF_MAXINSNS
#define MAX_FIXUPS 8
-#define MAX_NR_MAPS 13
+#define MAX_NR_MAPS 14
#define MAX_TEST_RUNS 8
#define POINTER_VALUE 0xcafe4all
#define TEST_DATA_LEN 64
@@ -76,6 +77,7 @@ struct bpf_test {
int fixup_map_in_map[MAX_FIXUPS];
int fixup_cgroup_storage[MAX_FIXUPS];
int fixup_percpu_cgroup_storage[MAX_FIXUPS];
+ int fixup_map_spin_lock[MAX_FIXUPS];
const char *errstr;
const char *errstr_unpriv;
uint32_t retval, retval_unpriv, insn_processed;
@@ -15599,6 +15601,331 @@ static struct bpf_test tests[] = {
.result_unpriv = ACCEPT,
.result = ACCEPT,
},
+ {
+ "spin_lock: test1 success",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = ACCEPT,
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test2 direct ld/st",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "cannot be accessed directly",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test3 direct ld/st",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 1),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "cannot be accessed directly",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test4 direct ld/st",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_6, 3),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "cannot be accessed directly",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test5 call within a locked region",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_get_prandom_u32),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "calls are not allowed",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test6 missing unlock",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 0),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "unlock is missing",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test7 unlock without lock",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "without taking a lock",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test8 double lock",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "calls are not allowed",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test9 different lock",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_7, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3, 11 },
+ .result = REJECT,
+ .errstr = "unlock of different lock",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
+ {
+ "spin_lock: test10 lock in subprog without unlock",
+ .insns = {
+ BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 5),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_unlock),
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+ BPF_FUNC_spin_lock),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup_map_spin_lock = { 3 },
+ .result = REJECT,
+ .errstr = "unlock is missing",
+ .result_unpriv = REJECT,
+ .errstr_unpriv = "",
+ .prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+ },
};
static int probe_filter_length(const struct bpf_insn *fp)
@@ -15729,6 +16056,98 @@ static int create_cgroup_storage(bool percpu)
return fd;
}
+#define BTF_INFO_ENC(kind, kind_flag, vlen) \
+ ((!!(kind_flag) << 31) | ((kind) << 24) | ((vlen) & BTF_MAX_VLEN))
+#define BTF_TYPE_ENC(name, info, size_or_type) \
+ (name), (info), (size_or_type)
+#define BTF_INT_ENC(encoding, bits_offset, nr_bits) \
+ ((encoding) << 24 | (bits_offset) << 16 | (nr_bits))
+#define BTF_TYPE_INT_ENC(name, encoding, bits_offset, bits, sz) \
+ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_INT, 0, 0), sz), \
+ BTF_INT_ENC(encoding, bits_offset, bits)
+#define BTF_MEMBER_ENC(name, type, bits_offset) \
+ (name), (type), (bits_offset)
+
+struct btf_raw_data {
+ __u32 raw_types[64];
+ const char *str_sec;
+ __u32 str_sec_size;
+};
+
+/* struct bpf_spin_lock {
+ * int val;
+ * };
+ * struct val {
+ * int cnt;
+ * struct bpf_spin_lock l;
+ * };
+ */
+static const char btf_str_sec[] = "\0bpf_spin_lock\0val\0cnt\0l";
+static __u32 btf_raw_types[] = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ /* struct bpf_spin_lock */ /* [2] */
+ BTF_TYPE_ENC(1, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 4),
+ BTF_MEMBER_ENC(15, 1, 0), /* int val; */
+ /* struct val */ /* [3] */
+ BTF_TYPE_ENC(15, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 8),
+ BTF_MEMBER_ENC(19, 1, 0), /* int cnt; */
+ BTF_MEMBER_ENC(23, 2, 32),/* struct bpf_spin_lock l; */
+};
+
+static int load_btf(void)
+{
+ struct btf_header hdr = {
+ .magic = BTF_MAGIC,
+ .version = BTF_VERSION,
+ .hdr_len = sizeof(struct btf_header),
+ .type_len = sizeof(btf_raw_types),
+ .str_off = sizeof(btf_raw_types),
+ .str_len = sizeof(btf_str_sec),
+ };
+ void *ptr, *raw_btf;
+ int btf_fd;
+
+ ptr = raw_btf = malloc(sizeof(hdr) + sizeof(btf_raw_types) +
+ sizeof(btf_str_sec));
+
+ memcpy(ptr, &hdr, sizeof(hdr));
+ ptr += sizeof(hdr);
+ memcpy(ptr, btf_raw_types, hdr.type_len);
+ ptr += hdr.type_len;
+ memcpy(ptr, btf_str_sec, hdr.str_len);
+ ptr += hdr.str_len;
+
+ btf_fd = bpf_load_btf(raw_btf, ptr - raw_btf, 0, 0, 0);
+ free(raw_btf);
+ if (btf_fd < 0)
+ return -1;
+ return btf_fd;
+}
+
+static int create_map_spin_lock(void)
+{
+ struct bpf_create_map_attr attr = {
+ .name = "test_map",
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .key_size = 4,
+ .value_size = 8,
+ .max_entries = 1,
+ .btf_key_type_id = 1,
+ .btf_value_type_id = 3,
+ };
+ int fd, btf_fd;
+
+ btf_fd = load_btf();
+ if (btf_fd < 0)
+ return -1;
+ attr.btf_fd = btf_fd;
+ fd = bpf_create_map_xattr(&attr);
+ if (fd < 0)
+ printf("Failed to create map with spin_lock\n");
+ return fd;
+}
+
static char bpf_vlog[UINT_MAX >> 8];
static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type,
@@ -15747,6 +16166,7 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type,
int *fixup_map_in_map = test->fixup_map_in_map;
int *fixup_cgroup_storage = test->fixup_cgroup_storage;
int *fixup_percpu_cgroup_storage = test->fixup_percpu_cgroup_storage;
+ int *fixup_map_spin_lock = test->fixup_map_spin_lock;
if (test->fill_helper)
test->fill_helper(test);
@@ -15863,6 +16283,13 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type,
fixup_map_stacktrace++;
} while (*fixup_map_stacktrace);
}
+ if (*fixup_map_spin_lock) {
+ map_fds[13] = create_map_spin_lock();
+ do {
+ prog[*fixup_map_spin_lock].imm = map_fds[13];
+ fixup_map_spin_lock++;
+ } while (*fixup_map_spin_lock);
+ }
}
static int set_admin(bool admin)
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 5/9] selftests/bpf: add bpf_spin_lock C test
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
` (3 preceding siblings ...)
2019-01-16 5:08 ` [PATCH bpf-next 4/9] selftests/bpf: add bpf_spin_lock tests Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags() Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 9/9] selftests/bpf: test for BPF_F_LOCK Alexei Starovoitov
6 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
add bpf_spin_lock C based test that requires latest llvm with BTF support
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 2 +-
tools/testing/selftests/bpf/bpf_helpers.h | 4 +
tools/testing/selftests/bpf/test_progs.c | 43 +++++++-
tools/testing/selftests/bpf/test_spin_lock.c | 108 +++++++++++++++++++
4 files changed, 155 insertions(+), 2 deletions(-)
create mode 100644 tools/testing/selftests/bpf/test_spin_lock.c
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 70229de510f5..fcfda51406f9 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -39,7 +39,7 @@ TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test
get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o \
test_sk_lookup_kern.o test_xdp_vlan.o test_queue_map.o test_stack_map.o \
- xdp_dummy.o test_map_in_map.o
+ xdp_dummy.o test_map_in_map.o test_spin_lock.o
# Order correspond to 'make run_tests' order
TEST_PROGS := test_kmod.sh \
diff --git a/tools/testing/selftests/bpf/bpf_helpers.h b/tools/testing/selftests/bpf/bpf_helpers.h
index 6c77cf7bedce..6a0ce0f055c5 100644
--- a/tools/testing/selftests/bpf/bpf_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_helpers.h
@@ -172,6 +172,10 @@ static int (*bpf_skb_vlan_pop)(void *ctx) =
(void *) BPF_FUNC_skb_vlan_pop;
static int (*bpf_rc_pointer_rel)(void *ctx, int rel_x, int rel_y) =
(void *) BPF_FUNC_rc_pointer_rel;
+static void (*bpf_spin_lock)(struct bpf_spin_lock *lock) =
+ (void *) BPF_FUNC_spin_lock;
+static void (*bpf_spin_unlock)(struct bpf_spin_lock *lock) =
+ (void *) BPF_FUNC_spin_unlock;
/* llvm builtin functions that eBPF C program may use to
* emit BPF_LD_ABS and BPF_LD_IND instructions
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index 126fc624290d..6425e95c3f16 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -28,7 +28,7 @@ typedef __u16 __sum16;
#include <sys/wait.h>
#include <sys/types.h>
#include <fcntl.h>
-
+#include <pthread.h>
#include <linux/bpf.h>
#include <linux/err.h>
#include <bpf/bpf.h>
@@ -1882,6 +1882,46 @@ static void test_queue_stack_map(int type)
bpf_object__close(obj);
}
+static void *parallel_bpf_prog_test_run(void *arg)
+{
+ __u32 duration, retval;
+ int err, prog_fd = *(u32 *) arg;
+
+ err = bpf_prog_test_run(prog_fd, 10000, &pkt_v4, sizeof(pkt_v4),
+ NULL, NULL, &retval, &duration);
+ CHECK(err || retval, "",
+ "err %d errno %d retval %d duration %d\n",
+ err, errno, retval, duration);
+ pthread_exit(arg);
+}
+
+static void test_spin_lock(void)
+{
+ const char *file = "./test_spin_lock.o";
+ pthread_t thread_id[4];
+ struct bpf_object *obj;
+ int prog_fd;
+ int err = 0, i;
+ void *ret;
+
+ err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
+ if (err) {
+ printf("test_spin_lock:bpf_prog_load errno %d\n", errno);
+ goto close_prog;
+ }
+ for (i = 0; i < 4; i++)
+ assert(pthread_create(&thread_id[i], NULL,
+ ¶llel_bpf_prog_test_run, &prog_fd) == 0);
+ for (i = 0; i < 4; i++)
+ assert(pthread_join(thread_id[i], &ret) == 0 &&
+ ret == (void *)&prog_fd);
+ goto close_prog_noerr;
+close_prog:
+ error_cnt++;
+close_prog_noerr:
+ bpf_object__close(obj);
+}
+
int main(void)
{
srand(time(NULL));
@@ -1909,6 +1949,7 @@ int main(void)
test_reference_tracking();
test_queue_stack_map(QUEUE);
test_queue_stack_map(STACK);
+ test_spin_lock();
printf("Summary: %d PASSED, %d FAILED\n", pass_cnt, error_cnt);
return error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
diff --git a/tools/testing/selftests/bpf/test_spin_lock.c b/tools/testing/selftests/bpf/test_spin_lock.c
new file mode 100644
index 000000000000..40f904312090
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_spin_lock.c
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 Facebook
+#include <linux/bpf.h>
+#include <linux/version.h>
+#include "bpf_helpers.h"
+
+struct hmap_elem {
+ volatile int cnt;
+ struct bpf_spin_lock lock;
+ int test_padding;
+};
+
+struct bpf_map_def SEC("maps") hmap = {
+ .type = BPF_MAP_TYPE_HASH,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct hmap_elem),
+ .max_entries = 1,
+};
+
+BPF_ANNOTATE_KV_PAIR(hmap, int, struct hmap_elem);
+
+
+struct cls_elem {
+ struct bpf_spin_lock lock;
+ volatile int cnt;
+};
+
+struct bpf_map_def SEC("maps") cls_map = {
+ .type = BPF_MAP_TYPE_CGROUP_STORAGE,
+ .key_size = sizeof(struct bpf_cgroup_storage_key),
+ .value_size = sizeof(struct cls_elem),
+};
+
+BPF_ANNOTATE_KV_PAIR(cls_map, struct bpf_cgroup_storage_key,
+ struct cls_elem);
+
+struct bpf_vqueue {
+ struct bpf_spin_lock lock;
+ /* 4 byte hole */
+ unsigned long long lasttime;
+ int credit;
+ unsigned int rate;
+};
+
+struct bpf_map_def SEC("maps") vqueue = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct bpf_vqueue),
+ .max_entries = 1,
+};
+
+BPF_ANNOTATE_KV_PAIR(vqueue, int, struct bpf_vqueue);
+#define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20)
+
+SEC("spin_lock_demo")
+int bpf_sping_lock_test(struct __sk_buff *skb)
+{
+ volatile int credit = 0, max_credit = 100, pkt_len = 64;
+ struct hmap_elem zero = {}, *val;
+ unsigned long long curtime;
+ struct bpf_vqueue *q;
+ struct cls_elem *cls;
+ int key = 0;
+ int err = 0;
+
+ val = bpf_map_lookup_elem(&hmap, &key);
+ if (!val) {
+ bpf_map_update_elem(&hmap, &key, &zero, 0);
+ val = bpf_map_lookup_elem(&hmap, &key);
+ if (!val) {
+ err = 1;
+ goto err;
+ }
+ }
+ /* spin_lock in hash map run time test */
+ bpf_spin_lock(&val->lock);
+ if (val->cnt)
+ val->cnt--;
+ else
+ val->cnt++;
+ if (val->cnt != 0 && val->cnt != 1)
+ err = 1;
+ bpf_spin_unlock(&val->lock);
+
+ /* spin_lock in array. virtual queue demo */
+ q = bpf_map_lookup_elem(&vqueue, &key);
+ if (!q)
+ goto err;
+ curtime = bpf_ktime_get_ns();
+ bpf_spin_lock(&q->lock);
+ q->credit += CREDIT_PER_NS(curtime - q->lasttime, q->rate);
+ q->lasttime = curtime;
+ if (q->credit > max_credit)
+ q->credit = max_credit;
+ q->credit -= pkt_len;
+ credit = q->credit;
+ bpf_spin_unlock(&q->lock);
+
+ /* spin_lock in cgroup local storage */
+ cls = bpf_get_local_storage(&cls_map, 0);
+ bpf_spin_lock(&cls->lock);
+ cls->cnt++;
+ bpf_spin_unlock(&cls->lock);
+
+err:
+ return err;
+}
+char _license[] SEC("license") = "GPL";
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags()
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
` (4 preceding siblings ...)
2019-01-16 5:08 ` [PATCH bpf-next 5/9] selftests/bpf: add bpf_spin_lock C test Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
2019-01-16 6:18 ` Yonghong Song
2019-01-16 5:08 ` [PATCH bpf-next 9/9] selftests/bpf: test for BPF_F_LOCK Alexei Starovoitov
6 siblings, 1 reply; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
Introduce
int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags)
helper to lookup array/hash/cgroup_local_storage elements with BPF_F_LOCK flag.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/lib/bpf/bpf.c | 13 ++
tools/lib/bpf/bpf.h | 2 +
tools/lib/bpf/libbpf.map | 4 +
tools/lib/bpf/tags | 254 +++++++++++++++++++++++++++++++++++++++
4 files changed, 273 insertions(+)
create mode 100644 tools/lib/bpf/tags
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 3caaa3428774..d55a77a05d5f 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -357,6 +357,19 @@ int bpf_map_lookup_elem(int fd, const void *key, void *value)
return sys_bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
}
+int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags)
+{
+ union bpf_attr attr;
+
+ bzero(&attr, sizeof(attr));
+ attr.map_fd = fd;
+ attr.key = ptr_to_u64(key);
+ attr.value = ptr_to_u64(value);
+ attr.flags = flags;
+
+ return sys_bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
+}
+
int bpf_map_lookup_and_delete_elem(int fd, const void *key, void *value)
{
union bpf_attr attr;
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 8f09de482839..ed09eed2dc3b 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -110,6 +110,8 @@ LIBBPF_API int bpf_map_update_elem(int fd, const void *key, const void *value,
__u64 flags);
LIBBPF_API int bpf_map_lookup_elem(int fd, const void *key, void *value);
+LIBBPF_API int bpf_map_lookup_elem_flags(int fd, const void *key, void *value,
+ __u64 flags);
LIBBPF_API int bpf_map_lookup_and_delete_elem(int fd, const void *key,
void *value);
LIBBPF_API int bpf_map_delete_elem(int fd, const void *key);
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index cd02cd4e2cc3..ca5155409a15 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -124,3 +124,7 @@ LIBBPF_0.0.1 {
local:
*;
};
+LIBBPF_0.0.2 {
+ global:
+ bpf_map_lookup_elem_flags;
+} LIBBPF_0.0.1;
diff --git a/tools/lib/bpf/tags b/tools/lib/bpf/tags
new file mode 100644
index 000000000000..be30548a028e
--- /dev/null
+++ b/tools/lib/bpf/tags
@@ -0,0 +1,254 @@
+!_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to lines/
+!_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/
+!_TAG_PROGRAM_AUTHOR Darren Hiebert /dhiebert@users.sourceforge.net/
+!_TAG_PROGRAM_NAME Exuberant Ctags //
+!_TAG_PROGRAM_URL http://ctags.sourceforge.net /official site/
+!_TAG_PROGRAM_VERSION 5.8 //
+BPF_EXTRAVERSION Makefile /^BPF_EXTRAVERSION = 1$/;" m
+BPF_FS_MAGIC libbpf.c 52;" d file:
+BPF_IN Makefile /^BPF_IN := $(OUTPUT)libbpf-in.o$/;" m
+BPF_LOG_BUF_SIZE bpf.h 43;" d
+BPF_PATCHLEVEL Makefile /^BPF_PATCHLEVEL = 0$/;" m
+BPF_PROG_TYPE_FNS libbpf.c 1702;" d file:
+BPF_VERSION Makefile /^BPF_VERSION = 0$/;" m
+CFLAGS Makefile /^ CFLAGS := $(EXTRA_CFLAGS)$/;" m
+CFLAGS Makefile /^ CFLAGS := -g -Wall$/;" m
+CHECK_ERR libbpf.c 140;" d file:
+CMD_TARGETS Makefile /^CMD_TARGETS = $(LIB_FILE)$/;" m
+DESTDIR Makefile /^DESTDIR ?=$/;" m
+DESTDIR_SQ Makefile /^DESTDIR_SQ = '$(subst ','\\'',$(DESTDIR))'$/;" m
+EM_BPF libbpf.c 48;" d file:
+ERRCODE_OFFSET libbpf.c 95;" d file:
+ERRNO_OFFSET libbpf.c 94;" d file:
+EXTRAVERSION Makefile /^EXTRAVERSION = $(BPF_EXTRAVERSION)$/;" m
+FEATURE_CHECK_CFLAGS-bpf Makefile /^FEATURE_CHECK_CFLAGS-bpf = $(INCLUDES)$/;" m
+FEATURE_DISPLAY Makefile /^FEATURE_DISPLAY = libelf bpf$/;" m
+FEATURE_TESTS Makefile /^FEATURE_TESTS = libelf libelf-getphdrnum libelf-mmap bpf$/;" m
+FEATURE_USER Makefile /^FEATURE_USER = .libbpf$/;" m
+INCLUDES Makefile /^INCLUDES = -I. -I$(srctree)\/tools\/include -I$(srctree)\/tools\/arch\/$(ARCH)\/include\/uapi -I$(srctree)\/tools\/include\/uapi$/;" m
+INSTALL Makefile /^INSTALL = install$/;" m
+LIBBPF_ELF_C_READ_MMAP libbpf.c 162;" d file:
+LIBBPF_ELF_C_READ_MMAP libbpf.c 164;" d file:
+LIBBPF_ERRNO__ENDIAN libbpf.h /^ LIBBPF_ERRNO__ENDIAN, \/* Endian mismatch *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__FORMAT libbpf.h /^ LIBBPF_ERRNO__FORMAT, \/* BPF object format invalid *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__INTERNAL libbpf.h /^ LIBBPF_ERRNO__INTERNAL, \/* Internal error in libbpf *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__KVER libbpf.h /^ LIBBPF_ERRNO__KVER, \/* Incorrect kernel version *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__KVERSION libbpf.h /^ LIBBPF_ERRNO__KVERSION, \/* Incorrect or no 'version' section *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__LIBELF libbpf.h /^ LIBBPF_ERRNO__LIBELF = __LIBBPF_ERRNO__START,$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__LOAD libbpf.h /^ LIBBPF_ERRNO__LOAD, \/* Load program failure for unknown reason *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__PROG2BIG libbpf.h /^ LIBBPF_ERRNO__PROG2BIG, \/* Program too big *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__PROGTYPE libbpf.h /^ LIBBPF_ERRNO__PROGTYPE, \/* Kernel doesn't support this program type *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__RELOC libbpf.h /^ LIBBPF_ERRNO__RELOC, \/* Relocation failed *\/$/;" e enum:libbpf_errno
+LIBBPF_ERRNO__VERIFY libbpf.h /^ LIBBPF_ERRNO__VERIFY, \/* Kernel verifier blocks program loading *\/$/;" e enum:libbpf_errno
+LIBBPF_VERSION Makefile /^LIBBPF_VERSION = $(BPF_VERSION).$(BPF_PATCHLEVEL).$(BPF_EXTRAVERSION)$/;" m
+LIB_FILE Makefile /^LIB_FILE := $(addprefix $(OUTPUT),$(LIB_FILE))$/;" m
+LIB_FILE Makefile /^LIB_FILE = libbpf.a libbpf.so$/;" m
+MAKEOVERRIDES Makefile /^MAKEOVERRIDES=$/;" m
+N Makefile /^N =$/;" m
+NON_CHECK_FEAT_TARGETS Makefile /^NON_CHECK_FEAT_TARGETS := clean TAGS tags cscope help$/;" m
+NR_ERRNO libbpf.c 96;" d file:
+OBJ Makefile /^OBJ = $@$/;" m
+PATCHLEVEL Makefile /^PATCHLEVEL = $(BPF_PATCHLEVEL)$/;" m
+Q Makefile /^ Q = @$/;" m
+Q Makefile /^ Q =$/;" m
+STRERR_BUFSIZE libbpf.c 92;" d file:
+TARGETS Makefile /^TARGETS = $(CMD_TARGETS)$/;" m
+VERBOSE Makefile /^ VERBOSE = $(V)$/;" m
+VERBOSE Makefile /^ VERBOSE = 0$/;" m
+VERSION Makefile /^VERSION = $(BPF_VERSION)$/;" m
+__BPF_BPF_H bpf.h 22;" d
+__BPF_LIBBPF_H libbpf.h 22;" d
+__LIBBPF_ERRNO__END libbpf.h /^ __LIBBPF_ERRNO__END,$/;" e enum:libbpf_errno
+__LIBBPF_ERRNO__START libbpf.h /^ __LIBBPF_ERRNO__START = 4000,$/;" e enum:libbpf_errno
+__NR_bpf bpf.c 35;" d file:
+__NR_bpf bpf.c 37;" d file:
+__NR_bpf bpf.c 39;" d file:
+__NR_bpf bpf.c 41;" d file:
+__NR_bpf bpf.c 43;" d file:
+__base_pr libbpf.c /^static int __base_pr(const char *format, ...)$/;" f file:
+__bpf_object__open libbpf.c /^__bpf_object__open(const char *path, void *obj_buf, size_t obj_buf_sz)$/;" f file:
+__pr libbpf.c 73;" d file:
+__pr_debug libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_debug;$/;" v
+__pr_info libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_info = __base_pr;$/;" v
+__pr_warning libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_warning = __base_pr;$/;" v
+__printf libbpf.c 55;" d file:
+allow-override Makefile /^define allow-override$/;" m
+bpf_create_map bpf.c /^int bpf_create_map(enum bpf_map_type map_type, int key_size,$/;" f
+bpf_create_map_in_map bpf.c /^int bpf_create_map_in_map(enum bpf_map_type map_type, const char *name,$/;" f
+bpf_create_map_in_map_node bpf.c /^int bpf_create_map_in_map_node(enum bpf_map_type map_type, const char *name,$/;" f
+bpf_create_map_name bpf.c /^int bpf_create_map_name(enum bpf_map_type map_type, const char *name,$/;" f
+bpf_create_map_node bpf.c /^int bpf_create_map_node(enum bpf_map_type map_type, const char *name,$/;" f
+bpf_load_program bpf.c /^int bpf_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,$/;" f
+bpf_load_program_name bpf.c /^int bpf_load_program_name(enum bpf_prog_type type, const char *name,$/;" f
+bpf_map libbpf.c /^struct bpf_map {$/;" s file:
+bpf_map__def libbpf.c /^const struct bpf_map_def *bpf_map__def(struct bpf_map *map)$/;" f
+bpf_map__fd libbpf.c /^int bpf_map__fd(struct bpf_map *map)$/;" f
+bpf_map__for_each libbpf.h 230;" d
+bpf_map__name libbpf.c /^const char *bpf_map__name(struct bpf_map *map)$/;" f
+bpf_map__next libbpf.c /^bpf_map__next(struct bpf_map *prev, struct bpf_object *obj)$/;" f
+bpf_map__pin libbpf.c /^int bpf_map__pin(struct bpf_map *map, const char *path)$/;" f
+bpf_map__priv libbpf.c /^void *bpf_map__priv(struct bpf_map *map)$/;" f
+bpf_map__set_priv libbpf.c /^int bpf_map__set_priv(struct bpf_map *map, void *priv,$/;" f
+bpf_map_clear_priv_t libbpf.h /^typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);$/;" t
+bpf_map_def libbpf.h /^struct bpf_map_def {$/;" s
+bpf_map_delete_elem bpf.c /^int bpf_map_delete_elem(int fd, const void *key)$/;" f
+bpf_map_get_fd_by_id bpf.c /^int bpf_map_get_fd_by_id(__u32 id)$/;" f
+bpf_map_get_next_id bpf.c /^int bpf_map_get_next_id(__u32 start_id, __u32 *next_id)$/;" f
+bpf_map_get_next_key bpf.c /^int bpf_map_get_next_key(int fd, const void *key, void *next_key)$/;" f
+bpf_map_lookup_elem bpf.c /^int bpf_map_lookup_elem(int fd, const void *key, void *value)$/;" f
+bpf_map_update_elem bpf.c /^int bpf_map_update_elem(int fd, const void *key, const void *value,$/;" f
+bpf_obj_get bpf.c /^int bpf_obj_get(const char *pathname)$/;" f
+bpf_obj_get_info_by_fd bpf.c /^int bpf_obj_get_info_by_fd(int prog_fd, void *info, __u32 *info_len)$/;" f
+bpf_obj_pin bpf.c /^int bpf_obj_pin(int fd, const char *pathname)$/;" f
+bpf_object libbpf.c /^struct bpf_object {$/;" s file:
+bpf_object__add_program libbpf.c /^bpf_object__add_program(struct bpf_object *obj, void *data, size_t size,$/;" f file:
+bpf_object__check_endianness libbpf.c /^bpf_object__check_endianness(struct bpf_object *obj)$/;" f file:
+bpf_object__close libbpf.c /^void bpf_object__close(struct bpf_object *obj)$/;" f
+bpf_object__collect_reloc libbpf.c /^static int bpf_object__collect_reloc(struct bpf_object *obj)$/;" f file:
+bpf_object__create_maps libbpf.c /^bpf_object__create_maps(struct bpf_object *obj)$/;" f file:
+bpf_object__elf_collect libbpf.c /^static int bpf_object__elf_collect(struct bpf_object *obj)$/;" f file:
+bpf_object__elf_finish libbpf.c /^static void bpf_object__elf_finish(struct bpf_object *obj)$/;" f file:
+bpf_object__elf_init libbpf.c /^static int bpf_object__elf_init(struct bpf_object *obj)$/;" f file:
+bpf_object__find_map_by_name libbpf.c /^bpf_object__find_map_by_name(struct bpf_object *obj, const char *name)$/;" f
+bpf_object__find_map_by_offset libbpf.c /^bpf_object__find_map_by_offset(struct bpf_object *obj, size_t offset)$/;" f
+bpf_object__find_prog_by_idx libbpf.c /^bpf_object__find_prog_by_idx(struct bpf_object *obj, int idx)$/;" f file:
+bpf_object__for_each_program libbpf.h 95;" d
+bpf_object__for_each_safe libbpf.h 79;" d
+bpf_object__init_kversion libbpf.c /^bpf_object__init_kversion(struct bpf_object *obj,$/;" f file:
+bpf_object__init_license libbpf.c /^bpf_object__init_license(struct bpf_object *obj,$/;" f file:
+bpf_object__init_maps libbpf.c /^bpf_object__init_maps(struct bpf_object *obj)$/;" f file:
+bpf_object__init_prog_names libbpf.c /^bpf_object__init_prog_names(struct bpf_object *obj)$/;" f file:
+bpf_object__kversion libbpf.c /^unsigned int bpf_object__kversion(struct bpf_object *obj)$/;" f
+bpf_object__load libbpf.c /^int bpf_object__load(struct bpf_object *obj)$/;" f
+bpf_object__load_progs libbpf.c /^bpf_object__load_progs(struct bpf_object *obj)$/;" f file:
+bpf_object__name libbpf.c /^const char *bpf_object__name(struct bpf_object *obj)$/;" f
+bpf_object__new libbpf.c /^static struct bpf_object *bpf_object__new(const char *path,$/;" f file:
+bpf_object__next libbpf.c /^bpf_object__next(struct bpf_object *prev)$/;" f
+bpf_object__open libbpf.c /^struct bpf_object *bpf_object__open(const char *path)$/;" f
+bpf_object__open_buffer libbpf.c /^struct bpf_object *bpf_object__open_buffer(void *obj_buf,$/;" f
+bpf_object__pin libbpf.c /^int bpf_object__pin(struct bpf_object *obj, const char *path)$/;" f
+bpf_object__priv libbpf.c /^void *bpf_object__priv(struct bpf_object *obj)$/;" f
+bpf_object__relocate libbpf.c /^bpf_object__relocate(struct bpf_object *obj)$/;" f file:
+bpf_object__set_priv libbpf.c /^int bpf_object__set_priv(struct bpf_object *obj, void *priv,$/;" f
+bpf_object__unload libbpf.c /^int bpf_object__unload(struct bpf_object *obj)$/;" f
+bpf_object__validate libbpf.c /^static int bpf_object__validate(struct bpf_object *obj)$/;" f file:
+bpf_object_clear_priv_t libbpf.h /^typedef void (*bpf_object_clear_priv_t)(struct bpf_object *, void *);$/;" t
+bpf_prog_attach bpf.c /^int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type,$/;" f
+bpf_prog_detach bpf.c /^int bpf_prog_detach(int target_fd, enum bpf_attach_type type)$/;" f
+bpf_prog_detach2 bpf.c /^int bpf_prog_detach2(int prog_fd, int target_fd, enum bpf_attach_type type)$/;" f
+bpf_prog_get_fd_by_id bpf.c /^int bpf_prog_get_fd_by_id(__u32 id)$/;" f
+bpf_prog_get_next_id bpf.c /^int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id)$/;" f
+bpf_prog_load libbpf.c /^int bpf_prog_load(const char *file, enum bpf_prog_type type,$/;" f
+bpf_prog_prep_result libbpf.h /^struct bpf_prog_prep_result {$/;" s
+bpf_prog_query bpf.c /^int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags,$/;" f
+bpf_prog_test_run bpf.c /^int bpf_prog_test_run(int prog_fd, int repeat, void *data, __u32 size,$/;" f
+bpf_program libbpf.c /^struct bpf_program {$/;" s file:
+bpf_program__collect_reloc libbpf.c /^bpf_program__collect_reloc(struct bpf_program *prog,$/;" f file:
+bpf_program__exit libbpf.c /^static void bpf_program__exit(struct bpf_program *prog)$/;" f file:
+bpf_program__fd libbpf.c /^int bpf_program__fd(struct bpf_program *prog)$/;" f
+bpf_program__init libbpf.c /^bpf_program__init(void *data, size_t size, char *section_name, int idx,$/;" f file:
+bpf_program__is_type libbpf.c /^static bool bpf_program__is_type(struct bpf_program *prog,$/;" f file:
+bpf_program__load libbpf.c /^bpf_program__load(struct bpf_program *prog,$/;" f file:
+bpf_program__next libbpf.c /^bpf_program__next(struct bpf_program *prev, struct bpf_object *obj)$/;" f
+bpf_program__nth_fd libbpf.c /^int bpf_program__nth_fd(struct bpf_program *prog, int n)$/;" f
+bpf_program__pin libbpf.c /^int bpf_program__pin(struct bpf_program *prog, const char *path)$/;" f
+bpf_program__pin_instance libbpf.c /^int bpf_program__pin_instance(struct bpf_program *prog, const char *path,$/;" f
+bpf_program__priv libbpf.c /^void *bpf_program__priv(struct bpf_program *prog)$/;" f
+bpf_program__relocate libbpf.c /^bpf_program__relocate(struct bpf_program *prog, struct bpf_object *obj)$/;" f file:
+bpf_program__set_prep libbpf.c /^int bpf_program__set_prep(struct bpf_program *prog, int nr_instances,$/;" f
+bpf_program__set_priv libbpf.c /^int bpf_program__set_priv(struct bpf_program *prog, void *priv,$/;" f
+bpf_program__set_type libbpf.c /^void bpf_program__set_type(struct bpf_program *prog, enum bpf_prog_type type)$/;" f
+bpf_program__title libbpf.c /^const char *bpf_program__title(struct bpf_program *prog, bool needs_copy)$/;" f
+bpf_program__unload libbpf.c /^static void bpf_program__unload(struct bpf_program *prog)$/;" f file:
+bpf_program_clear_priv_t libbpf.h /^typedef void (*bpf_program_clear_priv_t)(struct bpf_program *,$/;" t
+bpf_program_prep_t libbpf.h /^typedef int (*bpf_program_prep_t)(struct bpf_program *prog, int n,$/;" t
+bpf_verify_program bpf.c /^int bpf_verify_program(enum bpf_prog_type type, const struct bpf_insn *insns,$/;" f
+check_feat Makefile /^ check_feat := 0$/;" m
+check_feat Makefile /^check_feat := 1$/;" m
+check_path libbpf.c /^static int check_path(const char *path)$/;" f file:
+clear_priv libbpf.c /^ bpf_map_clear_priv_t clear_priv;$/;" m struct:bpf_map file:
+clear_priv libbpf.c /^ bpf_object_clear_priv_t clear_priv;$/;" m struct:bpf_object file:
+clear_priv libbpf.c /^ bpf_program_clear_priv_t clear_priv;$/;" m struct:bpf_program file:
+compare_bpf_map libbpf.c /^static int compare_bpf_map(const void *_a, const void *_b)$/;" f file:
+data libbpf.c /^ Elf_Data *data;$/;" m struct:bpf_object::__anon3::__anon4 file:
+def libbpf.c /^ struct bpf_map_def def;$/;" m struct:bpf_map typeref:struct:bpf_map::bpf_map_def file:
+do_install Makefile /^define do_install$/;" m
+efile libbpf.c /^ } efile;$/;" m struct:bpf_object typeref:struct:bpf_object::__anon3 file:
+ehdr libbpf.c /^ GElf_Ehdr ehdr;$/;" m struct:bpf_object::__anon3 file:
+elf libbpf.c /^ Elf *elf;$/;" m struct:bpf_object::__anon3 file:
+fd libbpf.c /^ int fd;$/;" m struct:bpf_object::__anon3 file:
+fd libbpf.c /^ int fd;$/;" m struct:bpf_map file:
+fds libbpf.c /^ int *fds;$/;" m struct:bpf_program::__anon2 file:
+idx libbpf.c /^ int idx;$/;" m struct:bpf_program file:
+insn_idx libbpf.c /^ int insn_idx;$/;" m struct:bpf_program::__anon1 file:
+insns libbpf.c /^ struct bpf_insn *insns;$/;" m struct:bpf_program typeref:struct:bpf_program::bpf_insn file:
+insns_cnt libbpf.c /^ size_t insns_cnt;$/;" m struct:bpf_program file:
+instances libbpf.c /^ } instances;$/;" m struct:bpf_program typeref:struct:bpf_program::__anon2 file:
+kern_version libbpf.c /^ u32 kern_version;$/;" m struct:bpf_object file:
+key_size libbpf.h /^ unsigned int key_size;$/;" m struct:bpf_map_def
+libbpf_errno libbpf.h /^enum libbpf_errno {$/;" g
+libbpf_get_error libbpf.c /^long libbpf_get_error(const void *ptr)$/;" f
+libbpf_print_fn_t libbpf.h /^typedef int (*libbpf_print_fn_t)(const char *, ...)$/;" t
+libbpf_set_print libbpf.c /^void libbpf_set_print(libbpf_print_fn_t warn,$/;" f
+libbpf_strerror libbpf.c /^int libbpf_strerror(int err, char *buf, size_t size)$/;" f
+libbpf_strerror_table libbpf.c /^static const char *libbpf_strerror_table[NR_ERRNO] = {$/;" v file:
+libdir Makefile /^libdir = $(prefix)\/$(libdir_relative)$/;" m
+libdir_SQ Makefile /^libdir_SQ = $(subst ','\\'',$(libdir))$/;" m
+libdir_relative Makefile /^ libdir_relative = lib$/;" m
+libdir_relative Makefile /^ libdir_relative = lib64$/;" m
+libdir_relative_SQ Makefile /^libdir_relative_SQ = $(subst ','\\'',$(libdir_relative))$/;" m
+license libbpf.c /^ char license[64];$/;" m struct:bpf_object file:
+list libbpf.c /^ struct list_head list;$/;" m struct:bpf_object typeref:struct:bpf_object::list_head file:
+load_program libbpf.c /^load_program(enum bpf_prog_type type, const char *name, struct bpf_insn *insns,$/;" f file:
+loaded libbpf.c /^ bool loaded;$/;" m struct:bpf_object file:
+make_dir libbpf.c /^static int make_dir(const char *path)$/;" f file:
+man_dir Makefile /^man_dir = $(prefix)\/share\/man$/;" m
+man_dir_SQ Makefile /^man_dir_SQ = '$(subst ','\\'',$(man_dir))'$/;" m
+map_flags libbpf.h /^ unsigned int map_flags;$/;" m struct:bpf_map_def
+map_idx libbpf.c /^ int map_idx;$/;" m struct:bpf_program::__anon1 file:
+maps libbpf.c /^ struct bpf_map *maps;$/;" m struct:bpf_object typeref:struct:bpf_object::bpf_map file:
+maps_shndx libbpf.c /^ int maps_shndx;$/;" m struct:bpf_object::__anon3 file:
+max_entries libbpf.h /^ unsigned int max_entries;$/;" m struct:bpf_map_def
+min bpf.c 49;" d file:
+name libbpf.c /^ char *name;$/;" m struct:bpf_map file:
+name libbpf.c /^ char *name;$/;" m struct:bpf_program file:
+new_insn_cnt libbpf.h /^ int new_insn_cnt;$/;" m struct:bpf_prog_prep_result
+new_insn_ptr libbpf.h /^ struct bpf_insn *new_insn_ptr;$/;" m struct:bpf_prog_prep_result typeref:struct:bpf_prog_prep_result::bpf_insn
+nr libbpf.c /^ int nr;$/;" m struct:bpf_program::__anon2 file:
+nr_maps libbpf.c /^ size_t nr_maps;$/;" m struct:bpf_object file:
+nr_programs libbpf.c /^ size_t nr_programs;$/;" m struct:bpf_object file:
+nr_reloc libbpf.c /^ int nr_reloc;$/;" m struct:bpf_object::__anon3 file:
+nr_reloc libbpf.c /^ int nr_reloc;$/;" m struct:bpf_program file:
+obj libbpf.c /^ struct bpf_object *obj;$/;" m struct:bpf_program typeref:struct:bpf_program::bpf_object file:
+obj_buf libbpf.c /^ void *obj_buf;$/;" m struct:bpf_object::__anon3 file:
+obj_buf_sz libbpf.c /^ size_t obj_buf_sz;$/;" m struct:bpf_object::__anon3 file:
+obj_elf_valid libbpf.c 250;" d file:
+offset libbpf.c /^ size_t offset;$/;" m struct:bpf_map file:
+path libbpf.c /^ char path[];$/;" m struct:bpf_object file:
+pfd libbpf.h /^ int *pfd;$/;" m struct:bpf_prog_prep_result
+plugin_dir_SQ Makefile /^plugin_dir_SQ = $(subst ','\\'',$(plugin_dir))$/;" m
+pr_debug libbpf.c 81;" d file:
+pr_info libbpf.c 80;" d file:
+pr_warning libbpf.c 79;" d file:
+prefix Makefile /^prefix ?= \/usr\/local$/;" m
+preprocessor libbpf.c /^ bpf_program_prep_t preprocessor;$/;" m struct:bpf_program file:
+priv libbpf.c /^ void *priv;$/;" m struct:bpf_map file:
+priv libbpf.c /^ void *priv;$/;" m struct:bpf_object file:
+priv libbpf.c /^ void *priv;$/;" m struct:bpf_program file:
+programs libbpf.c /^ struct bpf_program *programs;$/;" m struct:bpf_object typeref:struct:bpf_object::bpf_program file:
+ptr_to_u64 bpf.c /^static inline __u64 ptr_to_u64(const void *ptr)$/;" f file:
+reloc libbpf.c /^ } *reloc;$/;" m struct:bpf_object::__anon3 typeref:struct:bpf_object::__anon3::__anon4 file:
+reloc_desc libbpf.c /^ } *reloc_desc;$/;" m struct:bpf_program typeref:struct:bpf_program::__anon1 file:
+section_name libbpf.c /^ char *section_name;$/;" m struct:bpf_program file:
+shdr libbpf.c /^ GElf_Shdr shdr;$/;" m struct:bpf_object::__anon3::__anon4 file:
+srctree Makefile /^srctree := $(patsubst %\/,%,$(dir $(CURDIR)))$/;" m
+srctree Makefile /^srctree := $(patsubst %\/,%,$(dir $(srctree)))$/;" m
+strtabidx libbpf.c /^ size_t strtabidx;$/;" m struct:bpf_object::__anon3 file:
+symbols libbpf.c /^ Elf_Data *symbols;$/;" m struct:bpf_object::__anon3 file:
+sys_bpf bpf.c /^static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr,$/;" f file:
+type libbpf.c /^ enum bpf_prog_type type;$/;" m struct:bpf_program typeref:enum:bpf_program::bpf_prog_type file:
+type libbpf.h /^ unsigned int type;$/;" m struct:bpf_map_def
+update_dir Makefile /^define update_dir$/;" m
+value_size libbpf.h /^ unsigned int value_size;$/;" m struct:bpf_map_def
+zclose libbpf.c 153;" d file:
+zfree libbpf.c 149;" d file:
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next 9/9] selftests/bpf: test for BPF_F_LOCK
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
` (5 preceding siblings ...)
2019-01-16 5:08 ` [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags() Alexei Starovoitov
@ 2019-01-16 5:08 ` Alexei Starovoitov
6 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 5:08 UTC (permalink / raw)
To: davem; +Cc: daniel, jakub.kicinski, netdev, kernel-team
Add C based test that runs 4 bpf programs in parallel
that update the same hash and array maps.
And another 2 threads that read from these two maps
via lookup(key, value, BPF_F_LOCK) api
to make sure the user space sees consistent value in both
hash and array elements while user space races with kernel bpf progs.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 2 +-
tools/testing/selftests/bpf/test_map_lock.c | 66 ++++++++++++++++++
tools/testing/selftests/bpf/test_progs.c | 74 +++++++++++++++++++++
3 files changed, 141 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/bpf/test_map_lock.c
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index fcfda51406f9..368ca8249732 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -39,7 +39,7 @@ TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test
get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o \
test_sk_lookup_kern.o test_xdp_vlan.o test_queue_map.o test_stack_map.o \
- xdp_dummy.o test_map_in_map.o test_spin_lock.o
+ xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o
# Order correspond to 'make run_tests' order
TEST_PROGS := test_kmod.sh \
diff --git a/tools/testing/selftests/bpf/test_map_lock.c b/tools/testing/selftests/bpf/test_map_lock.c
new file mode 100644
index 000000000000..af8cc68ed2f9
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_map_lock.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2019 Facebook
+#include <linux/bpf.h>
+#include <linux/version.h>
+#include "bpf_helpers.h"
+
+#define VAR_NUM 16
+
+struct hmap_elem {
+ struct bpf_spin_lock lock;
+ int var[VAR_NUM];
+};
+
+struct bpf_map_def SEC("maps") hash_map = {
+ .type = BPF_MAP_TYPE_HASH,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct hmap_elem),
+ .max_entries = 1,
+};
+
+BPF_ANNOTATE_KV_PAIR(hash_map, int, struct hmap_elem);
+
+struct array_elem {
+ struct bpf_spin_lock lock;
+ int var[VAR_NUM];
+};
+
+struct bpf_map_def SEC("maps") array_map = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct array_elem),
+ .max_entries = 1,
+};
+
+BPF_ANNOTATE_KV_PAIR(array_map, int, struct array_elem);
+
+SEC("map_lock_demo")
+int bpf_map_lock_test(struct __sk_buff *skb)
+{
+ struct hmap_elem zero = {}, *val;
+ int rnd = bpf_get_prandom_u32();
+ int key = 0, err = 1, i;
+ struct array_elem *q;
+
+ val = bpf_map_lookup_elem(&hash_map, &key);
+ if (!val)
+ goto err;
+ /* spin_lock in hash map */
+ bpf_spin_lock(&val->lock);
+ for (i = 0; i < VAR_NUM; i++)
+ val->var[i] = rnd;
+ bpf_spin_unlock(&val->lock);
+
+ /* spin_lock in array */
+ q = bpf_map_lookup_elem(&array_map, &key);
+ if (!q)
+ goto err;
+ bpf_spin_lock(&q->lock);
+ for (i = 0; i < VAR_NUM; i++)
+ q->var[i] = rnd;
+ bpf_spin_unlock(&q->lock);
+ err = 0;
+err:
+ return err;
+}
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index 6425e95c3f16..1427338f14a2 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -1922,6 +1922,79 @@ static void test_spin_lock(void)
bpf_object__close(obj);
}
+static void *parallel_map_access(void *arg)
+{
+ int err, map_fd = *(u32 *) arg;
+ int vars[17], i, j, rnd, key = 0;
+
+ for (i = 0; i < 10000; i++) {
+ err = bpf_map_lookup_elem_flags(map_fd, &key, vars, BPF_F_LOCK);
+ if (err) {
+ printf("lookup failed\n");
+ error_cnt++;
+ goto out;
+ }
+ if (vars[0] != 0) {
+ printf("lookup #%d var[0]=%d\n", i, vars[0]);
+ error_cnt++;
+ goto out;
+ }
+ rnd = vars[1];
+ for (j = 2; j < 17; j++) {
+ if (vars[j] == rnd)
+ continue;
+ printf("lookup #%d var[1]=%d var[%d]=%d\n",
+ i, rnd, j, vars[j]);
+ error_cnt++;
+ goto out;
+ }
+ }
+out:
+ pthread_exit(arg);
+}
+
+static void test_map_lock(void)
+{
+ const char *file = "./test_map_lock.o";
+ int prog_fd, map_fd[2], vars[17] = {};
+ pthread_t thread_id[6];
+ struct bpf_object *obj;
+ int err = 0, key = 0, i;
+ void *ret;
+
+ err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
+ if (err) {
+ printf("test_map_lock:bpf_prog_load errno %d\n", errno);
+ goto close_prog;
+ }
+ map_fd[0] = bpf_find_map(__func__, obj, "hash_map");
+ if (map_fd[0] < 0)
+ goto close_prog;
+ map_fd[1] = bpf_find_map(__func__, obj, "array_map");
+ if (map_fd[1] < 0)
+ goto close_prog;
+
+ bpf_map_update_elem(map_fd[0], &key, vars, BPF_F_LOCK);
+
+ for (i = 0; i < 4; i++)
+ assert(pthread_create(&thread_id[i], NULL,
+ ¶llel_bpf_prog_test_run, &prog_fd) == 0);
+ for (i = 4; i < 6; i++)
+ assert(pthread_create(&thread_id[i], NULL,
+ ¶llel_map_access, &map_fd[i - 4]) == 0);
+ for (i = 0; i < 4; i++)
+ assert(pthread_join(thread_id[i], &ret) == 0 &&
+ ret == (void *)&prog_fd);
+ for (i = 4; i < 6; i++)
+ assert(pthread_join(thread_id[i], &ret) == 0 &&
+ ret == (void *)&map_fd[i - 4]);
+ goto close_prog_noerr;
+close_prog:
+ error_cnt++;
+close_prog_noerr:
+ bpf_object__close(obj);
+}
+
int main(void)
{
srand(time(NULL));
@@ -1950,6 +2023,7 @@ int main(void)
test_queue_stack_map(QUEUE);
test_queue_stack_map(STACK);
test_spin_lock();
+ test_map_lock();
printf("Summary: %d PASSED, %d FAILED\n", pass_cnt, error_cnt);
return error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
--
2.17.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags()
2019-01-16 5:08 ` [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags() Alexei Starovoitov
@ 2019-01-16 6:18 ` Yonghong Song
2019-01-16 6:28 ` Alexei Starovoitov
0 siblings, 1 reply; 20+ messages in thread
From: Yonghong Song @ 2019-01-16 6:18 UTC (permalink / raw)
To: Alexei Starovoitov, davem; +Cc: daniel, jakub.kicinski, netdev, Kernel Team
On 1/15/19 9:08 PM, Alexei Starovoitov wrote:
> Introduce
> int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags)
> helper to lookup array/hash/cgroup_local_storage elements with BPF_F_LOCK flag.
>
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> ---
> tools/lib/bpf/bpf.c | 13 ++
> tools/lib/bpf/bpf.h | 2 +
> tools/lib/bpf/libbpf.map | 4 +
> tools/lib/bpf/tags | 254 +++++++++++++++++++++++++++++++++++++++
I think this tools/lib/bpf/tags is not needed and accidentally slipped
in due to cscope?
> 4 files changed, 273 insertions(+)
> create mode 100644 tools/lib/bpf/tags
>
> diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
> index 3caaa3428774..d55a77a05d5f 100644
> --- a/tools/lib/bpf/bpf.c
> +++ b/tools/lib/bpf/bpf.c
> @@ -357,6 +357,19 @@ int bpf_map_lookup_elem(int fd, const void *key, void *value)
> return sys_bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
> }
>
> +int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags)
> +{
> + union bpf_attr attr;
> +
> + bzero(&attr, sizeof(attr));
> + attr.map_fd = fd;
> + attr.key = ptr_to_u64(key);
> + attr.value = ptr_to_u64(value);
> + attr.flags = flags;
> +
> + return sys_bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
> +}
> +
> int bpf_map_lookup_and_delete_elem(int fd, const void *key, void *value)
> {
> union bpf_attr attr;
> diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
> index 8f09de482839..ed09eed2dc3b 100644
> --- a/tools/lib/bpf/bpf.h
> +++ b/tools/lib/bpf/bpf.h
> @@ -110,6 +110,8 @@ LIBBPF_API int bpf_map_update_elem(int fd, const void *key, const void *value,
> __u64 flags);
>
> LIBBPF_API int bpf_map_lookup_elem(int fd, const void *key, void *value);
> +LIBBPF_API int bpf_map_lookup_elem_flags(int fd, const void *key, void *value,
> + __u64 flags);
> LIBBPF_API int bpf_map_lookup_and_delete_elem(int fd, const void *key,
> void *value);
> LIBBPF_API int bpf_map_delete_elem(int fd, const void *key);
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index cd02cd4e2cc3..ca5155409a15 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -124,3 +124,7 @@ LIBBPF_0.0.1 {
> local:
> *;
> };
> +LIBBPF_0.0.2 {
> + global:
> + bpf_map_lookup_elem_flags;
> +} LIBBPF_0.0.1;
> diff --git a/tools/lib/bpf/tags b/tools/lib/bpf/tags
> new file mode 100644
> index 000000000000..be30548a028e
> --- /dev/null
> +++ b/tools/lib/bpf/tags
> @@ -0,0 +1,254 @@
> +!_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to lines/
> +!_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/
> +!_TAG_PROGRAM_AUTHOR Darren Hiebert /dhiebert@users.sourceforge.net/
> +!_TAG_PROGRAM_NAME Exuberant Ctags //
> +!_TAG_PROGRAM_URL http://ctags.sourceforge.net /official site/
> +!_TAG_PROGRAM_VERSION 5.8 //
> +BPF_EXTRAVERSION Makefile /^BPF_EXTRAVERSION = 1$/;" m
> +BPF_FS_MAGIC libbpf.c 52;" d file:
> +BPF_IN Makefile /^BPF_IN := $(OUTPUT)libbpf-in.o$/;" m
> +BPF_LOG_BUF_SIZE bpf.h 43;" d
> +BPF_PATCHLEVEL Makefile /^BPF_PATCHLEVEL = 0$/;" m
> +BPF_PROG_TYPE_FNS libbpf.c 1702;" d file:
> +BPF_VERSION Makefile /^BPF_VERSION = 0$/;" m
> +CFLAGS Makefile /^ CFLAGS := $(EXTRA_CFLAGS)$/;" m
> +CFLAGS Makefile /^ CFLAGS := -g -Wall$/;" m
> +CHECK_ERR libbpf.c 140;" d file:
> +CMD_TARGETS Makefile /^CMD_TARGETS = $(LIB_FILE)$/;" m
> +DESTDIR Makefile /^DESTDIR ?=$/;" m
> +DESTDIR_SQ Makefile /^DESTDIR_SQ = '$(subst ','\\'',$(DESTDIR))'$/;" m
> +EM_BPF libbpf.c 48;" d file:
> +ERRCODE_OFFSET libbpf.c 95;" d file:
> +ERRNO_OFFSET libbpf.c 94;" d file:
> +EXTRAVERSION Makefile /^EXTRAVERSION = $(BPF_EXTRAVERSION)$/;" m
> +FEATURE_CHECK_CFLAGS-bpf Makefile /^FEATURE_CHECK_CFLAGS-bpf = $(INCLUDES)$/;" m
> +FEATURE_DISPLAY Makefile /^FEATURE_DISPLAY = libelf bpf$/;" m
> +FEATURE_TESTS Makefile /^FEATURE_TESTS = libelf libelf-getphdrnum libelf-mmap bpf$/;" m
> +FEATURE_USER Makefile /^FEATURE_USER = .libbpf$/;" m
> +INCLUDES Makefile /^INCLUDES = -I. -I$(srctree)\/tools\/include -I$(srctree)\/tools\/arch\/$(ARCH)\/include\/uapi -I$(srctree)\/tools\/include\/uapi$/;" m
> +INSTALL Makefile /^INSTALL = install$/;" m
> +LIBBPF_ELF_C_READ_MMAP libbpf.c 162;" d file:
> +LIBBPF_ELF_C_READ_MMAP libbpf.c 164;" d file:
> +LIBBPF_ERRNO__ENDIAN libbpf.h /^ LIBBPF_ERRNO__ENDIAN, \/* Endian mismatch *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__FORMAT libbpf.h /^ LIBBPF_ERRNO__FORMAT, \/* BPF object format invalid *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__INTERNAL libbpf.h /^ LIBBPF_ERRNO__INTERNAL, \/* Internal error in libbpf *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__KVER libbpf.h /^ LIBBPF_ERRNO__KVER, \/* Incorrect kernel version *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__KVERSION libbpf.h /^ LIBBPF_ERRNO__KVERSION, \/* Incorrect or no 'version' section *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__LIBELF libbpf.h /^ LIBBPF_ERRNO__LIBELF = __LIBBPF_ERRNO__START,$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__LOAD libbpf.h /^ LIBBPF_ERRNO__LOAD, \/* Load program failure for unknown reason *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__PROG2BIG libbpf.h /^ LIBBPF_ERRNO__PROG2BIG, \/* Program too big *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__PROGTYPE libbpf.h /^ LIBBPF_ERRNO__PROGTYPE, \/* Kernel doesn't support this program type *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__RELOC libbpf.h /^ LIBBPF_ERRNO__RELOC, \/* Relocation failed *\/$/;" e enum:libbpf_errno
> +LIBBPF_ERRNO__VERIFY libbpf.h /^ LIBBPF_ERRNO__VERIFY, \/* Kernel verifier blocks program loading *\/$/;" e enum:libbpf_errno
> +LIBBPF_VERSION Makefile /^LIBBPF_VERSION = $(BPF_VERSION).$(BPF_PATCHLEVEL).$(BPF_EXTRAVERSION)$/;" m
> +LIB_FILE Makefile /^LIB_FILE := $(addprefix $(OUTPUT),$(LIB_FILE))$/;" m
> +LIB_FILE Makefile /^LIB_FILE = libbpf.a libbpf.so$/;" m
> +MAKEOVERRIDES Makefile /^MAKEOVERRIDES=$/;" m
> +N Makefile /^N =$/;" m
> +NON_CHECK_FEAT_TARGETS Makefile /^NON_CHECK_FEAT_TARGETS := clean TAGS tags cscope help$/;" m
> +NR_ERRNO libbpf.c 96;" d file:
> +OBJ Makefile /^OBJ = $@$/;" m
> +PATCHLEVEL Makefile /^PATCHLEVEL = $(BPF_PATCHLEVEL)$/;" m
> +Q Makefile /^ Q = @$/;" m
> +Q Makefile /^ Q =$/;" m
> +STRERR_BUFSIZE libbpf.c 92;" d file:
> +TARGETS Makefile /^TARGETS = $(CMD_TARGETS)$/;" m
> +VERBOSE Makefile /^ VERBOSE = $(V)$/;" m
> +VERBOSE Makefile /^ VERBOSE = 0$/;" m
> +VERSION Makefile /^VERSION = $(BPF_VERSION)$/;" m
> +__BPF_BPF_H bpf.h 22;" d
> +__BPF_LIBBPF_H libbpf.h 22;" d
> +__LIBBPF_ERRNO__END libbpf.h /^ __LIBBPF_ERRNO__END,$/;" e enum:libbpf_errno
> +__LIBBPF_ERRNO__START libbpf.h /^ __LIBBPF_ERRNO__START = 4000,$/;" e enum:libbpf_errno
> +__NR_bpf bpf.c 35;" d file:
> +__NR_bpf bpf.c 37;" d file:
> +__NR_bpf bpf.c 39;" d file:
> +__NR_bpf bpf.c 41;" d file:
> +__NR_bpf bpf.c 43;" d file:
> +__base_pr libbpf.c /^static int __base_pr(const char *format, ...)$/;" f file:
> +__bpf_object__open libbpf.c /^__bpf_object__open(const char *path, void *obj_buf, size_t obj_buf_sz)$/;" f file:
> +__pr libbpf.c 73;" d file:
> +__pr_debug libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_debug;$/;" v
> +__pr_info libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_info = __base_pr;$/;" v
> +__pr_warning libbpf.c /^static __printf(1, 2) libbpf_print_fn_t __pr_warning = __base_pr;$/;" v
> +__printf libbpf.c 55;" d file:
> +allow-override Makefile /^define allow-override$/;" m
> +bpf_create_map bpf.c /^int bpf_create_map(enum bpf_map_type map_type, int key_size,$/;" f
> +bpf_create_map_in_map bpf.c /^int bpf_create_map_in_map(enum bpf_map_type map_type, const char *name,$/;" f
> +bpf_create_map_in_map_node bpf.c /^int bpf_create_map_in_map_node(enum bpf_map_type map_type, const char *name,$/;" f
> +bpf_create_map_name bpf.c /^int bpf_create_map_name(enum bpf_map_type map_type, const char *name,$/;" f
> +bpf_create_map_node bpf.c /^int bpf_create_map_node(enum bpf_map_type map_type, const char *name,$/;" f
> +bpf_load_program bpf.c /^int bpf_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,$/;" f
> +bpf_load_program_name bpf.c /^int bpf_load_program_name(enum bpf_prog_type type, const char *name,$/;" f
> +bpf_map libbpf.c /^struct bpf_map {$/;" s file:
> +bpf_map__def libbpf.c /^const struct bpf_map_def *bpf_map__def(struct bpf_map *map)$/;" f
> +bpf_map__fd libbpf.c /^int bpf_map__fd(struct bpf_map *map)$/;" f
> +bpf_map__for_each libbpf.h 230;" d
> +bpf_map__name libbpf.c /^const char *bpf_map__name(struct bpf_map *map)$/;" f
> +bpf_map__next libbpf.c /^bpf_map__next(struct bpf_map *prev, struct bpf_object *obj)$/;" f
> +bpf_map__pin libbpf.c /^int bpf_map__pin(struct bpf_map *map, const char *path)$/;" f
> +bpf_map__priv libbpf.c /^void *bpf_map__priv(struct bpf_map *map)$/;" f
> +bpf_map__set_priv libbpf.c /^int bpf_map__set_priv(struct bpf_map *map, void *priv,$/;" f
> +bpf_map_clear_priv_t libbpf.h /^typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);$/;" t
> +bpf_map_def libbpf.h /^struct bpf_map_def {$/;" s
> +bpf_map_delete_elem bpf.c /^int bpf_map_delete_elem(int fd, const void *key)$/;" f
> +bpf_map_get_fd_by_id bpf.c /^int bpf_map_get_fd_by_id(__u32 id)$/;" f
> +bpf_map_get_next_id bpf.c /^int bpf_map_get_next_id(__u32 start_id, __u32 *next_id)$/;" f
> +bpf_map_get_next_key bpf.c /^int bpf_map_get_next_key(int fd, const void *key, void *next_key)$/;" f
> +bpf_map_lookup_elem bpf.c /^int bpf_map_lookup_elem(int fd, const void *key, void *value)$/;" f
> +bpf_map_update_elem bpf.c /^int bpf_map_update_elem(int fd, const void *key, const void *value,$/;" f
> +bpf_obj_get bpf.c /^int bpf_obj_get(const char *pathname)$/;" f
> +bpf_obj_get_info_by_fd bpf.c /^int bpf_obj_get_info_by_fd(int prog_fd, void *info, __u32 *info_len)$/;" f
> +bpf_obj_pin bpf.c /^int bpf_obj_pin(int fd, const char *pathname)$/;" f
> +bpf_object libbpf.c /^struct bpf_object {$/;" s file:
> +bpf_object__add_program libbpf.c /^bpf_object__add_program(struct bpf_object *obj, void *data, size_t size,$/;" f file:
> +bpf_object__check_endianness libbpf.c /^bpf_object__check_endianness(struct bpf_object *obj)$/;" f file:
> +bpf_object__close libbpf.c /^void bpf_object__close(struct bpf_object *obj)$/;" f
> +bpf_object__collect_reloc libbpf.c /^static int bpf_object__collect_reloc(struct bpf_object *obj)$/;" f file:
> +bpf_object__create_maps libbpf.c /^bpf_object__create_maps(struct bpf_object *obj)$/;" f file:
> +bpf_object__elf_collect libbpf.c /^static int bpf_object__elf_collect(struct bpf_object *obj)$/;" f file:
> +bpf_object__elf_finish libbpf.c /^static void bpf_object__elf_finish(struct bpf_object *obj)$/;" f file:
> +bpf_object__elf_init libbpf.c /^static int bpf_object__elf_init(struct bpf_object *obj)$/;" f file:
> +bpf_object__find_map_by_name libbpf.c /^bpf_object__find_map_by_name(struct bpf_object *obj, const char *name)$/;" f
> +bpf_object__find_map_by_offset libbpf.c /^bpf_object__find_map_by_offset(struct bpf_object *obj, size_t offset)$/;" f
> +bpf_object__find_prog_by_idx libbpf.c /^bpf_object__find_prog_by_idx(struct bpf_object *obj, int idx)$/;" f file:
> +bpf_object__for_each_program libbpf.h 95;" d
> +bpf_object__for_each_safe libbpf.h 79;" d
> +bpf_object__init_kversion libbpf.c /^bpf_object__init_kversion(struct bpf_object *obj,$/;" f file:
> +bpf_object__init_license libbpf.c /^bpf_object__init_license(struct bpf_object *obj,$/;" f file:
> +bpf_object__init_maps libbpf.c /^bpf_object__init_maps(struct bpf_object *obj)$/;" f file:
> +bpf_object__init_prog_names libbpf.c /^bpf_object__init_prog_names(struct bpf_object *obj)$/;" f file:
> +bpf_object__kversion libbpf.c /^unsigned int bpf_object__kversion(struct bpf_object *obj)$/;" f
> +bpf_object__load libbpf.c /^int bpf_object__load(struct bpf_object *obj)$/;" f
> +bpf_object__load_progs libbpf.c /^bpf_object__load_progs(struct bpf_object *obj)$/;" f file:
> +bpf_object__name libbpf.c /^const char *bpf_object__name(struct bpf_object *obj)$/;" f
> +bpf_object__new libbpf.c /^static struct bpf_object *bpf_object__new(const char *path,$/;" f file:
> +bpf_object__next libbpf.c /^bpf_object__next(struct bpf_object *prev)$/;" f
> +bpf_object__open libbpf.c /^struct bpf_object *bpf_object__open(const char *path)$/;" f
> +bpf_object__open_buffer libbpf.c /^struct bpf_object *bpf_object__open_buffer(void *obj_buf,$/;" f
> +bpf_object__pin libbpf.c /^int bpf_object__pin(struct bpf_object *obj, const char *path)$/;" f
> +bpf_object__priv libbpf.c /^void *bpf_object__priv(struct bpf_object *obj)$/;" f
> +bpf_object__relocate libbpf.c /^bpf_object__relocate(struct bpf_object *obj)$/;" f file:
> +bpf_object__set_priv libbpf.c /^int bpf_object__set_priv(struct bpf_object *obj, void *priv,$/;" f
> +bpf_object__unload libbpf.c /^int bpf_object__unload(struct bpf_object *obj)$/;" f
> +bpf_object__validate libbpf.c /^static int bpf_object__validate(struct bpf_object *obj)$/;" f file:
> +bpf_object_clear_priv_t libbpf.h /^typedef void (*bpf_object_clear_priv_t)(struct bpf_object *, void *);$/;" t
> +bpf_prog_attach bpf.c /^int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type,$/;" f
> +bpf_prog_detach bpf.c /^int bpf_prog_detach(int target_fd, enum bpf_attach_type type)$/;" f
> +bpf_prog_detach2 bpf.c /^int bpf_prog_detach2(int prog_fd, int target_fd, enum bpf_attach_type type)$/;" f
> +bpf_prog_get_fd_by_id bpf.c /^int bpf_prog_get_fd_by_id(__u32 id)$/;" f
> +bpf_prog_get_next_id bpf.c /^int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id)$/;" f
> +bpf_prog_load libbpf.c /^int bpf_prog_load(const char *file, enum bpf_prog_type type,$/;" f
> +bpf_prog_prep_result libbpf.h /^struct bpf_prog_prep_result {$/;" s
> +bpf_prog_query bpf.c /^int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags,$/;" f
> +bpf_prog_test_run bpf.c /^int bpf_prog_test_run(int prog_fd, int repeat, void *data, __u32 size,$/;" f
> +bpf_program libbpf.c /^struct bpf_program {$/;" s file:
> +bpf_program__collect_reloc libbpf.c /^bpf_program__collect_reloc(struct bpf_program *prog,$/;" f file:
> +bpf_program__exit libbpf.c /^static void bpf_program__exit(struct bpf_program *prog)$/;" f file:
> +bpf_program__fd libbpf.c /^int bpf_program__fd(struct bpf_program *prog)$/;" f
> +bpf_program__init libbpf.c /^bpf_program__init(void *data, size_t size, char *section_name, int idx,$/;" f file:
> +bpf_program__is_type libbpf.c /^static bool bpf_program__is_type(struct bpf_program *prog,$/;" f file:
> +bpf_program__load libbpf.c /^bpf_program__load(struct bpf_program *prog,$/;" f file:
> +bpf_program__next libbpf.c /^bpf_program__next(struct bpf_program *prev, struct bpf_object *obj)$/;" f
> +bpf_program__nth_fd libbpf.c /^int bpf_program__nth_fd(struct bpf_program *prog, int n)$/;" f
> +bpf_program__pin libbpf.c /^int bpf_program__pin(struct bpf_program *prog, const char *path)$/;" f
> +bpf_program__pin_instance libbpf.c /^int bpf_program__pin_instance(struct bpf_program *prog, const char *path,$/;" f
> +bpf_program__priv libbpf.c /^void *bpf_program__priv(struct bpf_program *prog)$/;" f
> +bpf_program__relocate libbpf.c /^bpf_program__relocate(struct bpf_program *prog, struct bpf_object *obj)$/;" f file:
> +bpf_program__set_prep libbpf.c /^int bpf_program__set_prep(struct bpf_program *prog, int nr_instances,$/;" f
> +bpf_program__set_priv libbpf.c /^int bpf_program__set_priv(struct bpf_program *prog, void *priv,$/;" f
> +bpf_program__set_type libbpf.c /^void bpf_program__set_type(struct bpf_program *prog, enum bpf_prog_type type)$/;" f
> +bpf_program__title libbpf.c /^const char *bpf_program__title(struct bpf_program *prog, bool needs_copy)$/;" f
> +bpf_program__unload libbpf.c /^static void bpf_program__unload(struct bpf_program *prog)$/;" f file:
> +bpf_program_clear_priv_t libbpf.h /^typedef void (*bpf_program_clear_priv_t)(struct bpf_program *,$/;" t
> +bpf_program_prep_t libbpf.h /^typedef int (*bpf_program_prep_t)(struct bpf_program *prog, int n,$/;" t
> +bpf_verify_program bpf.c /^int bpf_verify_program(enum bpf_prog_type type, const struct bpf_insn *insns,$/;" f
> +check_feat Makefile /^ check_feat := 0$/;" m
> +check_feat Makefile /^check_feat := 1$/;" m
> +check_path libbpf.c /^static int check_path(const char *path)$/;" f file:
> +clear_priv libbpf.c /^ bpf_map_clear_priv_t clear_priv;$/;" m struct:bpf_map file:
> +clear_priv libbpf.c /^ bpf_object_clear_priv_t clear_priv;$/;" m struct:bpf_object file:
> +clear_priv libbpf.c /^ bpf_program_clear_priv_t clear_priv;$/;" m struct:bpf_program file:
> +compare_bpf_map libbpf.c /^static int compare_bpf_map(const void *_a, const void *_b)$/;" f file:
> +data libbpf.c /^ Elf_Data *data;$/;" m struct:bpf_object::__anon3::__anon4 file:
> +def libbpf.c /^ struct bpf_map_def def;$/;" m struct:bpf_map typeref:struct:bpf_map::bpf_map_def file:
> +do_install Makefile /^define do_install$/;" m
> +efile libbpf.c /^ } efile;$/;" m struct:bpf_object typeref:struct:bpf_object::__anon3 file:
> +ehdr libbpf.c /^ GElf_Ehdr ehdr;$/;" m struct:bpf_object::__anon3 file:
> +elf libbpf.c /^ Elf *elf;$/;" m struct:bpf_object::__anon3 file:
> +fd libbpf.c /^ int fd;$/;" m struct:bpf_object::__anon3 file:
> +fd libbpf.c /^ int fd;$/;" m struct:bpf_map file:
> +fds libbpf.c /^ int *fds;$/;" m struct:bpf_program::__anon2 file:
> +idx libbpf.c /^ int idx;$/;" m struct:bpf_program file:
> +insn_idx libbpf.c /^ int insn_idx;$/;" m struct:bpf_program::__anon1 file:
> +insns libbpf.c /^ struct bpf_insn *insns;$/;" m struct:bpf_program typeref:struct:bpf_program::bpf_insn file:
> +insns_cnt libbpf.c /^ size_t insns_cnt;$/;" m struct:bpf_program file:
> +instances libbpf.c /^ } instances;$/;" m struct:bpf_program typeref:struct:bpf_program::__anon2 file:
> +kern_version libbpf.c /^ u32 kern_version;$/;" m struct:bpf_object file:
> +key_size libbpf.h /^ unsigned int key_size;$/;" m struct:bpf_map_def
> +libbpf_errno libbpf.h /^enum libbpf_errno {$/;" g
> +libbpf_get_error libbpf.c /^long libbpf_get_error(const void *ptr)$/;" f
> +libbpf_print_fn_t libbpf.h /^typedef int (*libbpf_print_fn_t)(const char *, ...)$/;" t
> +libbpf_set_print libbpf.c /^void libbpf_set_print(libbpf_print_fn_t warn,$/;" f
> +libbpf_strerror libbpf.c /^int libbpf_strerror(int err, char *buf, size_t size)$/;" f
> +libbpf_strerror_table libbpf.c /^static const char *libbpf_strerror_table[NR_ERRNO] = {$/;" v file:
> +libdir Makefile /^libdir = $(prefix)\/$(libdir_relative)$/;" m
> +libdir_SQ Makefile /^libdir_SQ = $(subst ','\\'',$(libdir))$/;" m
> +libdir_relative Makefile /^ libdir_relative = lib$/;" m
> +libdir_relative Makefile /^ libdir_relative = lib64$/;" m
> +libdir_relative_SQ Makefile /^libdir_relative_SQ = $(subst ','\\'',$(libdir_relative))$/;" m
> +license libbpf.c /^ char license[64];$/;" m struct:bpf_object file:
> +list libbpf.c /^ struct list_head list;$/;" m struct:bpf_object typeref:struct:bpf_object::list_head file:
> +load_program libbpf.c /^load_program(enum bpf_prog_type type, const char *name, struct bpf_insn *insns,$/;" f file:
> +loaded libbpf.c /^ bool loaded;$/;" m struct:bpf_object file:
> +make_dir libbpf.c /^static int make_dir(const char *path)$/;" f file:
> +man_dir Makefile /^man_dir = $(prefix)\/share\/man$/;" m
> +man_dir_SQ Makefile /^man_dir_SQ = '$(subst ','\\'',$(man_dir))'$/;" m
> +map_flags libbpf.h /^ unsigned int map_flags;$/;" m struct:bpf_map_def
> +map_idx libbpf.c /^ int map_idx;$/;" m struct:bpf_program::__anon1 file:
> +maps libbpf.c /^ struct bpf_map *maps;$/;" m struct:bpf_object typeref:struct:bpf_object::bpf_map file:
> +maps_shndx libbpf.c /^ int maps_shndx;$/;" m struct:bpf_object::__anon3 file:
> +max_entries libbpf.h /^ unsigned int max_entries;$/;" m struct:bpf_map_def
> +min bpf.c 49;" d file:
> +name libbpf.c /^ char *name;$/;" m struct:bpf_map file:
> +name libbpf.c /^ char *name;$/;" m struct:bpf_program file:
> +new_insn_cnt libbpf.h /^ int new_insn_cnt;$/;" m struct:bpf_prog_prep_result
> +new_insn_ptr libbpf.h /^ struct bpf_insn *new_insn_ptr;$/;" m struct:bpf_prog_prep_result typeref:struct:bpf_prog_prep_result::bpf_insn
> +nr libbpf.c /^ int nr;$/;" m struct:bpf_program::__anon2 file:
> +nr_maps libbpf.c /^ size_t nr_maps;$/;" m struct:bpf_object file:
> +nr_programs libbpf.c /^ size_t nr_programs;$/;" m struct:bpf_object file:
> +nr_reloc libbpf.c /^ int nr_reloc;$/;" m struct:bpf_object::__anon3 file:
> +nr_reloc libbpf.c /^ int nr_reloc;$/;" m struct:bpf_program file:
> +obj libbpf.c /^ struct bpf_object *obj;$/;" m struct:bpf_program typeref:struct:bpf_program::bpf_object file:
> +obj_buf libbpf.c /^ void *obj_buf;$/;" m struct:bpf_object::__anon3 file:
> +obj_buf_sz libbpf.c /^ size_t obj_buf_sz;$/;" m struct:bpf_object::__anon3 file:
> +obj_elf_valid libbpf.c 250;" d file:
> +offset libbpf.c /^ size_t offset;$/;" m struct:bpf_map file:
> +path libbpf.c /^ char path[];$/;" m struct:bpf_object file:
> +pfd libbpf.h /^ int *pfd;$/;" m struct:bpf_prog_prep_result
> +plugin_dir_SQ Makefile /^plugin_dir_SQ = $(subst ','\\'',$(plugin_dir))$/;" m
> +pr_debug libbpf.c 81;" d file:
> +pr_info libbpf.c 80;" d file:
> +pr_warning libbpf.c 79;" d file:
> +prefix Makefile /^prefix ?= \/usr\/local$/;" m
> +preprocessor libbpf.c /^ bpf_program_prep_t preprocessor;$/;" m struct:bpf_program file:
> +priv libbpf.c /^ void *priv;$/;" m struct:bpf_map file:
> +priv libbpf.c /^ void *priv;$/;" m struct:bpf_object file:
> +priv libbpf.c /^ void *priv;$/;" m struct:bpf_program file:
> +programs libbpf.c /^ struct bpf_program *programs;$/;" m struct:bpf_object typeref:struct:bpf_object::bpf_program file:
> +ptr_to_u64 bpf.c /^static inline __u64 ptr_to_u64(const void *ptr)$/;" f file:
> +reloc libbpf.c /^ } *reloc;$/;" m struct:bpf_object::__anon3 typeref:struct:bpf_object::__anon3::__anon4 file:
> +reloc_desc libbpf.c /^ } *reloc_desc;$/;" m struct:bpf_program typeref:struct:bpf_program::__anon1 file:
> +section_name libbpf.c /^ char *section_name;$/;" m struct:bpf_program file:
> +shdr libbpf.c /^ GElf_Shdr shdr;$/;" m struct:bpf_object::__anon3::__anon4 file:
> +srctree Makefile /^srctree := $(patsubst %\/,%,$(dir $(CURDIR)))$/;" m
> +srctree Makefile /^srctree := $(patsubst %\/,%,$(dir $(srctree)))$/;" m
> +strtabidx libbpf.c /^ size_t strtabidx;$/;" m struct:bpf_object::__anon3 file:
> +symbols libbpf.c /^ Elf_Data *symbols;$/;" m struct:bpf_object::__anon3 file:
> +sys_bpf bpf.c /^static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr,$/;" f file:
> +type libbpf.c /^ enum bpf_prog_type type;$/;" m struct:bpf_program typeref:enum:bpf_program::bpf_prog_type file:
> +type libbpf.h /^ unsigned int type;$/;" m struct:bpf_map_def
> +update_dir Makefile /^define update_dir$/;" m
> +value_size libbpf.h /^ unsigned int value_size;$/;" m struct:bpf_map_def
> +zclose libbpf.c 153;" d file:
> +zfree libbpf.c 149;" d file:
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags()
2019-01-16 6:18 ` Yonghong Song
@ 2019-01-16 6:28 ` Alexei Starovoitov
0 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 6:28 UTC (permalink / raw)
To: Yonghong Song
Cc: Alexei Starovoitov, davem, daniel, jakub.kicinski, netdev, Kernel Team
On Wed, Jan 16, 2019 at 06:18:30AM +0000, Yonghong Song wrote:
>
>
> On 1/15/19 9:08 PM, Alexei Starovoitov wrote:
> > Introduce
> > int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags)
> > helper to lookup array/hash/cgroup_local_storage elements with BPF_F_LOCK flag.
> >
> > Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> > ---
> > tools/lib/bpf/bpf.c | 13 ++
> > tools/lib/bpf/bpf.h | 2 +
> > tools/lib/bpf/libbpf.map | 4 +
> > tools/lib/bpf/tags | 254 +++++++++++++++++++++++++++++++++++++++
>
> I think this tools/lib/bpf/tags is not needed and accidentally slipped
> in due to cscope?
Oops.
Funny how checkpatch.pl commented about it:
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#69:
new file mode 100644
it didn't mention the file by name, so I ignored that warning.
Please continue reviewing. I will respin.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
@ 2019-01-16 22:48 ` Daniel Borkmann
2019-01-16 23:16 ` Daniel Borkmann
2019-01-16 23:23 ` Alexei Starovoitov
2019-01-17 0:16 ` Martin Lau
1 sibling, 2 replies; 20+ messages in thread
From: Daniel Borkmann @ 2019-01-16 22:48 UTC (permalink / raw)
To: Alexei Starovoitov, davem; +Cc: jakub.kicinski, netdev, kernel-team
On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
> Introduce 'struct bpf_spin_lock' and bpf_spin_lock/unlock() helpers to let
> bpf program serialize access to other variables.
>
> Example:
> struct hash_elem {
> int cnt;
> struct bpf_spin_lock lock;
> };
> struct hash_elem * val = bpf_map_lookup_elem(&hash_map, &key);
> if (val) {
> bpf_spin_lock(&val->lock);
> val->cnt++;
> bpf_spin_unlock(&val->lock);
> }
>
> Restrictions and safety checks:
> - bpf_spin_lock is only allowed inside HASH and ARRAY maps.
> - BTF description of the map is mandatory for safety analysis.
> - bpf program can take one bpf_spin_lock at a time, since two or more can
> cause dead locks.
> - only one 'struct bpf_spin_lock' is allowed per map element.
> It drastically simplifies implementation yet allows bpf program to use
> any number of bpf_spin_locks.
> - when bpf_spin_lock is taken the calls (either bpf2bpf or helpers) are not allowed.
> - bpf program must bpf_spin_unlock() before return.
> - bpf program can access 'struct bpf_spin_lock' only via
> bpf_spin_lock()/bpf_spin_unlock() helpers.
> - load/store into 'struct bpf_spin_lock lock;' field is not allowed.
> - to use bpf_spin_lock() helper the BTF description of map value must be
> a struct and have 'struct bpf_spin_lock anyname;' field at the top level.
> Nested lock inside another struct is not allowed.
> - syscall map_lookup doesn't copy bpf_spin_lock field to user space.
> - syscall map_update and program map_update do not update bpf_spin_lock field.
> - bpf_spin_lock cannot be on the stack or inside networking packet.
> bpf_spin_lock can only be inside HASH or ARRAY map value.
> - bpf_spin_lock is available to root only and to all program types.
>
> Implementation details:
> - on !SMP bpf_spin_lock() becomes nop
> - presence of bpf_spin_lock inside map value could have been indicated via
> extra flag during map_create, but specifying it via BTF is cleaner.
> It provides introspection for map key/value and reduces user coding mistakes.
>
> Next steps:
> - allow bpf_spin_lock in other map types (like cgroup local storage)
> - introduce BPF_F_LOCK flag for bpf_map_update() syscall and helper
> to request kernel to grab bpf_spin_lock before rewriting the value.
> That will serialize access to map elements.
>
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
[...]
> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
> index a74972b07e74..591fdedae7bf 100644
> --- a/kernel/bpf/helpers.c
> +++ b/kernel/bpf/helpers.c
> @@ -221,6 +221,41 @@ const struct bpf_func_proto bpf_get_current_comm_proto = {
> .arg2_type = ARG_CONST_SIZE,
> };
>
> +BPF_CALL_1(bpf_spin_lock, struct bpf_spin_lock *, lock)
> +{
> +#if defined(CONFIG_SMP)
> + struct qspinlock *qlock = (void *)lock;
> +
> + BUILD_BUG_ON(sizeof(*qlock) != sizeof(*lock));
> + queued_spin_lock(qlock);
> +#endif
> + return 0;
> +}
> +
> +const struct bpf_func_proto bpf_spin_lock_proto = {
> + .func = bpf_spin_lock,
> + .gpl_only = false,
> + .ret_type = RET_VOID,
> + .arg1_type = ARG_PTR_TO_SPIN_LOCK,
> +};
> +
> +BPF_CALL_1(bpf_spin_unlock, struct bpf_spin_lock *, lock)
> +{
> +#if defined(CONFIG_SMP)
> + struct qspinlock *qlock = (void *)lock;
> +
> + queued_spin_unlock(qlock);
> +#endif
> + return 0;
> +}
> +
> +const struct bpf_func_proto bpf_spin_unlock_proto = {
> + .func = bpf_spin_unlock,
> + .gpl_only = false,
> + .ret_type = RET_VOID,
> + .arg1_type = ARG_PTR_TO_SPIN_LOCK,
> +};
> +
> #ifdef CONFIG_CGROUPS
> BPF_CALL_0(bpf_get_current_cgroup_id)
> {
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index b155cd17c1bd..ebf0a673cb83 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -463,7 +463,7 @@ int map_check_no_btf(const struct bpf_map *map,
> return -ENOTSUPP;
> }
>
> -static int map_check_btf(const struct bpf_map *map, const struct btf *btf,
> +static int map_check_btf(struct bpf_map *map, const struct btf *btf,
> u32 btf_key_id, u32 btf_value_id)
> {
> const struct btf_type *key_type, *value_type;
> @@ -478,6 +478,21 @@ static int map_check_btf(const struct bpf_map *map, const struct btf *btf,
> if (!value_type || value_size != map->value_size)
> return -EINVAL;
>
> + map->spin_lock_off = btf_find_spin_lock(btf, value_type);
> +
> + if (map_value_has_spin_lock(map)) {
> + if (map->map_type != BPF_MAP_TYPE_HASH &&
> + map->map_type != BPF_MAP_TYPE_ARRAY)
> + return -ENOTSUPP;
> + if (map->spin_lock_off + sizeof(struct bpf_spin_lock) >
> + map->value_size) {
> + WARN_ONCE(1,
> + "verifier bug spin_lock_off %d value_size %d\n",
> + map->spin_lock_off, map->value_size);
> + return -EFAULT;
> + }
> + }
> +
> if (map->ops->map_check_btf)
> ret = map->ops->map_check_btf(map, btf, key_type, value_type);
>
> @@ -542,6 +557,8 @@ static int map_create(union bpf_attr *attr)
> map->btf = btf;
> map->btf_key_type_id = attr->btf_key_type_id;
> map->btf_value_type_id = attr->btf_value_type_id;
> + } else {
> + map->spin_lock_off = -EINVAL;
> }
>
> err = security_bpf_map_alloc(map);
> @@ -740,7 +757,7 @@ static int map_lookup_elem(union bpf_attr *attr)
> err = -ENOENT;
> } else {
> err = 0;
> - memcpy(value, ptr, value_size);
> + copy_map_value(map, value, ptr);
> }
> rcu_read_unlock();
> }
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 56674a7c3778..0f3d1fb30d7a 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -213,6 +213,7 @@ struct bpf_call_arg_meta {
> s64 msize_smax_value;
> u64 msize_umax_value;
> int ptr_id;
> + int func_id;
> };
>
> static DEFINE_MUTEX(bpf_verifier_lock);
> @@ -351,6 +352,12 @@ static bool reg_is_refcounted(const struct bpf_reg_state *reg)
> return type_is_refcounted(reg->type);
> }
>
> +static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg)
> +{
> + return reg->type == PTR_TO_MAP_VALUE &&
> + map_value_has_spin_lock(reg->map_ptr);
> +}
> +
> static bool reg_is_refcounted_or_null(const struct bpf_reg_state *reg)
> {
> return type_is_refcounted_or_null(reg->type);
> @@ -712,6 +719,7 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state,
> }
> dst_state->speculative = src->speculative;
> dst_state->curframe = src->curframe;
> + dst_state->active_spin_lock = src->active_spin_lock;
> for (i = 0; i <= src->curframe; i++) {
> dst = dst_state->frame[i];
> if (!dst) {
> @@ -1483,6 +1491,21 @@ static int check_map_access(struct bpf_verifier_env *env, u32 regno,
> if (err)
> verbose(env, "R%d max value is outside of the array range\n",
> regno);
> +
> + if (map_value_has_spin_lock(reg->map_ptr)) {
> + u32 lock = reg->map_ptr->spin_lock_off;
> +
> + /* if any part of struct bpf_spin_lock can be touched by
> + * load/store reject this program
> + */
> + if ((reg->smin_value + off <= lock &&
> + lock < reg->umax_value + off + size) ||
> + (reg->smin_value + off < lock + sizeof(struct bpf_spin_lock) &&
> + lock + sizeof(struct bpf_spin_lock) <= reg->umax_value + off + size)) {
> + verbose(env, "bpf_spin_lock cannot be accessed directly by load/store\n");
> + return -EACCES;
> + }
> + }
> return err;
> }
>
> @@ -2192,6 +2215,91 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
> }
> }
>
> +/* Implementation details:
> + * bpf_map_lookup returns PTR_TO_MAP_VALUE_OR_NULL
> + * Two bpf_map_lookups (even with the same key) will have different reg->id.
> + * For traditional PTR_TO_MAP_VALUE the verifier clears reg->id after
> + * value_or_null->value transition, since the verifier only cares about
> + * the range of access to valid map value pointer and doesn't care about actual
> + * address of the map element.
> + * For maps with 'struct bpf_spin_lock' inside map value the verifier keeps
> + * reg->id > 0 after value_or_null->value transition. By doing so
> + * two bpf_map_lookups will be considered two different pointers that
> + * point to different bpf_spin_locks.
> + * The verifier allows taking only one bpf_spin_lock at a time to avoid
> + * dead-locks.
> + * Since only one bpf_spin_lock is allowed the checks are simpler than
> + * reg_is_refcounted() logic. The verifier needs to remember only
> + * one spin_lock instead of array of acquired_refs.
> + * cur_state->active_spin_lock remembers which map value element got locked
> + * and clears it after bpf_spin_unlock.
> + */
> +static int process_spin_lock(struct bpf_verifier_env *env, int regno,
> + bool is_lock)
> +{
> + struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno];
> + struct bpf_verifier_state *cur = env->cur_state;
> + bool is_const = tnum_is_const(reg->var_off);
> + struct bpf_map *map = reg->map_ptr;
> + u64 val = reg->var_off.value;
> +
> + if (reg->type != PTR_TO_MAP_VALUE) {
> + verbose(env, "R%d is not a pointer to map_value\n", regno);
> + return -EINVAL;
> + }
> + if (!is_const) {
> + verbose(env,
> + "R%d doesn't have constant offset. bpf_spin_lock has to be at the constant offset\n",
> + regno);
> + return -EINVAL;
> + }
> + if (!map->btf) {
> + verbose(env,
> + "map '%s' has to have BTF in order to use bpf_spin_lock\n",
> + map->name);
> + return -EINVAL;
> + }
> + if (!map_value_has_spin_lock(map)) {
> + if (map->spin_lock_off == -E2BIG)
> + verbose(env,
> + "map '%s' has more than one 'struct bpf_spin_lock'\n",
> + map->name);
> + else if (map->spin_lock_off == -ENOENT)
> + verbose(env,
> + "map '%s' doesn't have 'struct bpf_spin_lock'\n",
> + map->name);
> + else
> + verbose(env,
> + "map '%s' is not a struct type or bpf_spin_lock is mangled\n",
> + map->name);
> + return -EINVAL;
> + }
> + if (map->spin_lock_off != val + reg->off) {
> + verbose(env, "off %lld doesn't point to 'struct bpf_spin_lock'\n",
> + val + reg->off);
> + return -EINVAL;
> + }
> + if (is_lock) {
> + if (cur->active_spin_lock) {
> + verbose(env,
> + "Locking two bpf_spin_locks are not allowed\n");
> + return -EINVAL;
> + }
> + cur->active_spin_lock = reg->id;
> + } else {
> + if (!cur->active_spin_lock) {
> + verbose(env, "bpf_spin_unlock without taking a lock\n");
> + return -EINVAL;
> + }
> + if (cur->active_spin_lock != reg->id) {
> + verbose(env, "bpf_spin_unlock of different lock\n");
> + return -EINVAL;
> + }
> + cur->active_spin_lock = 0;
> + }
> + return 0;
> +}
> +
> static bool arg_type_is_mem_ptr(enum bpf_arg_type type)
> {
> return type == ARG_PTR_TO_MEM ||
> @@ -2268,6 +2376,17 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno,
> return -EFAULT;
> }
> meta->ptr_id = reg->id;
> + } else if (arg_type == ARG_PTR_TO_SPIN_LOCK) {
> + if (meta->func_id == BPF_FUNC_spin_lock) {
> + if (process_spin_lock(env, regno, true))
> + return -EACCES;
> + } else if (meta->func_id == BPF_FUNC_spin_unlock) {
> + if (process_spin_lock(env, regno, false))
> + return -EACCES;
> + } else {
> + verbose(env, "verifier internal error\n");
> + return -EFAULT;
> + }
> } else if (arg_type_is_mem_ptr(arg_type)) {
> expected_type = PTR_TO_STACK;
> /* One exception here. In case function allows for NULL to be
> @@ -2887,6 +3006,7 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn
> return err;
> }
>
> + meta.func_id = func_id;
> /* check args */
> err = check_func_arg(env, BPF_REG_1, fn->arg1_type, &meta);
> if (err)
> @@ -4344,7 +4464,8 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state,
> } else if (reg->type == PTR_TO_SOCKET_OR_NULL) {
> reg->type = PTR_TO_SOCKET;
> }
> - if (is_null || !reg_is_refcounted(reg)) {
> + if (is_null || !(reg_is_refcounted(reg) ||
> + reg_may_point_to_spin_lock(reg))) {
> /* We don't need id from this point onwards anymore,
> * thus we should better reset it, so that state
> * pruning has chances to take effect.
> @@ -5651,6 +5772,9 @@ static bool states_equal(struct bpf_verifier_env *env,
> if (old->speculative && !cur->speculative)
> return false;
>
> + if (old->active_spin_lock != cur->active_spin_lock)
> + return false;
> +
> /* for states to be equal callsites have to be the same
> * and all frame states need to be equivalent
> */
> @@ -6068,6 +6192,12 @@ static int do_check(struct bpf_verifier_env *env)
> return -EINVAL;
> }
>
> + if (env->cur_state->active_spin_lock &&
> + (insn->src_reg == BPF_PSEUDO_CALL ||
> + insn->imm != BPF_FUNC_spin_unlock)) {
> + verbose(env, "function calls are not allowed while holding a lock\n");
> + return -EINVAL;
> + }
> if (insn->src_reg == BPF_PSEUDO_CALL)
> err = check_func_call(env, insn, &env->insn_idx);
> else
> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
> return -EINVAL;
> }
>
> + if (env->cur_state->active_spin_lock) {
> + verbose(env, "bpf_spin_unlock is missing\n");
> + return -EINVAL;
> + }
> +
> if (state->curframe) {
> /* exit from nested function */
> env->prev_insn_idx = env->insn_idx;
I think if I'm not mistaken there should still be a possibility for causing a
deadlock, namely if in the middle of the critical section I'm using an LD_ABS
or LD_IND instruction with oob index such that I cause an implicit return 0
while lock is held. At least I don't see this being caught, probably also for
such case a test_verifier snippet would be good.
Wouldn't we also need to mark queued spinlock functions as notrace such that
e.g. from kprobe one cannot attach to these causing a deadlock?
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 22:48 ` Daniel Borkmann
@ 2019-01-16 23:16 ` Daniel Borkmann
2019-01-16 23:30 ` Alexei Starovoitov
2019-01-16 23:23 ` Alexei Starovoitov
1 sibling, 1 reply; 20+ messages in thread
From: Daniel Borkmann @ 2019-01-16 23:16 UTC (permalink / raw)
To: Alexei Starovoitov, davem; +Cc: jakub.kicinski, netdev, kernel-team
On 01/16/2019 11:48 PM, Daniel Borkmann wrote:
> On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
[...]
>> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
>> return -EINVAL;
>> }
>>
>> + if (env->cur_state->active_spin_lock) {
>> + verbose(env, "bpf_spin_unlock is missing\n");
>> + return -EINVAL;
>> + }
>> +
>> if (state->curframe) {
>> /* exit from nested function */
>> env->prev_insn_idx = env->insn_idx;
>
> I think if I'm not mistaken there should still be a possibility for causing a
> deadlock, namely if in the middle of the critical section I'm using an LD_ABS
> or LD_IND instruction with oob index such that I cause an implicit return 0
> while lock is held. At least I don't see this being caught, probably also for
> such case a test_verifier snippet would be good.
>
> Wouldn't we also need to mark queued spinlock functions as notrace such that
> e.g. from kprobe one cannot attach to these causing a deadlock?
I think there may be another problem: haven't verified, but it might be possible
at least from reading the code that I have two programs which share a common
array/hash with spin_lock in BTF provided. Program A is properly using spin_lock
as in one of your examples. Program B is using map in map with inner map being
that same map using spin_lock. When we return that fake inner_map_meta as
reg->map_ptr then we can bypass any read/write restrictions into spin_lock area
which is normally prevented by verifier. Meaning, map in map needs to be made
aware of spin_lock case as well.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 22:48 ` Daniel Borkmann
2019-01-16 23:16 ` Daniel Borkmann
@ 2019-01-16 23:23 ` Alexei Starovoitov
1 sibling, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 23:23 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On Wed, Jan 16, 2019 at 11:48:15PM +0100, Daniel Borkmann wrote:
>
> I think if I'm not mistaken there should still be a possibility for causing a
> deadlock, namely if in the middle of the critical section I'm using an LD_ABS
> or LD_IND instruction with oob index such that I cause an implicit return 0
> while lock is held. At least I don't see this being caught, probably also for
> such case a test_verifier snippet would be good.
good catch. My earlier implementation was reusing check_reference_leak()
that is called for both bpf_exit and bpf_ld_abs, but then I realized we cannot
call bpf_exit from callee when lock is held and moved that check before
prepare_func_exit() forgetting about ldabs. argh. Will fix.
> Wouldn't we also need to mark queued spinlock functions as notrace such that
> e.g. from kprobe one cannot attach to these causing a deadlock?
there is recursion check already, so I'm not sure that is necessary, but
will add it since it doesn't hurt and safer indeed.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 23:16 ` Daniel Borkmann
@ 2019-01-16 23:30 ` Alexei Starovoitov
2019-01-17 0:21 ` Daniel Borkmann
0 siblings, 1 reply; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-16 23:30 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On Thu, Jan 17, 2019 at 12:16:44AM +0100, Daniel Borkmann wrote:
> On 01/16/2019 11:48 PM, Daniel Borkmann wrote:
> > On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
> [...]
> >> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
> >> return -EINVAL;
> >> }
> >>
> >> + if (env->cur_state->active_spin_lock) {
> >> + verbose(env, "bpf_spin_unlock is missing\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> if (state->curframe) {
> >> /* exit from nested function */
> >> env->prev_insn_idx = env->insn_idx;
> >
> > I think if I'm not mistaken there should still be a possibility for causing a
> > deadlock, namely if in the middle of the critical section I'm using an LD_ABS
> > or LD_IND instruction with oob index such that I cause an implicit return 0
> > while lock is held. At least I don't see this being caught, probably also for
> > such case a test_verifier snippet would be good.
> >
> > Wouldn't we also need to mark queued spinlock functions as notrace such that
> > e.g. from kprobe one cannot attach to these causing a deadlock?
>
> I think there may be another problem: haven't verified, but it might be possible
> at least from reading the code that I have two programs which share a common
> array/hash with spin_lock in BTF provided. Program A is properly using spin_lock
> as in one of your examples. Program B is using map in map with inner map being
> that same map using spin_lock. When we return that fake inner_map_meta as
> reg->map_ptr then we can bypass any read/write restrictions into spin_lock area
> which is normally prevented by verifier. Meaning, map in map needs to be made
> aware of spin_lock case as well.
2nd great catch. thanks!
Indeed inner_map_meta doesn't preserve all the fields from struct bpf_map.
It seems long term we'll be able to support spin_lock in inner map too,
but for now I'll disable it.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
2019-01-16 22:48 ` Daniel Borkmann
@ 2019-01-17 0:16 ` Martin Lau
2019-01-17 1:02 ` Alexei Starovoitov
1 sibling, 1 reply; 20+ messages in thread
From: Martin Lau @ 2019-01-17 0:16 UTC (permalink / raw)
To: Alexei Starovoitov; +Cc: davem, daniel, jakub.kicinski, netdev, Kernel Team
On Tue, Jan 15, 2019 at 09:08:22PM -0800, Alexei Starovoitov wrote:
[ ... ]
> +/* copy everything but bpf_spin_lock */
> +static inline void copy_map_value(struct bpf_map *map, void *dst, void *src)
> +{
> + if (unlikely(map_value_has_spin_lock(map))) {
> + u32 off = map->spin_lock_off;
> +
> + memcpy(dst, src, off);
> + memcpy(dst + off + sizeof(struct bpf_spin_lock),
> + src + off + sizeof(struct bpf_spin_lock),
> + map->value_size - off - sizeof(struct bpf_spin_lock));
> + } else {
> + memcpy(dst, src, map->value_size);
> + }
> +}
> +
[ ... ]
> +int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t)
> +{
> + const struct btf_member *member;
> + u32 i, off = -ENOENT;
> +
> + if (BTF_INFO_KIND(t->info) != BTF_KIND_STRUCT)
> + return -EINVAL;
> +
> + for_each_member(i, t, member) {
> + const struct btf_type *member_type = btf_type_by_id(btf,
> + member->type);
> + if (!btf_type_is_struct(member_type))
may be using "BTF_INFO_KIND(t->info) != BTF_KIND_STRUCT" here also.
> + continue;
> + if (member_type->size != sizeof(struct bpf_spin_lock))
> + continue;
> + if (strcmp(__btf_name_by_offset(btf, member_type->name_off),
> + "bpf_spin_lock"))
> + continue;
> + if (off != -ENOENT)
> + /* only one 'struct bpf_spin_lock' is allowed */
> + return -E2BIG;
> + off = btf_member_bit_offset(t, member);
> + if (off % 8)
> + /* valid C code cannot generate such BTF */
> + return -EINVAL;
> + off /= 8;
> + if (off % __alignof__(struct bpf_spin_lock))
> + /* valid struct bpf_spin_lock will be 4 byte aligned */
> + return -EINVAL;
> + }
> + return off;
> +}
> +
[ ... ]
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index b155cd17c1bd..ebf0a673cb83 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
[ ... ]
> err = security_bpf_map_alloc(map);
> @@ -740,7 +757,7 @@ static int map_lookup_elem(union bpf_attr *attr)
> err = -ENOENT;
> } else {
> err = 0;
> - memcpy(value, ptr, value_size);
> + copy_map_value(map, value, ptr);
copy_map_value() skips the bpf_spin_lock and "value" has not been zero-ed.
"value" is then copied to the "__user *uvalue".
May be init the bpf_spin_lock part of the "uvalue" to 0?
btw, somehow patch 6 and 7 are missing:
https://patchwork.ozlabs.org/cover/1025640/
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-16 23:30 ` Alexei Starovoitov
@ 2019-01-17 0:21 ` Daniel Borkmann
2019-01-17 1:16 ` Alexei Starovoitov
0 siblings, 1 reply; 20+ messages in thread
From: Daniel Borkmann @ 2019-01-17 0:21 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On 01/17/2019 12:30 AM, Alexei Starovoitov wrote:
> On Thu, Jan 17, 2019 at 12:16:44AM +0100, Daniel Borkmann wrote:
>> On 01/16/2019 11:48 PM, Daniel Borkmann wrote:
>>> On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
>> [...]
>>>> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
>>>> return -EINVAL;
>>>> }
>>>>
>>>> + if (env->cur_state->active_spin_lock) {
>>>> + verbose(env, "bpf_spin_unlock is missing\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> if (state->curframe) {
>>>> /* exit from nested function */
>>>> env->prev_insn_idx = env->insn_idx;
>>>
>>> I think if I'm not mistaken there should still be a possibility for causing a
>>> deadlock, namely if in the middle of the critical section I'm using an LD_ABS
>>> or LD_IND instruction with oob index such that I cause an implicit return 0
>>> while lock is held. At least I don't see this being caught, probably also for
>>> such case a test_verifier snippet would be good.
>>>
>>> Wouldn't we also need to mark queued spinlock functions as notrace such that
>>> e.g. from kprobe one cannot attach to these causing a deadlock?
>>
>> I think there may be another problem: haven't verified, but it might be possible
>> at least from reading the code that I have two programs which share a common
>> array/hash with spin_lock in BTF provided. Program A is properly using spin_lock
>> as in one of your examples. Program B is using map in map with inner map being
>> that same map using spin_lock. When we return that fake inner_map_meta as
>> reg->map_ptr then we can bypass any read/write restrictions into spin_lock area
>> which is normally prevented by verifier. Meaning, map in map needs to be made
>> aware of spin_lock case as well.
>
> 2nd great catch. thanks!
> Indeed inner_map_meta doesn't preserve all the fields from struct bpf_map.
> It seems long term we'll be able to support spin_lock in inner map too,
> but for now I'll disable it.
There's also one more potential issue in pruning I _think_. In regsafe() we
make the basic assumption that for PTR_TO_MAP_VALUE id has been zeroed which
is true up to here, and as such we prune state not taking id into account.
The only other case we have is PTR_TO_SOCKET{,_OR_NULL} which only allows
for exact matches. Potentially there could be a case where you have two map
pointers from different branches but with same basic map properties read/
writing map data, and in first run for PTR_TO_MAP_VALUE w/o spin_lock path
it was considered safe such that we would get a match in regsafe() as well
and could potentially prune the access? I guess definitely worth adding such
test case to test_verifier to make sure.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-17 0:16 ` Martin Lau
@ 2019-01-17 1:02 ` Alexei Starovoitov
0 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-17 1:02 UTC (permalink / raw)
To: Martin Lau
Cc: Alexei Starovoitov, davem, daniel, jakub.kicinski, netdev, Kernel Team
On Thu, Jan 17, 2019 at 12:16:18AM +0000, Martin Lau wrote:
> On Tue, Jan 15, 2019 at 09:08:22PM -0800, Alexei Starovoitov wrote:
> [ ... ]
>
> > +/* copy everything but bpf_spin_lock */
> > +static inline void copy_map_value(struct bpf_map *map, void *dst, void *src)
> > +{
> > + if (unlikely(map_value_has_spin_lock(map))) {
> > + u32 off = map->spin_lock_off;
> > +
> > + memcpy(dst, src, off);
> > + memcpy(dst + off + sizeof(struct bpf_spin_lock),
> > + src + off + sizeof(struct bpf_spin_lock),
> > + map->value_size - off - sizeof(struct bpf_spin_lock));
> > + } else {
> > + memcpy(dst, src, map->value_size);
> > + }
> > +}
> > +
> [ ... ]
>
> > +int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t)
> > +{
> > + const struct btf_member *member;
> > + u32 i, off = -ENOENT;
> > +
> > + if (BTF_INFO_KIND(t->info) != BTF_KIND_STRUCT)
> > + return -EINVAL;
> > +
> > + for_each_member(i, t, member) {
> > + const struct btf_type *member_type = btf_type_by_id(btf,
> > + member->type);
> > + if (!btf_type_is_struct(member_type))
> may be using "BTF_INFO_KIND(t->info) != BTF_KIND_STRUCT" here also.
good point. will do.
> > + continue;
> > + if (member_type->size != sizeof(struct bpf_spin_lock))
> > + continue;
> > + if (strcmp(__btf_name_by_offset(btf, member_type->name_off),
> > + "bpf_spin_lock"))
> > + continue;
> > + if (off != -ENOENT)
> > + /* only one 'struct bpf_spin_lock' is allowed */
> > + return -E2BIG;
> > + off = btf_member_bit_offset(t, member);
> > + if (off % 8)
> > + /* valid C code cannot generate such BTF */
> > + return -EINVAL;
> > + off /= 8;
> > + if (off % __alignof__(struct bpf_spin_lock))
> > + /* valid struct bpf_spin_lock will be 4 byte aligned */
> > + return -EINVAL;
> > + }
> > + return off;
> > +}
> > +
> [ ... ]
>
>
> > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> > index b155cd17c1bd..ebf0a673cb83 100644
> > --- a/kernel/bpf/syscall.c
> > +++ b/kernel/bpf/syscall.c
> [ ... ]
>
> > err = security_bpf_map_alloc(map);
> > @@ -740,7 +757,7 @@ static int map_lookup_elem(union bpf_attr *attr)
> > err = -ENOENT;
> > } else {
> > err = 0;
> > - memcpy(value, ptr, value_size);
> > + copy_map_value(map, value, ptr);
> copy_map_value() skips the bpf_spin_lock and "value" has not been zero-ed.
> "value" is then copied to the "__user *uvalue".
> May be init the bpf_spin_lock part of the "uvalue" to 0?
I guess something went wrong with my scripts.
The patch on my side has this:
if (attr->flags & BPF_F_LOCK) {
/* lock 'ptr' elem and copy
* everything but the lock
*/
copy_map_value_locked(map, value, ptr, true);
/* mask lock, since value was kmalloced */
check_and_init_map_lock(map, value);
} else {
copy_map_value(map, value, ptr);
}
and lock is inited to zero before copying to user space.
But I don't see the same in patchworks, so you're absolutely right
to point it out as a bug.
> btw, somehow patch 6 and 7 are missing:
> https://patchwork.ozlabs.org/cover/1025640/
Indeed and I cannot explain why. Hopefully v2 won't have this weirdness.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-17 0:21 ` Daniel Borkmann
@ 2019-01-17 1:16 ` Alexei Starovoitov
2019-01-17 11:27 ` Daniel Borkmann
0 siblings, 1 reply; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-17 1:16 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On Thu, Jan 17, 2019 at 01:21:32AM +0100, Daniel Borkmann wrote:
> On 01/17/2019 12:30 AM, Alexei Starovoitov wrote:
> > On Thu, Jan 17, 2019 at 12:16:44AM +0100, Daniel Borkmann wrote:
> >> On 01/16/2019 11:48 PM, Daniel Borkmann wrote:
> >>> On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
> >> [...]
> >>>> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
> >>>> return -EINVAL;
> >>>> }
> >>>>
> >>>> + if (env->cur_state->active_spin_lock) {
> >>>> + verbose(env, "bpf_spin_unlock is missing\n");
> >>>> + return -EINVAL;
> >>>> + }
> >>>> +
> >>>> if (state->curframe) {
> >>>> /* exit from nested function */
> >>>> env->prev_insn_idx = env->insn_idx;
> >>>
> >>> I think if I'm not mistaken there should still be a possibility for causing a
> >>> deadlock, namely if in the middle of the critical section I'm using an LD_ABS
> >>> or LD_IND instruction with oob index such that I cause an implicit return 0
> >>> while lock is held. At least I don't see this being caught, probably also for
> >>> such case a test_verifier snippet would be good.
> >>>
> >>> Wouldn't we also need to mark queued spinlock functions as notrace such that
> >>> e.g. from kprobe one cannot attach to these causing a deadlock?
> >>
> >> I think there may be another problem: haven't verified, but it might be possible
> >> at least from reading the code that I have two programs which share a common
> >> array/hash with spin_lock in BTF provided. Program A is properly using spin_lock
> >> as in one of your examples. Program B is using map in map with inner map being
> >> that same map using spin_lock. When we return that fake inner_map_meta as
> >> reg->map_ptr then we can bypass any read/write restrictions into spin_lock area
> >> which is normally prevented by verifier. Meaning, map in map needs to be made
> >> aware of spin_lock case as well.
> >
> > 2nd great catch. thanks!
> > Indeed inner_map_meta doesn't preserve all the fields from struct bpf_map.
> > It seems long term we'll be able to support spin_lock in inner map too,
> > but for now I'll disable it.
>
> There's also one more potential issue in pruning I _think_. In regsafe() we
> make the basic assumption that for PTR_TO_MAP_VALUE id has been zeroed which
> is true up to here, and as such we prune state not taking id into account.
> The only other case we have is PTR_TO_SOCKET{,_OR_NULL} which only allows
> for exact matches. Potentially there could be a case where you have two map
> pointers from different branches but with same basic map properties read/
> writing map data, and in first run for PTR_TO_MAP_VALUE w/o spin_lock path
> it was considered safe such that we would get a match in regsafe() as well
> and could potentially prune the access? I guess definitely worth adding such
> test case to test_verifier to make sure.
Hmm. Something to think about for sure.
I belive if (old->active_spin_lock != cur->active_spin_lock) check
protects from all cases where spin_lock-ed paths are mixed with non-spin.
Like going through non-locked ld/st of map_value in the first pass through
the prog and then jumping half way into that pass after taking spin_lock
to trigger regsafe().
I cannot quite see how to construct such test without triggering
old->active_spin_lock != cur->active_spin_lock
before reaching regsafe().
But I will keep thinking.
If you have more concrete description for the test please suggest.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-17 1:16 ` Alexei Starovoitov
@ 2019-01-17 11:27 ` Daniel Borkmann
2019-01-18 5:51 ` Alexei Starovoitov
0 siblings, 1 reply; 20+ messages in thread
From: Daniel Borkmann @ 2019-01-17 11:27 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On 01/17/2019 02:16 AM, Alexei Starovoitov wrote:
> On Thu, Jan 17, 2019 at 01:21:32AM +0100, Daniel Borkmann wrote:
>> On 01/17/2019 12:30 AM, Alexei Starovoitov wrote:
>>> On Thu, Jan 17, 2019 at 12:16:44AM +0100, Daniel Borkmann wrote:
>>>> On 01/16/2019 11:48 PM, Daniel Borkmann wrote:
>>>>> On 01/16/2019 06:08 AM, Alexei Starovoitov wrote:
>>>> [...]
>>>>>> @@ -6096,6 +6226,11 @@ static int do_check(struct bpf_verifier_env *env)
>>>>>> return -EINVAL;
>>>>>> }
>>>>>>
>>>>>> + if (env->cur_state->active_spin_lock) {
>>>>>> + verbose(env, "bpf_spin_unlock is missing\n");
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> if (state->curframe) {
>>>>>> /* exit from nested function */
>>>>>> env->prev_insn_idx = env->insn_idx;
>>>>>
>>>>> I think if I'm not mistaken there should still be a possibility for causing a
>>>>> deadlock, namely if in the middle of the critical section I'm using an LD_ABS
>>>>> or LD_IND instruction with oob index such that I cause an implicit return 0
>>>>> while lock is held. At least I don't see this being caught, probably also for
>>>>> such case a test_verifier snippet would be good.
>>>>>
>>>>> Wouldn't we also need to mark queued spinlock functions as notrace such that
>>>>> e.g. from kprobe one cannot attach to these causing a deadlock?
>>>>
>>>> I think there may be another problem: haven't verified, but it might be possible
>>>> at least from reading the code that I have two programs which share a common
>>>> array/hash with spin_lock in BTF provided. Program A is properly using spin_lock
>>>> as in one of your examples. Program B is using map in map with inner map being
>>>> that same map using spin_lock. When we return that fake inner_map_meta as
>>>> reg->map_ptr then we can bypass any read/write restrictions into spin_lock area
>>>> which is normally prevented by verifier. Meaning, map in map needs to be made
>>>> aware of spin_lock case as well.
>>>
>>> 2nd great catch. thanks!
>>> Indeed inner_map_meta doesn't preserve all the fields from struct bpf_map.
>>> It seems long term we'll be able to support spin_lock in inner map too,
>>> but for now I'll disable it.
>>
>> There's also one more potential issue in pruning I _think_. In regsafe() we
>> make the basic assumption that for PTR_TO_MAP_VALUE id has been zeroed which
>> is true up to here, and as such we prune state not taking id into account.
>> The only other case we have is PTR_TO_SOCKET{,_OR_NULL} which only allows
>> for exact matches. Potentially there could be a case where you have two map
>> pointers from different branches but with same basic map properties read/
>> writing map data, and in first run for PTR_TO_MAP_VALUE w/o spin_lock path
>> it was considered safe such that we would get a match in regsafe() as well
>> and could potentially prune the access? I guess definitely worth adding such
>> test case to test_verifier to make sure.
>
> Hmm. Something to think about for sure.
> I belive if (old->active_spin_lock != cur->active_spin_lock) check
> protects from all cases where spin_lock-ed paths are mixed with non-spin.
> Like going through non-locked ld/st of map_value in the first pass through
> the prog and then jumping half way into that pass after taking spin_lock
> to trigger regsafe().
> I cannot quite see how to construct such test without triggering
> old->active_spin_lock != cur->active_spin_lock
> before reaching regsafe().
> But I will keep thinking.
> If you have more concrete description for the test please suggest.
Was thinking something like this, in very rough pseudo code:
Prog A (normal spin lock use of mapA):
val = bpf_map_lookup_elem(&mapA, &key);
if (val) {
bpf_spin_lock(&val->lock);
[...]
bpf_spin_unlock(&val->lock);
}
Prog B:
if (non_const_condition_A) {
map_ptr = &mapB; // mapB is normal array map with same
// properties as mapA, but no BTF and
// thus no spinlock use.
} else {
map_ptr = &mapA;
}
val = bpf_map_lookup_elem(&map_ptr, &key);
map_ptr = 0; // clear map reg to match for
// both verification paths
if (val) {
// turning val into PTR_TO_MAP_VALUE
if (non_const_condition_B) {
// write into memory area of spin_lock;
// first path with mapB is considered
// safe (since map with no spin_lock so
// write into this area allowed);
// now when verifier is checking the
// non_const_condition_A's else path
// with mapA, then non_const_condition_B
// has pruning checkpoint and is going
// to compare reg with PTR_TO_MAP_VALUE;
// since id is not considered I /think/
// verifier would find it (wrongly) safe
// as well.
}
}
Wdyt?
Thanks,
Daniel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next 1/9] bpf: introduce bpf_spin_lock
2019-01-17 11:27 ` Daniel Borkmann
@ 2019-01-18 5:51 ` Alexei Starovoitov
0 siblings, 0 replies; 20+ messages in thread
From: Alexei Starovoitov @ 2019-01-18 5:51 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Alexei Starovoitov, davem, jakub.kicinski, netdev, kernel-team
On Thu, Jan 17, 2019 at 12:27:55PM +0100, Daniel Borkmann wrote:
>
> Was thinking something like this, in very rough pseudo code:
>
> Prog A (normal spin lock use of mapA):
>
> val = bpf_map_lookup_elem(&mapA, &key);
> if (val) {
> bpf_spin_lock(&val->lock);
> [...]
> bpf_spin_unlock(&val->lock);
> }
>
> Prog B:
>
> if (non_const_condition_A) {
> map_ptr = &mapB; // mapB is normal array map with same
> // properties as mapA, but no BTF and
> // thus no spinlock use.
> } else {
> map_ptr = &mapA;
> }
> val = bpf_map_lookup_elem(&map_ptr, &key);
> map_ptr = 0; // clear map reg to match for
> // both verification paths
> if (val) {
> // turning val into PTR_TO_MAP_VALUE
> if (non_const_condition_B) {
> // write into memory area of spin_lock;
> // first path with mapB is considered
> // safe (since map with no spin_lock so
> // write into this area allowed);
> // now when verifier is checking the
> // non_const_condition_A's else path
> // with mapA, then non_const_condition_B
> // has pruning checkpoint and is going
> // to compare reg with PTR_TO_MAP_VALUE;
> // since id is not considered I /think/
> // verifier would find it (wrongly) safe
> // as well.
> }
> }
>
> Wdyt?
I've implemented it and it was rejected.
regsafe() is doing:
memcmp(rold, rcur, offsetof(struct bpf_reg_state, id));
'map_ptr' is before 'id' in bpf_reg_state.
lookups from different maps will have different register states
as expected.
I still felt that we need to compare id, so I tried
if (random)
val = bpf_map_lookup_elem(&hash_map, &key1);
else
val = bpf_map_lookup_elem(&hash_map, &key2);
to force pruning of two states that point to the same map.
The val->spin_lock addresses will be different and intuitively it
may feel that we need to compare id, but it's unnecessary.
If the rest of the program is valid with val for key1
it's a good thing to prune verification for key2 to avoid
spending more cycles in the verifier.
All map elements have spin_locks. If bpf code is valid
for one element it's valid for all elements.
Then I tried to trick the verifier with the following:
int flag = 0;
if (random) {
val = bpf_map_lookup_elem(&hash_map, &key1);
} else {
flag = 1;
val = bpf_map_lookup_elem(&hash_map, &key2);
}
if (!val)
goto err;
bpf_spin_lock(&val->lock);
bpf_spin_unlock(&val->lock);
if (flag == 1) // access spin_lock with ld/st
the first pass of the verifier will correctly avoid
exploring 'flag == 1' condition (because the verifier is
smart enough to know that the flag is 0 there), but
the second pass with different reg->id for 'val' will not be pruned,
since the register (or stack) where 'flag' is stored
is different and the verifier will proceed and will
catch ld/st into spin_lock field and the prog will be rejected.
So I believe the verifier should not compare 'id' in regsafe()
for PTR_TO_MAP_VALUE to have better pruning.
I'll add a comment there explaining this.
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2019-01-18 5:51 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-16 5:08 [PATCH bpf-next 0/9] introduce bpf_spin_lock Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 1/9] bpf: " Alexei Starovoitov
2019-01-16 22:48 ` Daniel Borkmann
2019-01-16 23:16 ` Daniel Borkmann
2019-01-16 23:30 ` Alexei Starovoitov
2019-01-17 0:21 ` Daniel Borkmann
2019-01-17 1:16 ` Alexei Starovoitov
2019-01-17 11:27 ` Daniel Borkmann
2019-01-18 5:51 ` Alexei Starovoitov
2019-01-16 23:23 ` Alexei Starovoitov
2019-01-17 0:16 ` Martin Lau
2019-01-17 1:02 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 2/9] bpf: add support for bpf_spin_lock to cgroup local storage Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 3/9] tools/bpf: sync include/uapi/linux/bpf.h Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 4/9] selftests/bpf: add bpf_spin_lock tests Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 5/9] selftests/bpf: add bpf_spin_lock C test Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 8/9] libbpf: introduce bpf_map_lookup_elem_flags() Alexei Starovoitov
2019-01-16 6:18 ` Yonghong Song
2019-01-16 6:28 ` Alexei Starovoitov
2019-01-16 5:08 ` [PATCH bpf-next 9/9] selftests/bpf: test for BPF_F_LOCK Alexei Starovoitov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.