[PATCH v2 bpf-next 00/14] bpf: cgroup local storage

[PATCH v2 bpf-next 00/14] bpf: cgroup local storage

[Roman Gushchin]

This patchset implements cgroup local storage for bpf programs.
The main idea is to provide a fast accessible memory for storing
various per-cgroup data, e.g. number of transmitted packets.

Cgroup local storage looks as a special type of map for userspace,
and is accessible using generic bpf maps API for reading and
updating of the data. The (cgroup inode id, attachment type) pair
is used as a map key.

A user can't create new entries or destroy existing entries;
it happens automatically when a user attaches/detaches a bpf program
to a cgroup.

From a bpf program's point of view, cgroup storage is accessible
without lookup using the special get_local_storage() helper function.
It takes a map fd as an argument. It always returns a valid pointer
to the corresponding memory area.
To implement such a lookup-free access a pointer to the cgroup
storage is saved for an attachment of a bpf program to a cgroup,
if required by the program. Before running the program, it's saved
in a special global per-cpu variable, which is accessible from the
get_local_storage() helper.

This patchset implement only cgroup local storage, however the API
is intentionally made extensible to support other local storage types
further: e.g. thread local storage, socket local storage, etc.

Patch (1) adds an ability to charge bpf maps for consuming memory
dynamically.
Patch (2) introduces cgroup storage maps.
Patch (3) implements a mechanism to pass cgroup storage pointer
to a bpf program.
Patch (4) implements allocation/releasing of cgroup local storage
on attaching/detaching of a bpf program to/from a cgroup.
Patch (5) extends bpf_prog_array to store cgroup storage pointers.
Patch (6) introduces BPF_PTR_TO_MAP_VALUE, required to skip
non-necessary NULL-check in bpf programs.
Patch (7) disables creation of maps of cgroup storage maps.
Patch (8) introduces the get_local_storage() helper.
Patch (9) syncs bpf.h to tools/.
Patch (10) adds cgroup storage maps support to bpftool.
Patch (11) adds support for testing programs which are using
cgroup storage without actually attaching them to cgroups.
Patches (12), (13) and (14) are adding necessary tests.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>

v2->v1:
  - fixed build issues
  - removed explicit rlimit calls in patch 14
  - rebased to bpf-next

Roman Gushchin (14):
  bpf: add ability to charge bpf maps memory dynamically
  bpf: introduce cgroup storage maps
  bpf: pass a pointer to a cgroup storage using pcpu variable
  bpf: allocate cgroup storage entries on attaching bpf programs
  bpf: extend bpf_prog_array to store pointers to the cgroup storage
  bpf/verifier: introduce BPF_PTR_TO_MAP_VALUE
  bpf: don't allow create maps of cgroup local storages
  bpf: introduce the bpf_get_local_storage() helper function
  bpf: sync bpf.h to tools/
  bpftool: add support for CGROUP_STORAGE maps
  bpf/test_run: support cgroup local storage
  selftests/bpf: add verifier cgroup storage tests
  selftests/bpf: add a cgroup storage test
  samples/bpf: extend test_cgrp2_attach2 test to use cgroup storage

 include/linux/bpf-cgroup.h                        |  54 ++++
 include/linux/bpf.h                               |  25 +-
 include/linux/bpf_types.h                         |   3 +
 include/uapi/linux/bpf.h                          |  19 +-
 kernel/bpf/Makefile                               |   1 +
 kernel/bpf/cgroup.c                               |  54 +++-
 kernel/bpf/core.c                                 |  77 ++---
 kernel/bpf/helpers.c                              |  20 ++
 kernel/bpf/local_storage.c                        | 369 ++++++++++++++++++++++
 kernel/bpf/map_in_map.c                           |   3 +-
 kernel/bpf/syscall.c                              |  53 +++-
 kernel/bpf/verifier.c                             |  38 ++-
 net/bpf/test_run.c                                |  13 +-
 net/core/filter.c                                 |  23 +-
 samples/bpf/test_cgrp2_attach2.c                  |  21 +-
 tools/bpf/bpftool/map.c                           |   1 +
 tools/include/uapi/linux/bpf.h                    |   9 +-
 tools/testing/selftests/bpf/Makefile              |   4 +-
 tools/testing/selftests/bpf/bpf_helpers.h         |   2 +
 tools/testing/selftests/bpf/test_cgroup_storage.c | 130 ++++++++
 tools/testing/selftests/bpf/test_verifier.c       | 123 +++++++-
 21 files changed, 961 insertions(+), 81 deletions(-)
 create mode 100644 kernel/bpf/local_storage.c
 create mode 100644 tools/testing/selftests/bpf/test_cgroup_storage.c

-- 
2.14.4

[PATCH v2 bpf-next 01/14] bpf: add ability to charge bpf maps memory dynamically

[Roman Gushchin]

This commits extends existing bpf maps memory charging API
to support dynamic charging/uncharging.

This is required to account memory used by maps,
if all entries are created dynamically after
the map initialization.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf.h  |  2 ++
 kernel/bpf/syscall.c | 53 +++++++++++++++++++++++++++++++++++++---------------
 2 files changed, 40 insertions(+), 15 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 8827e797ff97..3d1933707374 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -434,6 +434,8 @@ struct bpf_map * __must_check bpf_map_inc(struct bpf_map *map, bool uref);
 void bpf_map_put_with_uref(struct bpf_map *map);
 void bpf_map_put(struct bpf_map *map);
 int bpf_map_precharge_memlock(u32 pages);
+int bpf_map_charge_memlock(struct bpf_map *map, u32 pages);
+void bpf_map_uncharge_memlock(struct bpf_map *map, u32 pages);
 void *bpf_map_area_alloc(size_t size, int numa_node);
 void bpf_map_area_free(void *base);
 void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index d10ecd78105f..cee452a19538 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -181,32 +181,55 @@ int bpf_map_precharge_memlock(u32 pages)
 	return 0;
 }
 
-static int bpf_map_charge_memlock(struct bpf_map *map)
+static int bpf_charge_memlock(struct user_struct *user, u32 pages)
 {
-	struct user_struct *user = get_current_user();
-	unsigned long memlock_limit;
+	unsigned long memlock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
 
-	memlock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
+	if (atomic_long_add_return(pages, &user->locked_vm) > memlock_limit) {
+		atomic_long_sub(pages, &user->locked_vm);
+		return -EPERM;
+	}
+	return 0;
+}
 
-	atomic_long_add(map->pages, &user->locked_vm);
+static int bpf_map_init_memlock(struct bpf_map *map)
+{
+	struct user_struct *user = get_current_user();
+	int ret;
 
-	if (atomic_long_read(&user->locked_vm) > memlock_limit) {
-		atomic_long_sub(map->pages, &user->locked_vm);
+	ret = bpf_charge_memlock(user, map->pages);
+	if (ret) {
 		free_uid(user);
-		return -EPERM;
+		return ret;
 	}
 	map->user = user;
-	return 0;
+	return ret;
 }
 
-static void bpf_map_uncharge_memlock(struct bpf_map *map)
+static void bpf_map_release_memlock(struct bpf_map *map)
 {
 	struct user_struct *user = map->user;
-
-	atomic_long_sub(map->pages, &user->locked_vm);
+	atomic_long_sub(map->pages, &map->user->locked_vm);
 	free_uid(user);
 }
 
+int bpf_map_charge_memlock(struct bpf_map *map, u32 pages)
+{
+	int ret;
+
+	ret = bpf_charge_memlock(map->user, pages);
+	if (ret)
+		return ret;
+	map->pages += pages;
+	return ret;
+}
+
+void bpf_map_uncharge_memlock(struct bpf_map *map, u32 pages)
+{
+	atomic_long_sub(pages, &map->user->locked_vm);
+	map->pages -= pages;
+}
+
 static int bpf_map_alloc_id(struct bpf_map *map)
 {
 	int id;
@@ -256,7 +279,7 @@ static void bpf_map_free_deferred(struct work_struct *work)
 {
 	struct bpf_map *map = container_of(work, struct bpf_map, work);
 
-	bpf_map_uncharge_memlock(map);
+	bpf_map_release_memlock(map);
 	security_bpf_map_free(map);
 	/* implementation dependent freeing */
 	map->ops->map_free(map);
@@ -492,7 +515,7 @@ static int map_create(union bpf_attr *attr)
 	if (err)
 		goto free_map_nouncharge;
 
-	err = bpf_map_charge_memlock(map);
+	err = bpf_map_init_memlock(map);
 	if (err)
 		goto free_map_sec;
 
@@ -515,7 +538,7 @@ static int map_create(union bpf_attr *attr)
 	return err;
 
 free_map:
-	bpf_map_uncharge_memlock(map);
+	bpf_map_release_memlock(map);
 free_map_sec:
 	security_bpf_map_free(map);
 free_map_nouncharge:
-- 
2.14.4

[PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[Roman Gushchin]

This commit introduces BPF_MAP_TYPE_CGROUP_STORAGE maps:
a special type of maps which are implementing the cgroup storage.

From the userspace point of view it's almost a generic
hash map with the (cgroup inode id, attachment type) pair
used as a key.

The only difference is that some operations are restricted:
  1) a user can't create new entries,
  2) a user can't remove existing entries.

The lookup from userspace is o(log(n)).

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf-cgroup.h |  38 +++++
 include/linux/bpf.h        |   1 +
 include/linux/bpf_types.h  |   3 +
 include/uapi/linux/bpf.h   |   6 +
 kernel/bpf/Makefile        |   1 +
 kernel/bpf/local_storage.c | 367 +++++++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/verifier.c      |  12 ++
 7 files changed, 428 insertions(+)
 create mode 100644 kernel/bpf/local_storage.c

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 79795c5fa7c3..6b0e7bd4b154 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -3,19 +3,39 @@
 #define _BPF_CGROUP_H
 
 #include <linux/jump_label.h>
+#include <linux/rbtree.h>
 #include <uapi/linux/bpf.h>
 
 struct sock;
 struct sockaddr;
 struct cgroup;
 struct sk_buff;
+struct bpf_map;
+struct bpf_prog;
 struct bpf_sock_ops_kern;
+struct bpf_cgroup_storage;
 
 #ifdef CONFIG_CGROUP_BPF
 
 extern struct static_key_false cgroup_bpf_enabled_key;
 #define cgroup_bpf_enabled static_branch_unlikely(&cgroup_bpf_enabled_key)
 
+struct bpf_cgroup_storage_map;
+
+struct bpf_storage_buffer {
+	struct rcu_head rcu;
+	char data[0];
+};
+
+struct bpf_cgroup_storage {
+	struct bpf_storage_buffer *buf;
+	struct bpf_cgroup_storage_map *map;
+	struct bpf_cgroup_storage_key key;
+	struct list_head list;
+	struct rb_node node;
+	struct rcu_head rcu;
+};
+
 struct bpf_prog_list {
 	struct list_head node;
 	struct bpf_prog *prog;
@@ -76,6 +96,15 @@ int __cgroup_bpf_run_filter_sock_ops(struct sock *sk,
 int __cgroup_bpf_check_dev_permission(short dev_type, u32 major, u32 minor,
 				      short access, enum bpf_attach_type type);
 
+struct bpf_cgroup_storage *bpf_cgroup_storage_alloc(struct bpf_prog *prog);
+void bpf_cgroup_storage_free(struct bpf_cgroup_storage *storage);
+void bpf_cgroup_storage_link(struct bpf_cgroup_storage *storage,
+			     struct cgroup *cgroup,
+			     enum bpf_attach_type type);
+void bpf_cgroup_storage_unlink(struct bpf_cgroup_storage *storage);
+int bpf_cgroup_storage_assign(struct bpf_prog *prog, struct bpf_map *map);
+void bpf_cgroup_storage_release(struct bpf_prog *prog, struct bpf_map *map);
+
 /* Wrappers for __cgroup_bpf_run_filter_skb() guarded by cgroup_bpf_enabled. */
 #define BPF_CGROUP_RUN_PROG_INET_INGRESS(sk, skb)			      \
 ({									      \
@@ -220,6 +249,15 @@ static inline int cgroup_bpf_prog_query(const union bpf_attr *attr,
 	return -EINVAL;
 }
 
+static inline int bpf_cgroup_storage_assign(struct bpf_prog *prog,
+					    struct bpf_map *map) { return 0; }
+static inline void bpf_cgroup_storage_release(struct bpf_prog *prog,
+					      struct bpf_map *map) {}
+static inline struct bpf_cgroup_storage *bpf_cgroup_storage_alloc(
+	struct bpf_prog *prog) { return 0; }
+static inline void bpf_cgroup_storage_free(
+	struct bpf_cgroup_storage *storage) {}
+
 #define cgroup_bpf_enabled (0)
 #define BPF_CGROUP_PRE_CONNECT_ENABLED(sk) (0)
 #define BPF_CGROUP_RUN_PROG_INET_INGRESS(sk,skb) ({ 0; })
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 3d1933707374..1205e81871d9 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -281,6 +281,7 @@ struct bpf_prog_aux {
 	struct bpf_prog *prog;
 	struct user_struct *user;
 	u64 load_time; /* ns since boottime */
+	struct bpf_map *cgroup_storage;
 	char name[BPF_OBJ_NAME_LEN];
 #ifdef CONFIG_SECURITY
 	void *security;
diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
index c5700c2d5549..add08be53b6f 100644
--- a/include/linux/bpf_types.h
+++ b/include/linux/bpf_types.h
@@ -37,6 +37,9 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_PERF_EVENT_ARRAY, perf_event_array_map_ops)
 #ifdef CONFIG_CGROUPS
 BPF_MAP_TYPE(BPF_MAP_TYPE_CGROUP_ARRAY, cgroup_array_map_ops)
 #endif
+#ifdef CONFIG_CGROUP_BPF
+BPF_MAP_TYPE(BPF_MAP_TYPE_CGROUP_STORAGE, cgroup_storage_map_ops)
+#endif
 BPF_MAP_TYPE(BPF_MAP_TYPE_HASH, htab_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_PERCPU_HASH, htab_percpu_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_LRU_HASH, htab_lru_map_ops)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index b7db3261c62d..6f9b907a8a0e 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -75,6 +75,11 @@ struct bpf_lpm_trie_key {
 	__u8	data[0];	/* Arbitrary size */
 };
 
+struct bpf_cgroup_storage_key {
+	__u64	cgroup_inode_id;	/* cgroup inode id */
+	__u32	attach_type;		/* program attach type */
+};
+
 /* BPF syscall commands, see bpf(2) man-page for details. */
 enum bpf_cmd {
 	BPF_MAP_CREATE,
@@ -120,6 +125,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_CPUMAP,
 	BPF_MAP_TYPE_XSKMAP,
 	BPF_MAP_TYPE_SOCKHASH,
+	BPF_MAP_TYPE_CGROUP_STORAGE,
 };
 
 enum bpf_prog_type {
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index f27f5496d6fe..e8906cbad81f 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -3,6 +3,7 @@ obj-y := core.o
 
 obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o
 obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o
+obj-$(CONFIG_BPF_SYSCALL) += local_storage.o
 obj-$(CONFIG_BPF_SYSCALL) += disasm.o
 obj-$(CONFIG_BPF_SYSCALL) += btf.o
 ifeq ($(CONFIG_NET),y)
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
new file mode 100644
index 000000000000..940889eda2c7
--- /dev/null
+++ b/kernel/bpf/local_storage.c
@@ -0,0 +1,367 @@
+//SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf-cgroup.h>
+#include <linux/bpf.h>
+#include <linux/bug.h>
+#include <linux/filter.h>
+#include <linux/mm.h>
+#include <linux/rbtree.h>
+#include <linux/slab.h>
+
+#ifdef CONFIG_CGROUP_BPF
+
+struct bpf_cgroup_storage_map {
+	struct bpf_map map;
+	struct bpf_prog *prog;
+
+	spinlock_t lock;
+	struct rb_root root;
+	struct list_head list;
+};
+
+static struct bpf_cgroup_storage_map *map_to_storage(struct bpf_map *map)
+{
+	return container_of(map, struct bpf_cgroup_storage_map, map);
+}
+
+static int bpf_cgroup_storage_key_cmp(
+	const struct bpf_cgroup_storage_key *key1,
+	const struct bpf_cgroup_storage_key *key2)
+{
+	if (key1->cgroup_inode_id < key2->cgroup_inode_id)
+		return -1;
+	else if (key1->cgroup_inode_id > key2->cgroup_inode_id)
+		return 1;
+	else if (key1->attach_type < key2->attach_type)
+		return -1;
+	else if (key1->attach_type > key2->attach_type)
+		return 1;
+	return 0;
+}
+
+static struct bpf_cgroup_storage *cgroup_storage_lookup(
+	struct bpf_cgroup_storage_map *map, struct bpf_cgroup_storage_key *key,
+	bool locked)
+{
+	struct rb_root *root = &map->root;
+	struct rb_node *node;
+
+	/*
+	 * This lock protects rbtree and list of storage entries,
+	 * which are used from the syscall context only.
+	 * So, simple spin_lock()/unlock() is fine here.
+	 */
+	if (!locked)
+		spin_lock(&map->lock);
+
+	node = root->rb_node;
+	while (node) {
+		struct bpf_cgroup_storage *storage;
+
+		storage = container_of(node, struct bpf_cgroup_storage, node);
+
+		switch (bpf_cgroup_storage_key_cmp(key, &storage->key)) {
+		case -1:
+			node = node->rb_left;
+			break;
+		case 1:
+			node = node->rb_right;
+			break;
+		default:
+			if (!locked)
+				spin_unlock(&map->lock);
+			return storage;
+		}
+	}
+
+	if (!locked)
+		spin_unlock(&map->lock);
+
+	return NULL;
+}
+
+static int cgroup_storage_insert(struct bpf_cgroup_storage_map *map,
+				 struct bpf_cgroup_storage *storage)
+{
+	struct rb_root *root = &map->root;
+	struct rb_node **new = &(root->rb_node), *parent = NULL;
+
+	while (*new) {
+		struct bpf_cgroup_storage *this;
+
+		this = container_of(*new, struct bpf_cgroup_storage, node);
+
+		parent = *new;
+		switch (bpf_cgroup_storage_key_cmp(&storage->key, &this->key)) {
+		case -1:
+			new = &((*new)->rb_left);
+			break;
+		case 1:
+			new = &((*new)->rb_right);
+			break;
+		default:
+			return -EEXIST;
+		}
+	}
+
+	rb_link_node(&storage->node, parent, new);
+	rb_insert_color(&storage->node, root);
+
+	return 0;
+}
+
+static void *cgroup_storage_lookup_elem(struct bpf_map *_map, void *_key)
+{
+	struct bpf_cgroup_storage_map *map = map_to_storage(_map);
+	struct bpf_cgroup_storage_key *key = _key;
+	struct bpf_cgroup_storage *storage;
+
+	storage = cgroup_storage_lookup(map, key, false);
+	if (!storage)
+		return NULL;
+
+	return &READ_ONCE(storage->buf)->data[0];
+}
+
+static int cgroup_storage_update_elem(struct bpf_map *map, void *_key,
+				      void *value, u64 flags)
+{
+	struct bpf_cgroup_storage_key *key = _key;
+	struct bpf_cgroup_storage *storage;
+	struct bpf_storage_buffer *new;
+
+	if (flags & BPF_NOEXIST)
+		return -EINVAL;
+
+	storage = cgroup_storage_lookup((struct bpf_cgroup_storage_map *)map,
+					key, false);
+	if (!storage)
+		return -ENOENT;
+
+	new = kmalloc_node(sizeof(struct bpf_storage_buffer) +
+			   map->value_size, __GFP_ZERO | GFP_USER,
+			   map->numa_node);
+	if (!new)
+		return -ENOMEM;
+
+	memcpy(&new->data[0], value, map->value_size);
+
+	new = xchg(&storage->buf, new);
+	kfree_rcu(new, rcu);
+
+	return 0;
+}
+
+static int cgroup_storage_get_next_key(struct bpf_map *_map, void *_key,
+				       void *_next_key)
+{
+	struct bpf_cgroup_storage_map *map = map_to_storage(_map);
+	struct bpf_cgroup_storage_key *key = _key;
+	struct bpf_cgroup_storage_key *next = _next_key;
+	struct bpf_cgroup_storage *storage;
+
+	spin_lock(&map->lock);
+
+	if (list_empty(&map->list))
+		goto enoent;
+
+	if (key) {
+		storage = cgroup_storage_lookup(map, key, true);
+		if (!storage)
+			goto enoent;
+
+		storage = list_next_entry(storage, list);
+		if (!storage)
+			goto enoent;
+	} else {
+		storage = list_first_entry(&map->list,
+					 struct bpf_cgroup_storage, list);
+	}
+
+	spin_unlock(&map->lock);
+	next->attach_type = storage->key.attach_type;
+	next->cgroup_inode_id = storage->key.cgroup_inode_id;
+	return 0;
+
+enoent:
+	spin_unlock(&map->lock);
+	return -ENOENT;
+}
+
+static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr)
+{
+	int numa_node = bpf_map_attr_numa_node(attr);
+	struct bpf_cgroup_storage_map *map;
+
+	if (attr->key_size != sizeof(struct bpf_cgroup_storage_key))
+		return ERR_PTR(-EINVAL);
+
+	if (attr->value_size > PAGE_SIZE)
+		return ERR_PTR(-E2BIG);
+
+	map = kmalloc_node(sizeof(struct bpf_cgroup_storage_map),
+			   __GFP_ZERO | GFP_USER, numa_node);
+	if (!map)
+		return ERR_PTR(-ENOMEM);
+
+	map->map.pages = round_up(sizeof(struct bpf_cgroup_storage_map),
+				  PAGE_SIZE) >> PAGE_SHIFT;
+
+	/* copy mandatory map attributes */
+	bpf_map_init_from_attr(&map->map, attr);
+
+	spin_lock_init(&map->lock);
+	map->root = RB_ROOT;
+	INIT_LIST_HEAD(&map->list);
+
+	return &map->map;
+}
+
+static void cgroup_storage_map_free(struct bpf_map *_map)
+{
+	struct bpf_cgroup_storage_map *map = map_to_storage(_map);
+
+	WARN_ON(!RB_EMPTY_ROOT(&map->root));
+	WARN_ON(!list_empty(&map->list));
+
+	kfree(map);
+}
+
+static int cgroup_storage_delete_elem(struct bpf_map *map, void *key)
+{
+	return -EINVAL;
+}
+
+const struct bpf_map_ops cgroup_storage_map_ops = {
+	.map_alloc = cgroup_storage_map_alloc,
+	.map_free = cgroup_storage_map_free,
+	.map_get_next_key = cgroup_storage_get_next_key,
+	.map_lookup_elem = cgroup_storage_lookup_elem,
+	.map_update_elem = cgroup_storage_update_elem,
+	.map_delete_elem = cgroup_storage_delete_elem,
+};
+
+/*
+ * Called by the verifier. bpf_verifier_lock must be locked.
+ */
+int bpf_cgroup_storage_assign(struct bpf_prog *prog, struct bpf_map *_map)
+{
+	struct bpf_cgroup_storage_map *map = map_to_storage(_map);
+
+	if (map->prog && map->prog != prog)
+		return -EBUSY;
+	if (prog->aux->cgroup_storage && prog->aux->cgroup_storage != _map)
+		return -EBUSY;
+
+	map->prog = prog;
+	prog->aux->cgroup_storage = _map;
+
+	return 0;
+}
+
+/*
+ * Called by the verifier. bpf_verifier_lock must be locked.
+ */
+void bpf_cgroup_storage_release(struct bpf_prog *prog, struct bpf_map *_map)
+{
+	struct bpf_cgroup_storage_map *map = map_to_storage(_map);
+
+	if (map->prog == prog) {
+		WARN_ON(prog->aux->cgroup_storage != _map);
+		map->prog = NULL;
+	}
+}
+
+struct bpf_cgroup_storage *bpf_cgroup_storage_alloc(struct bpf_prog *prog)
+{
+	struct bpf_cgroup_storage *storage;
+	struct bpf_map *map;
+	u32 pages;
+
+	map = prog->aux->cgroup_storage;
+	if (!map)
+		return NULL;
+
+	pages = round_up(sizeof(struct bpf_cgroup_storage) +
+			 sizeof(struct bpf_storage_buffer) +
+			 map->value_size, PAGE_SIZE) >> PAGE_SHIFT;
+	if (bpf_map_charge_memlock(map, pages))
+		return ERR_PTR(-EPERM);
+
+	storage = kmalloc_node(sizeof(struct bpf_cgroup_storage),
+			       __GFP_ZERO | GFP_USER, map->numa_node);
+	if (!storage) {
+		bpf_map_uncharge_memlock(map, pages);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	storage->buf = kmalloc_node(sizeof(struct bpf_storage_buffer) +
+				    map->value_size, __GFP_ZERO | GFP_USER,
+				    map->numa_node);
+	if (!storage->buf) {
+		bpf_map_uncharge_memlock(map, pages);
+		kfree(storage);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	storage->map = (struct bpf_cgroup_storage_map *)map;
+
+	return storage;
+}
+
+void bpf_cgroup_storage_free(struct bpf_cgroup_storage *storage)
+{
+	u32 pages;
+	struct bpf_map *map;
+
+	if (!storage)
+		return;
+
+	map = &storage->map->map;
+	pages = round_up(sizeof(struct bpf_cgroup_storage) +
+			 sizeof(struct bpf_storage_buffer) +
+			 map->value_size, PAGE_SIZE) >> PAGE_SHIFT;
+	bpf_map_uncharge_memlock(map, pages);
+
+	kfree_rcu(storage->buf, rcu);
+	kfree_rcu(storage, rcu);
+}
+
+void bpf_cgroup_storage_link(struct bpf_cgroup_storage *storage,
+			     struct cgroup *cgroup,
+			     enum bpf_attach_type type)
+{
+	struct bpf_cgroup_storage_map *map;
+
+	if (!storage)
+		return;
+
+	storage->key.attach_type = type;
+	storage->key.cgroup_inode_id = cgroup->kn->id.id;
+
+	map = storage->map;
+
+	spin_lock(&map->lock);
+	WARN_ON(cgroup_storage_insert(map, storage));
+	list_add(&storage->list, &map->list);
+	spin_unlock(&map->lock);
+}
+
+void bpf_cgroup_storage_unlink(struct bpf_cgroup_storage *storage)
+{
+	struct bpf_cgroup_storage_map *map;
+	struct rb_root *root;
+
+	if (!storage)
+		return;
+
+	map = storage->map;
+
+	spin_lock(&map->lock);
+	root = &map->root;
+	rb_erase(&storage->node, root);
+
+	list_del(&storage->list);
+	spin_unlock(&map->lock);
+}
+
+#endif
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e2bf834f13a..de097a642c3f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5140,6 +5140,14 @@ static int replace_map_fd_with_map_ptr(struct bpf_verifier_env *env)
 				return -E2BIG;
 			}
 
+			if (map->map_type == BPF_MAP_TYPE_CGROUP_STORAGE &&
+			    bpf_cgroup_storage_assign(env->prog, map)) {
+				verbose(env,
+					"only one cgroup storage is allowed\n");
+				fdput(f);
+				return -EBUSY;
+			}
+
 			/* hold the map. If the program is rejected by verifier,
 			 * the map will be released by release_maps() or it
 			 * will be used by the valid program until it's unloaded
@@ -5148,6 +5156,10 @@ static int replace_map_fd_with_map_ptr(struct bpf_verifier_env *env)
 			map = bpf_map_inc(map, false);
 			if (IS_ERR(map)) {
 				fdput(f);
+				if (map->map_type ==
+				    BPF_MAP_TYPE_CGROUP_STORAGE)
+					bpf_cgroup_storage_release(env->prog,
+								   map);
 				return PTR_ERR(map);
 			}
 			env->used_maps[env->used_map_cnt++] = map;
-- 
2.14.4

[PATCH v2 bpf-next 03/14] bpf: pass a pointer to a cgroup storage using pcpu variable

[Roman Gushchin]

This commit introduces the bpf_cgroup_storage_set() helper,
which will be used to pass a pointer to a cgroup storage
to the bpf helper.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf-cgroup.h | 15 +++++++++++++++
 kernel/bpf/local_storage.c |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 6b0e7bd4b154..ce77a6dffb6d 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -3,6 +3,7 @@
 #define _BPF_CGROUP_H
 
 #include <linux/jump_label.h>
+#include <linux/percpu-defs.h>
 #include <linux/rbtree.h>
 #include <uapi/linux/bpf.h>
 
@@ -20,6 +21,8 @@ struct bpf_cgroup_storage;
 extern struct static_key_false cgroup_bpf_enabled_key;
 #define cgroup_bpf_enabled static_branch_unlikely(&cgroup_bpf_enabled_key)
 
+DECLARE_PER_CPU(void*, bpf_cgroup_storage);
+
 struct bpf_cgroup_storage_map;
 
 struct bpf_storage_buffer {
@@ -96,6 +99,17 @@ int __cgroup_bpf_run_filter_sock_ops(struct sock *sk,
 int __cgroup_bpf_check_dev_permission(short dev_type, u32 major, u32 minor,
 				      short access, enum bpf_attach_type type);
 
+static inline void bpf_cgroup_storage_set(struct bpf_cgroup_storage *storage)
+{
+	struct bpf_storage_buffer *buf;
+
+	if (!storage)
+		return;
+
+	buf = rcu_dereference(storage->buf);
+	this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
+}
+
 struct bpf_cgroup_storage *bpf_cgroup_storage_alloc(struct bpf_prog *prog);
 void bpf_cgroup_storage_free(struct bpf_cgroup_storage *storage);
 void bpf_cgroup_storage_link(struct bpf_cgroup_storage *storage,
@@ -249,6 +263,7 @@ static inline int cgroup_bpf_prog_query(const union bpf_attr *attr,
 	return -EINVAL;
 }
 
+static inline void bpf_cgroup_storage_set(struct bpf_cgroup_storage *storage) {}
 static inline int bpf_cgroup_storage_assign(struct bpf_prog *prog,
 					    struct bpf_map *map) { return 0; }
 static inline void bpf_cgroup_storage_release(struct bpf_prog *prog,
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
index 940889eda2c7..38810a712971 100644
--- a/kernel/bpf/local_storage.c
+++ b/kernel/bpf/local_storage.c
@@ -7,6 +7,8 @@
 #include <linux/rbtree.h>
 #include <linux/slab.h>
 
+DEFINE_PER_CPU(void*, bpf_cgroup_storage);
+
 #ifdef CONFIG_CGROUP_BPF
 
 struct bpf_cgroup_storage_map {
-- 
2.14.4

[PATCH v2 bpf-next 04/14] bpf: allocate cgroup storage entries on attaching bpf programs

[Roman Gushchin]

If a bpf program is using cgroup local storage, allocate
a bpf_cgroup_storage structure automatically on attaching the program
to a cgroup and save the pointer into the corresponding bpf_prog_list
entry.
Analogically, release the cgroup local storage on detaching
of the bpf program.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf-cgroup.h |  1 +
 kernel/bpf/cgroup.c        | 28 ++++++++++++++++++++++++++--
 2 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index ce77a6dffb6d..1d2167deeb6c 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -42,6 +42,7 @@ struct bpf_cgroup_storage {
 struct bpf_prog_list {
 	struct list_head node;
 	struct bpf_prog *prog;
+	struct bpf_cgroup_storage *storage;
 };
 
 struct bpf_prog_array;
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 3d83ee7df381..808497a7c911 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -34,6 +34,8 @@ void cgroup_bpf_put(struct cgroup *cgrp)
 		list_for_each_entry_safe(pl, tmp, progs, node) {
 			list_del(&pl->node);
 			bpf_prog_put(pl->prog);
+			bpf_cgroup_storage_unlink(pl->storage);
+			bpf_cgroup_storage_free(pl->storage);
 			kfree(pl);
 			static_branch_dec(&cgroup_bpf_enabled_key);
 		}
@@ -189,6 +191,7 @@ int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
 {
 	struct list_head *progs = &cgrp->bpf.progs[type];
 	struct bpf_prog *old_prog = NULL;
+	struct bpf_cgroup_storage *storage, *old_storage = NULL;
 	struct cgroup_subsys_state *css;
 	struct bpf_prog_list *pl;
 	bool pl_was_allocated;
@@ -211,6 +214,10 @@ int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
 	if (prog_list_length(progs) >= BPF_CGROUP_MAX_PROGS)
 		return -E2BIG;
 
+	storage = bpf_cgroup_storage_alloc(prog);
+	if (IS_ERR(storage))
+		return -ENOMEM;
+
 	if (flags & BPF_F_ALLOW_MULTI) {
 		list_for_each_entry(pl, progs, node)
 			if (pl->prog == prog)
@@ -218,24 +225,33 @@ int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
 				return -EINVAL;
 
 		pl = kmalloc(sizeof(*pl), GFP_KERNEL);
-		if (!pl)
+		if (!pl) {
+			bpf_cgroup_storage_free(storage);
 			return -ENOMEM;
+		}
+
 		pl_was_allocated = true;
 		pl->prog = prog;
+		pl->storage = storage;
 		list_add_tail(&pl->node, progs);
 	} else {
 		if (list_empty(progs)) {
 			pl = kmalloc(sizeof(*pl), GFP_KERNEL);
-			if (!pl)
+			if (!pl) {
+				bpf_cgroup_storage_free(storage);
 				return -ENOMEM;
+			}
 			pl_was_allocated = true;
 			list_add_tail(&pl->node, progs);
 		} else {
 			pl = list_first_entry(progs, typeof(*pl), node);
 			old_prog = pl->prog;
+			old_storage = pl->storage;
+			bpf_cgroup_storage_unlink(old_storage);
 			pl_was_allocated = false;
 		}
 		pl->prog = prog;
+		pl->storage = storage;
 	}
 
 	cgrp->bpf.flags[type] = flags;
@@ -258,10 +274,13 @@ int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
 	}
 
 	static_branch_inc(&cgroup_bpf_enabled_key);
+	if (old_storage)
+		bpf_cgroup_storage_free(old_storage);
 	if (old_prog) {
 		bpf_prog_put(old_prog);
 		static_branch_dec(&cgroup_bpf_enabled_key);
 	}
+	bpf_cgroup_storage_link(storage, cgrp, type);
 	return 0;
 
 cleanup:
@@ -277,6 +296,9 @@ int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
 
 	/* and cleanup the prog list */
 	pl->prog = old_prog;
+	bpf_cgroup_storage_free(pl->storage);
+	pl->storage = old_storage;
+	bpf_cgroup_storage_link(old_storage, cgrp, type);
 	if (pl_was_allocated) {
 		list_del(&pl->node);
 		kfree(pl);
@@ -357,6 +379,8 @@ int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
 
 	/* now can actually delete it from this cgroup list */
 	list_del(&pl->node);
+	bpf_cgroup_storage_unlink(pl->storage);
+	bpf_cgroup_storage_free(pl->storage);
 	kfree(pl);
 	if (list_empty(progs))
 		/* last program was detached, reset flags to zero */
-- 
2.14.4

[PATCH v2 bpf-next 05/14] bpf: extend bpf_prog_array to store pointers to the cgroup storage

[Roman Gushchin]

This patch converts bpf_prog_array from an array of prog pointers
to the array of struct bpf_prog_array_item elements.

This allows to save a cgroup storage pointer for each bpf program
efficiently attached to a cgroup.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf.h | 19 +++++++++-----
 kernel/bpf/cgroup.c | 24 ++++++++++-------
 kernel/bpf/core.c   | 76 +++++++++++++++++++++++++++--------------------------
 3 files changed, 66 insertions(+), 53 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 1205e81871d9..31a94a842449 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -348,9 +348,14 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
  * The 'struct bpf_prog_array *' should only be replaced with xchg()
  * since other cpus are walking the array of pointers in parallel.
  */
+struct bpf_prog_array_item {
+	struct bpf_prog *prog;
+	struct bpf_cgroup_storage *cgroup_storage;
+};
+
 struct bpf_prog_array {
 	struct rcu_head rcu;
-	struct bpf_prog *progs[0];
+	struct bpf_prog_array_item items[0];
 };
 
 struct bpf_prog_array __rcu *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
@@ -371,7 +376,8 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array,
 
 #define __BPF_PROG_RUN_ARRAY(array, ctx, func, check_non_null)	\
 	({						\
-		struct bpf_prog **_prog, *__prog;	\
+		struct bpf_prog_array_item *_item;	\
+		struct bpf_prog *_prog;			\
 		struct bpf_prog_array *_array;		\
 		u32 _ret = 1;				\
 		preempt_disable();			\
@@ -379,10 +385,11 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array,
 		_array = rcu_dereference(array);	\
 		if (unlikely(check_non_null && !_array))\
 			goto _out;			\
-		_prog = _array->progs;			\
-		while ((__prog = READ_ONCE(*_prog))) {	\
-			_ret &= func(__prog, ctx);	\
-			_prog++;			\
+		_item = &_array->items[0];		\
+		while ((_prog = READ_ONCE(_item->prog))) {		\
+			bpf_cgroup_storage_set(_item->cgroup_storage);	\
+			_ret &= func(_prog, ctx);	\
+			_item++;			\
 		}					\
 _out:							\
 		rcu_read_unlock();			\
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 808497a7c911..dd232a276668 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -117,16 +117,20 @@ static int compute_effective_progs(struct cgroup *cgrp,
 	cnt = 0;
 	p = cgrp;
 	do {
-		if (cnt == 0 || (p->bpf.flags[type] & BPF_F_ALLOW_MULTI))
-			list_for_each_entry(pl,
-					    &p->bpf.progs[type], node) {
-				if (!pl->prog)
-					continue;
-				rcu_dereference_protected(progs, 1)->
-					progs[cnt++] = pl->prog;
-			}
-		p = cgroup_parent(p);
-	} while (p);
+		if (cnt > 0 && !(p->bpf.flags[type] & BPF_F_ALLOW_MULTI))
+			continue;
+
+		list_for_each_entry(pl, &p->bpf.progs[type], node) {
+			if (!pl->prog)
+				continue;
+
+			rcu_dereference_protected(progs, 1)->
+				items[cnt].prog = pl->prog;
+			rcu_dereference_protected(progs, 1)->
+				items[cnt].cgroup_storage = pl->storage;
+			cnt++;
+		}
+	} while ((p = cgroup_parent(p)));
 
 	*array = progs;
 	return 0;
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 1e5625d46414..5903be928722 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1542,7 +1542,8 @@ struct bpf_prog_array __rcu *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags)
 {
 	if (prog_cnt)
 		return kzalloc(sizeof(struct bpf_prog_array) +
-			       sizeof(struct bpf_prog *) * (prog_cnt + 1),
+			       sizeof(struct bpf_prog_array_item) *
+			       (prog_cnt + 1),
 			       flags);
 
 	return &empty_prog_array.hdr;
@@ -1556,43 +1557,45 @@ void bpf_prog_array_free(struct bpf_prog_array __rcu *progs)
 	kfree_rcu(progs, rcu);
 }
 
-int bpf_prog_array_length(struct bpf_prog_array __rcu *progs)
+int bpf_prog_array_length(struct bpf_prog_array __rcu *array)
 {
-	struct bpf_prog **prog;
+	struct bpf_prog_array_item *item;
 	u32 cnt = 0;
 
 	rcu_read_lock();
-	prog = rcu_dereference(progs)->progs;
-	for (; *prog; prog++)
-		if (*prog != &dummy_bpf_prog.prog)
+	item = rcu_dereference(array)->items;
+	for (; item->prog; item++)
+		if (item->prog != &dummy_bpf_prog.prog)
 			cnt++;
 	rcu_read_unlock();
 	return cnt;
 }
 
-static bool bpf_prog_array_copy_core(struct bpf_prog **prog,
+
+static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
 				     u32 *prog_ids,
 				     u32 request_cnt)
 {
+	struct bpf_prog_array_item *item;
 	int i = 0;
 
-	for (; *prog; prog++) {
-		if (*prog == &dummy_bpf_prog.prog)
+	item = rcu_dereference(array)->items;
+	for (; item->prog; item++) {
+		if (item->prog == &dummy_bpf_prog.prog)
 			continue;
-		prog_ids[i] = (*prog)->aux->id;
+		prog_ids[i] = item->prog->aux->id;
 		if (++i == request_cnt) {
-			prog++;
+			item++;
 			break;
 		}
 	}
 
-	return !!(*prog);
+	return !!(item->prog);
 }
 
-int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *progs,
+int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *array,
 				__u32 __user *prog_ids, u32 cnt)
 {
-	struct bpf_prog **prog;
 	unsigned long err = 0;
 	bool nospc;
 	u32 *ids;
@@ -1611,8 +1614,7 @@ int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *progs,
 	if (!ids)
 		return -ENOMEM;
 	rcu_read_lock();
-	prog = rcu_dereference(progs)->progs;
-	nospc = bpf_prog_array_copy_core(prog, ids, cnt);
+	nospc = bpf_prog_array_copy_core(array, ids, cnt);
 	rcu_read_unlock();
 	err = copy_to_user(prog_ids, ids, cnt * sizeof(u32));
 	kfree(ids);
@@ -1623,14 +1625,14 @@ int bpf_prog_array_copy_to_user(struct bpf_prog_array __rcu *progs,
 	return 0;
 }
 
-void bpf_prog_array_delete_safe(struct bpf_prog_array __rcu *progs,
+void bpf_prog_array_delete_safe(struct bpf_prog_array __rcu *array,
 				struct bpf_prog *old_prog)
 {
-	struct bpf_prog **prog = progs->progs;
+	struct bpf_prog_array_item *item = array->items;
 
-	for (; *prog; prog++)
-		if (*prog == old_prog) {
-			WRITE_ONCE(*prog, &dummy_bpf_prog.prog);
+	for (; item->prog; item++)
+		if (item->prog == old_prog) {
+			WRITE_ONCE(item->prog, &dummy_bpf_prog.prog);
 			break;
 		}
 }
@@ -1641,7 +1643,7 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array,
 			struct bpf_prog_array **new_array)
 {
 	int new_prog_cnt, carry_prog_cnt = 0;
-	struct bpf_prog **existing_prog;
+	struct bpf_prog_array_item *existing;
 	struct bpf_prog_array *array;
 	bool found_exclude = false;
 	int new_prog_idx = 0;
@@ -1650,15 +1652,15 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array,
 	 * the new array.
 	 */
 	if (old_array) {
-		existing_prog = old_array->progs;
-		for (; *existing_prog; existing_prog++) {
-			if (*existing_prog == exclude_prog) {
+		existing = old_array->items;
+		for (; existing->prog; existing++) {
+			if (existing->prog == exclude_prog) {
 				found_exclude = true;
 				continue;
 			}
-			if (*existing_prog != &dummy_bpf_prog.prog)
+			if (existing->prog != &dummy_bpf_prog.prog)
 				carry_prog_cnt++;
-			if (*existing_prog == include_prog)
+			if (existing->prog == include_prog)
 				return -EEXIST;
 		}
 	}
@@ -1684,15 +1686,17 @@ int bpf_prog_array_copy(struct bpf_prog_array __rcu *old_array,
 
 	/* Fill in the new prog array */
 	if (carry_prog_cnt) {
-		existing_prog = old_array->progs;
-		for (; *existing_prog; existing_prog++)
-			if (*existing_prog != exclude_prog &&
-			    *existing_prog != &dummy_bpf_prog.prog)
-				array->progs[new_prog_idx++] = *existing_prog;
+		existing = old_array->items;
+		for (; existing->prog; existing++)
+			if (existing->prog != exclude_prog &&
+			    existing->prog != &dummy_bpf_prog.prog) {
+				array->items[new_prog_idx++].prog =
+					existing->prog;
+			}
 	}
 	if (include_prog)
-		array->progs[new_prog_idx++] = include_prog;
-	array->progs[new_prog_idx] = NULL;
+		array->items[new_prog_idx++].prog = include_prog;
+	array->items[new_prog_idx].prog = NULL;
 	*new_array = array;
 	return 0;
 }
@@ -1701,7 +1705,6 @@ int bpf_prog_array_copy_info(struct bpf_prog_array __rcu *array,
 			     u32 *prog_ids, u32 request_cnt,
 			     u32 *prog_cnt)
 {
-	struct bpf_prog **prog;
 	u32 cnt = 0;
 
 	if (array)
@@ -1714,8 +1717,7 @@ int bpf_prog_array_copy_info(struct bpf_prog_array __rcu *array,
 		return 0;
 
 	/* this function is called under trace/bpf_trace.c: bpf_event_mutex */
-	prog = rcu_dereference_check(array, 1)->progs;
-	return bpf_prog_array_copy_core(prog, prog_ids, request_cnt) ? -ENOSPC
+	return bpf_prog_array_copy_core(array, prog_ids, request_cnt) ? -ENOSPC
 								     : 0;
 }
 
-- 
2.14.4

[PATCH v2 bpf-next 06/14] bpf/verifier: introduce BPF_PTR_TO_MAP_VALUE

[Roman Gushchin]

BPF_MAP_TYPE_CGROUP_STORAGE maps are special in a way
that the access from the bpf program side is lookup-free.
That means the result is guaranteed to be a valid
pointer to the cgroup storage; no NULL-check is required.

This patch introduces BPF_PTR_TO_MAP_VALUE return type,
which is required to cause the verifier accept programs,
which are not checking the map value pointer for being NULL.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf.h   | 1 +
 kernel/bpf/verifier.c | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 31a94a842449..577fc6cd830e 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -154,6 +154,7 @@ enum bpf_arg_type {
 enum bpf_return_type {
 	RET_INTEGER,			/* function returns integer */
 	RET_VOID,			/* function doesn't return anything */
+	RET_PTR_TO_MAP_VALUE,		/* returns a pointer to map elem value */
 	RET_PTR_TO_MAP_VALUE_OR_NULL,	/* returns a pointer to map elem value or NULL */
 };
 
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index de097a642c3f..cc0c7990f849 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2545,8 +2545,12 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn
 		mark_reg_unknown(env, regs, BPF_REG_0);
 	} else if (fn->ret_type == RET_VOID) {
 		regs[BPF_REG_0].type = NOT_INIT;
-	} else if (fn->ret_type == RET_PTR_TO_MAP_VALUE_OR_NULL) {
-		regs[BPF_REG_0].type = PTR_TO_MAP_VALUE_OR_NULL;
+	} else if (fn->ret_type == RET_PTR_TO_MAP_VALUE_OR_NULL ||
+		   fn->ret_type == RET_PTR_TO_MAP_VALUE) {
+		if (fn->ret_type == RET_PTR_TO_MAP_VALUE)
+			regs[BPF_REG_0].type = PTR_TO_MAP_VALUE;
+		else
+			regs[BPF_REG_0].type = PTR_TO_MAP_VALUE_OR_NULL;
 		/* There is no offset yet applied, variable or fixed */
 		mark_reg_known_zero(env, regs, BPF_REG_0);
 		regs[BPF_REG_0].off = 0;
-- 
2.14.4

[PATCH v2 bpf-next 07/14] bpf: don't allow create maps of cgroup local storages

[Roman Gushchin]

As there is one-to-one relation between a bpf program
and cgroup local storage map, there is no sense in
creating a map of cgroup local storage maps.

Forbid it explicitly to avoid possible side effects.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 kernel/bpf/map_in_map.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/map_in_map.c b/kernel/bpf/map_in_map.c
index 1da574612bea..3bfbf4464416 100644
--- a/kernel/bpf/map_in_map.c
+++ b/kernel/bpf/map_in_map.c
@@ -23,7 +23,8 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
 	 * is a runtime binding.  Doing static check alone
 	 * in the verifier is not enough.
 	 */
-	if (inner_map->map_type == BPF_MAP_TYPE_PROG_ARRAY) {
+	if (inner_map->map_type == BPF_MAP_TYPE_PROG_ARRAY ||
+	    inner_map->map_type == BPF_MAP_TYPE_CGROUP_STORAGE) {
 		fdput(f);
 		return ERR_PTR(-ENOTSUPP);
 	}
-- 
2.14.4

[PATCH v2 bpf-next 08/14] bpf: introduce the bpf_get_local_storage() helper function

[Roman Gushchin]

The bpf_get_local_storage() helper function is used
to get a pointer to the bpf local storage from a bpf program.

It takes a pointer to a storage map and flags as arguments.
Right now it accepts only cgroup storage maps, and flags
argument has to be 0. Further it can be extended to support
other types of local storage: e.g. thread local storage etc.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 include/linux/bpf.h      |  2 ++
 include/uapi/linux/bpf.h | 13 ++++++++++++-
 kernel/bpf/cgroup.c      |  2 ++
 kernel/bpf/core.c        |  1 +
 kernel/bpf/helpers.c     | 20 ++++++++++++++++++++
 kernel/bpf/verifier.c    | 18 ++++++++++++++++++
 net/core/filter.c        | 23 ++++++++++++++++++++++-
 7 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 577fc6cd830e..97a3832fb5ff 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -779,6 +779,8 @@ extern const struct bpf_func_proto bpf_sock_map_update_proto;
 extern const struct bpf_func_proto bpf_sock_hash_update_proto;
 extern const struct bpf_func_proto bpf_get_current_cgroup_id_proto;
 
+extern const struct bpf_func_proto bpf_get_local_storage_proto;
+
 /* Shared helpers among cBPF and eBPF. */
 void bpf_user_rnd_init_once(void);
 u64 bpf_user_rnd_u32(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 6f9b907a8a0e..ee8b57eba728 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -2083,6 +2083,16 @@ union bpf_attr {
  * 	Return
  * 		A 64-bit integer containing the current cgroup id based
  * 		on the cgroup within which the current task is running.
+ *
+ * void* get_local_storage(void *map, u64 flags)
+ *	Description
+ *		Get the pointer to the local storage area.
+ *		The type and the size of the local storage is defined
+ *		by the *map* argument.
+ *		The *flags* meaning is specific for each map type,
+ *		and has to be 0 for cgroup local storage.
+ *	Return
+ *		Pointer to the local storage area.
  */
 #define __BPF_FUNC_MAPPER(FN)		\
 	FN(unspec),			\
@@ -2165,7 +2175,8 @@ union bpf_attr {
 	FN(rc_repeat),			\
 	FN(rc_keydown),			\
 	FN(skb_cgroup_id),		\
-	FN(get_current_cgroup_id),
+	FN(get_current_cgroup_id),	\
+	FN(get_local_storage),
 
 /* integer value in 'imm' field of BPF_CALL instruction selects which helper
  * function eBPF program intends to call
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index dd232a276668..4f7b5bb7d97e 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -683,6 +683,8 @@ cgroup_dev_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_map_delete_elem_proto;
 	case BPF_FUNC_get_current_uid_gid:
 		return &bpf_get_current_uid_gid_proto;
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	case BPF_FUNC_trace_printk:
 		if (capable(CAP_SYS_ADMIN))
 			return bpf_get_trace_printk_proto();
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 5903be928722..efbcb4e58a7c 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1795,6 +1795,7 @@ const struct bpf_func_proto bpf_get_current_comm_proto __weak;
 const struct bpf_func_proto bpf_sock_map_update_proto __weak;
 const struct bpf_func_proto bpf_sock_hash_update_proto __weak;
 const struct bpf_func_proto bpf_get_current_cgroup_id_proto __weak;
+const struct bpf_func_proto bpf_get_local_storage_proto __weak;
 
 const struct bpf_func_proto * __weak bpf_get_trace_printk_proto(void)
 {
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 73065e2d23c2..1991466b8327 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -193,4 +193,24 @@ const struct bpf_func_proto bpf_get_current_cgroup_id_proto = {
 	.gpl_only	= false,
 	.ret_type	= RET_INTEGER,
 };
+
+DECLARE_PER_CPU(void*, bpf_cgroup_storage);
+
+BPF_CALL_2(bpf_get_local_storage, struct bpf_map *, map, u64, flags)
+{
+	/* map and flags arguments are not used now,
+	 * but provide an ability to extend the API
+	 * for other types of local storages.
+	 * verifier checks that their values are correct.
+	 */
+	return (unsigned long) this_cpu_read(bpf_cgroup_storage);
+}
+
+const struct bpf_func_proto bpf_get_local_storage_proto = {
+	.func		= bpf_get_local_storage,
+	.gpl_only	= false,
+	.ret_type	= RET_PTR_TO_MAP_VALUE,
+	.arg1_type	= ARG_CONST_MAP_PTR,
+	.arg2_type	= ARG_ANYTHING,
+};
 #endif
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index cc0c7990f849..a0f5c26fffc1 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2127,6 +2127,10 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
 		    func_id != BPF_FUNC_current_task_under_cgroup)
 			goto error;
 		break;
+	case BPF_MAP_TYPE_CGROUP_STORAGE:
+		if (func_id != BPF_FUNC_get_local_storage)
+			goto error;
+		break;
 	/* devmap returns a pointer to a live net_device ifindex that we cannot
 	 * allow to be modified from bpf side. So do not allow lookup elements
 	 * for now.
@@ -2209,6 +2213,10 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
 		if (map->map_type != BPF_MAP_TYPE_SOCKHASH)
 			goto error;
 		break;
+	case BPF_FUNC_get_local_storage:
+		if (map->map_type != BPF_MAP_TYPE_CGROUP_STORAGE)
+			goto error;
+		break;
 	default:
 		break;
 	}
@@ -2533,6 +2541,16 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn
 	}
 
 	regs = cur_regs(env);
+
+	/* check that flags argument in get_local_storage(map, flags) is 0,
+	 * this is required because get_local_storage() can't return an error.
+	 */
+	if (func_id == BPF_FUNC_get_local_storage &&
+	    !tnum_equals_const(regs[BPF_REG_2].var_off, 0)) {
+		verbose(env, "get_local_storage() doesn't support non-zero flags\n");
+		return -EINVAL;
+	}
+
 	/* reset caller saved regs */
 	for (i = 0; i < CALLER_SAVED_REGS; i++) {
 		mark_reg_not_init(env, regs, caller_saved[i]);
diff --git a/net/core/filter.c b/net/core/filter.c
index 547fd34589be..753b99238516 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -4674,6 +4674,8 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	 */
 	case BPF_FUNC_get_current_uid_gid:
 		return &bpf_get_current_uid_gid_proto;
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	default:
 		return bpf_base_func_proto(func_id);
 	}
@@ -4696,6 +4698,8 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		default:
 			return NULL;
 		}
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	default:
 		return bpf_base_func_proto(func_id);
 	}
@@ -4718,6 +4722,17 @@ sk_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	}
 }
 
+static const struct bpf_func_proto *
+cg_skb_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
+{
+	switch (func_id) {
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
+	default:
+		return sk_filter_func_proto(func_id, prog);
+	}
+}
+
 static const struct bpf_func_proto *
 tc_cls_act_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 {
@@ -4838,6 +4853,8 @@ sock_ops_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_sock_map_update_proto;
 	case BPF_FUNC_sock_hash_update:
 		return &bpf_sock_hash_update_proto;
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	default:
 		return bpf_base_func_proto(func_id);
 	}
@@ -4857,6 +4874,8 @@ sk_msg_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_msg_cork_bytes_proto;
 	case BPF_FUNC_msg_pull_data:
 		return &bpf_msg_pull_data_proto;
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	default:
 		return bpf_base_func_proto(func_id);
 	}
@@ -4884,6 +4903,8 @@ sk_skb_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_sk_redirect_map_proto;
 	case BPF_FUNC_sk_redirect_hash:
 		return &bpf_sk_redirect_hash_proto;
+	case BPF_FUNC_get_local_storage:
+		return &bpf_get_local_storage_proto;
 	default:
 		return bpf_base_func_proto(func_id);
 	}
@@ -6686,7 +6707,7 @@ const struct bpf_prog_ops xdp_prog_ops = {
 };
 
 const struct bpf_verifier_ops cg_skb_verifier_ops = {
-	.get_func_proto		= sk_filter_func_proto,
+	.get_func_proto		= cg_skb_func_proto,
 	.is_valid_access	= sk_filter_is_valid_access,
 	.convert_ctx_access	= bpf_convert_ctx_access,
 };
-- 
2.14.4

[PATCH v2 bpf-next 09/14] bpf: sync bpf.h to tools/

[Roman Gushchin]

Sync cgroup storage related changes:
1) new BPF_MAP_TYPE_CGROUP_STORAGE map type
2) struct bpf_cgroup_sotrage_key definition
3) get_local_storage() helper

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 tools/include/uapi/linux/bpf.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 59b19b6a40d7..84480330c607 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -75,6 +75,11 @@ struct bpf_lpm_trie_key {
 	__u8	data[0];	/* Arbitrary size */
 };
 
+struct bpf_cgroup_storage_key {
+	__u64	cgroup_inode_id;	/* cgroup inode id */
+	__u32	attach_type;		/* program attach type */
+};
+
 /* BPF syscall commands, see bpf(2) man-page for details. */
 enum bpf_cmd {
 	BPF_MAP_CREATE,
@@ -120,6 +125,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_CPUMAP,
 	BPF_MAP_TYPE_XSKMAP,
 	BPF_MAP_TYPE_SOCKHASH,
+	BPF_MAP_TYPE_CGROUP_STORAGE,
 };
 
 enum bpf_prog_type {
@@ -2157,7 +2163,8 @@ union bpf_attr {
 	FN(rc_repeat),			\
 	FN(rc_keydown),			\
 	FN(skb_cgroup_id),		\
-	FN(get_current_cgroup_id),
+	FN(get_current_cgroup_id),	\
+	FN(get_local_storage),
 
 /* integer value in 'imm' field of BPF_CALL instruction selects which helper
  * function eBPF program intends to call
-- 
2.14.4

[PATCH v2 bpf-next 10/14] bpftool: add support for CGROUP_STORAGE maps

[Roman Gushchin]

Add BPF_MAP_TYPE_CGROUP_STORAGE maps to the list
of maps types which bpftool recognizes.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 tools/bpf/bpftool/map.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 5989e1575ae4..14bef5855d8c 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -65,6 +65,7 @@ static const char * const map_type_name[] = {
 	[BPF_MAP_TYPE_SOCKMAP]		= "sockmap",
 	[BPF_MAP_TYPE_CPUMAP]		= "cpumap",
 	[BPF_MAP_TYPE_SOCKHASH]		= "sockhash",
+	[BPF_MAP_TYPE_CGROUP_STORAGE]	= "cgroup_storage",
 };
 
 static bool map_is_per_cpu(__u32 type)
-- 
2.14.4

[PATCH v2 bpf-next 11/14] bpf/test_run: support cgroup local storage

[Roman Gushchin]

Allocate a temporary cgroup storage to use for bpf program test runs.

Because the test program is not actually attached to a cgroup,
the storage is allocated manually just for the execution
of the bpf program.

If the program is executed multiple times, the storage is not zeroed
on each run, emulating multiple runs of the program, attached to
a real cgroup.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 net/bpf/test_run.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index 68c3578343b4..74971a9b7cfb 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -11,12 +11,14 @@
 #include <linux/filter.h>
 #include <linux/sched/signal.h>
 
-static __always_inline u32 bpf_test_run_one(struct bpf_prog *prog, void *ctx)
+static __always_inline u32 bpf_test_run_one(struct bpf_prog *prog, void *ctx,
+					    struct bpf_cgroup_storage *storage)
 {
 	u32 ret;
 
 	preempt_disable();
 	rcu_read_lock();
+	bpf_cgroup_storage_set(storage);
 	ret = BPF_PROG_RUN(prog, ctx);
 	rcu_read_unlock();
 	preempt_enable();
@@ -26,14 +28,19 @@ static __always_inline u32 bpf_test_run_one(struct bpf_prog *prog, void *ctx)
 
 static u32 bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *time)
 {
+	struct bpf_cgroup_storage *storage = NULL;
 	u64 time_start, time_spent = 0;
 	u32 ret = 0, i;
 
+	storage = bpf_cgroup_storage_alloc(prog);
+	if (IS_ERR(storage))
+		return PTR_ERR(storage);
+
 	if (!repeat)
 		repeat = 1;
 	time_start = ktime_get_ns();
 	for (i = 0; i < repeat; i++) {
-		ret = bpf_test_run_one(prog, ctx);
+		ret = bpf_test_run_one(prog, ctx, storage);
 		if (need_resched()) {
 			if (signal_pending(current))
 				break;
@@ -46,6 +53,8 @@ static u32 bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *time)
 	do_div(time_spent, repeat);
 	*time = time_spent > U32_MAX ? U32_MAX : (u32)time_spent;
 
+	bpf_cgroup_storage_free(storage);
+
 	return ret;
 }
 
-- 
2.14.4

[PATCH v2 bpf-next 12/14] selftests/bpf: add verifier cgroup storage tests

[Roman Gushchin]

Add the following verifier tests to cover the cgroup storage
functionality:
1) valid access to the cgroup storage
2) invalid access: use regular hashmap instead of cgroup storage map
3) invalid access: use invalid map fd
4) invalid access: try access memory after the cgroup storage
5) invalid access: try access memory before the cgroup storage
6) invalid access: call get_local_storage() with non-zero flags

For tests 2)-6) check returned error strings.

Expected output:
  $ ./test_verifier
  #0/u add+sub+mul OK
  #0/p add+sub+mul OK
  #1/u DIV32 by 0, zero check 1 OK
  ...
  #280/p valid cgroup storage access OK
  #281/p invalid cgroup storage access 1 OK
  #282/p invalid cgroup storage access 2 OK
  #283/p invalid per-cgroup storage access 3 OK
  #284/p invalid cgroup storage access 4 OK
  #285/p invalid cgroup storage access 5 OK
  ...
  #649/p pass modified ctx pointer to helper, 2 OK
  #650/p pass modified ctx pointer to helper, 3 OK
  Summary: 901 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 tools/testing/selftests/bpf/bpf_helpers.h   |   2 +
 tools/testing/selftests/bpf/test_verifier.c | 123 +++++++++++++++++++++++++++-
 2 files changed, 124 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/bpf_helpers.h b/tools/testing/selftests/bpf/bpf_helpers.h
index f2f28b6c8915..ccd959fd940e 100644
--- a/tools/testing/selftests/bpf/bpf_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_helpers.h
@@ -133,6 +133,8 @@ static int (*bpf_rc_keydown)(void *ctx, unsigned int protocol,
 	(void *) BPF_FUNC_rc_keydown;
 static unsigned long long (*bpf_get_current_cgroup_id)(void) =
 	(void *) BPF_FUNC_get_current_cgroup_id;
+static void *(*bpf_get_local_storage)(void *map, unsigned long long flags) =
+	(void *) BPF_FUNC_get_local_storage;
 
 /* llvm builtin functions that eBPF C program may use to
  * emit BPF_LD_ABS and BPF_LD_IND instructions
diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index 2ecd27b670d7..7016fb2964a1 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -50,7 +50,7 @@
 
 #define MAX_INSNS	BPF_MAXINSNS
 #define MAX_FIXUPS	8
-#define MAX_NR_MAPS	7
+#define MAX_NR_MAPS	8
 #define POINTER_VALUE	0xcafe4all
 #define TEST_DATA_LEN	64
 
@@ -70,6 +70,7 @@ struct bpf_test {
 	int fixup_prog1[MAX_FIXUPS];
 	int fixup_prog2[MAX_FIXUPS];
 	int fixup_map_in_map[MAX_FIXUPS];
+	int fixup_cgroup_storage[MAX_FIXUPS];
 	const char *errstr;
 	const char *errstr_unpriv;
 	uint32_t retval;
@@ -4630,6 +4631,104 @@ static struct bpf_test tests[] = {
 		.result = REJECT,
 		.flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS,
 	},
+	{
+		"valid cgroup storage access",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 0),
+			BPF_LD_MAP_FD(BPF_REG_1, 0),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+			BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+			BPF_ALU64_IMM(BPF_AND, BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.fixup_cgroup_storage = { 1 },
+		.result = ACCEPT,
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
+	{
+		"invalid cgroup storage access 1",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 0),
+			BPF_LD_MAP_FD(BPF_REG_1, 0),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+			BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+			BPF_ALU64_IMM(BPF_AND, BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.fixup_map1 = { 1 },
+		.result = REJECT,
+		.errstr = "cannot pass map_type 1 into func bpf_get_local_storage",
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
+	{
+		"invalid cgroup storage access 2",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 0),
+			BPF_LD_MAP_FD(BPF_REG_1, 1),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_ALU64_IMM(BPF_AND, BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.result = REJECT,
+		.errstr = "fd 1 is not pointing to valid bpf_map",
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
+	{
+		"invalid per-cgroup storage access 3",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 0),
+			BPF_LD_MAP_FD(BPF_REG_1, 0),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 256),
+			BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 1),
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			BPF_EXIT_INSN(),
+		},
+		.fixup_cgroup_storage = { 1 },
+		.result = REJECT,
+		.errstr = "invalid access to map value, value_size=64 off=256 size=4",
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
+	{
+		"invalid cgroup storage access 4",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 0),
+			BPF_LD_MAP_FD(BPF_REG_1, 0),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, -2),
+			BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+			BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 1),
+			BPF_EXIT_INSN(),
+		},
+		.fixup_cgroup_storage = { 1 },
+		.result = REJECT,
+		.errstr = "invalid access to map value, value_size=64 off=-2 size=4",
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
+	{
+		"invalid cgroup storage access 5",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_2, 7),
+			BPF_LD_MAP_FD(BPF_REG_1, 0),
+			BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+				     BPF_FUNC_get_local_storage),
+			BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+			BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+			BPF_ALU64_IMM(BPF_AND, BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.fixup_cgroup_storage = { 1 },
+		.result = REJECT,
+		.errstr = "get_local_storage() doesn't support non-zero flags",
+		.prog_type = BPF_PROG_TYPE_CGROUP_SKB,
+	},
 	{
 		"multiple registers share map_lookup_elem result",
 		.insns = {
@@ -12418,6 +12517,19 @@ static int create_map_in_map(void)
 	return outer_map_fd;
 }
 
+static int create_cgroup_storage(void)
+{
+	int fd;
+
+	fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_STORAGE,
+			    sizeof(struct bpf_cgroup_storage_key),
+			    TEST_DATA_LEN, 0, 0);
+	if (fd < 0)
+		printf("Failed to create array '%s'!\n", strerror(errno));
+
+	return fd;
+}
+
 static char bpf_vlog[UINT_MAX >> 8];
 
 static void do_test_fixup(struct bpf_test *test, struct bpf_insn *prog,
@@ -12430,6 +12542,7 @@ static void do_test_fixup(struct bpf_test *test, struct bpf_insn *prog,
 	int *fixup_prog1 = test->fixup_prog1;
 	int *fixup_prog2 = test->fixup_prog2;
 	int *fixup_map_in_map = test->fixup_map_in_map;
+	int *fixup_cgroup_storage = test->fixup_cgroup_storage;
 
 	if (test->fill_helper)
 		test->fill_helper(test);
@@ -12497,6 +12610,14 @@ static void do_test_fixup(struct bpf_test *test, struct bpf_insn *prog,
 			fixup_map_in_map++;
 		} while (*fixup_map_in_map);
 	}
+
+	if (*fixup_cgroup_storage) {
+		map_fds[7] = create_cgroup_storage();
+		do {
+			prog[*fixup_cgroup_storage].imm = map_fds[7];
+			fixup_cgroup_storage++;
+		} while (*fixup_cgroup_storage);
+	}
 }
 
 static void do_test_single(struct bpf_test *test, bool unpriv,
-- 
2.14.4

[PATCH v2 bpf-next 13/14] selftests/bpf: add a cgroup storage test

[Roman Gushchin]

Implement a test to cover the cgroup storage functionality.
The test implements a bpf program which drops every second packet
by using the cgroup storage as a persistent storage.

The test also use the userspace API to check the data
in the cgroup storage, alter it, and check that the loaded
and attached bpf program sees the update.

Expected output:
  $ ./test_cgroup_storage
  test_cgroup_storage:PASS

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 tools/testing/selftests/bpf/Makefile              |   4 +-
 tools/testing/selftests/bpf/test_cgroup_storage.c | 130 ++++++++++++++++++++++
 2 files changed, 133 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/test_cgroup_storage.c

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 7a6214e9ae58..81f38623fc9f 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -22,7 +22,8 @@ $(TEST_CUSTOM_PROGS): $(OUTPUT)/%: %.c
 # Order correspond to 'make run_tests' order
 TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \
 	test_align test_verifier_log test_dev_cgroup test_tcpbpf_user \
-	test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user
+	test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \
+	test_cgroup_storage
 
 TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test_obj_id.o \
 	test_pkt_md_access.o test_xdp_redirect.o test_xdp_meta.o sockmap_parse_prog.o     \
@@ -63,6 +64,7 @@ $(OUTPUT)/test_sock_addr: cgroup_helpers.c
 $(OUTPUT)/test_sockmap: cgroup_helpers.c
 $(OUTPUT)/test_progs: trace_helpers.c
 $(OUTPUT)/get_cgroup_id_user: cgroup_helpers.c
+$(OUTPUT)/test_cgroup_storage: cgroup_helpers.c
 
 .PHONY: force
 
diff --git a/tools/testing/selftests/bpf/test_cgroup_storage.c b/tools/testing/selftests/bpf/test_cgroup_storage.c
new file mode 100644
index 000000000000..0597943ce34b
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_cgroup_storage.c
@@ -0,0 +1,130 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <assert.h>
+#include <bpf/bpf.h>
+#include <linux/filter.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "cgroup_helpers.h"
+
+char bpf_log_buf[BPF_LOG_BUF_SIZE];
+
+#define TEST_CGROUP "/test-bpf-cgroup-storage-buf/"
+
+int main(int argc, char **argv)
+{
+	struct bpf_insn prog[] = {
+		BPF_LD_MAP_FD(BPF_REG_1, 0), /* map fd */
+		BPF_MOV64_IMM(BPF_REG_2, 0), /* flags, not used */
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
+			     BPF_FUNC_get_local_storage),
+		BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+		BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 1),
+		BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0),
+		BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 0x1),
+		BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+		BPF_EXIT_INSN(),
+	};
+	size_t insns_cnt = sizeof(prog) / sizeof(struct bpf_insn);
+	int error = EXIT_FAILURE;
+	int map_fd, prog_fd, cgroup_fd;
+	struct bpf_cgroup_storage_key key;
+	unsigned long long value;
+
+	map_fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_STORAGE, sizeof(key),
+				sizeof(value), 0, 0);
+	if (map_fd < 0) {
+		printf("Failed to create map: %s\n", strerror(errno));
+		goto out;
+	}
+
+	prog[0].imm = map_fd;
+	prog_fd = bpf_load_program(BPF_PROG_TYPE_CGROUP_SKB,
+				   prog, insns_cnt, "GPL", 0,
+				   bpf_log_buf, BPF_LOG_BUF_SIZE);
+	if (prog_fd < 0) {
+		printf("Failed to load bpf program: %s\n", bpf_log_buf);
+		goto out;
+	}
+
+	if (setup_cgroup_environment()) {
+		printf("Failed to setup cgroup environment\n");
+		goto err;
+	}
+
+	/* Create a cgroup, get fd, and join it */
+	cgroup_fd = create_and_get_cgroup(TEST_CGROUP);
+	if (!cgroup_fd) {
+		printf("Failed to create test cgroup\n");
+		goto err;
+	}
+
+	if (join_cgroup(TEST_CGROUP)) {
+		printf("Failed to join cgroup\n");
+		goto err;
+	}
+
+	/* Attach the bpf program */
+	if (bpf_prog_attach(prog_fd, cgroup_fd, BPF_CGROUP_INET_EGRESS, 0)) {
+		printf("Failed to attach bpf program\n");
+		goto err;
+	}
+
+	if (bpf_map_get_next_key(map_fd, NULL, &key)) {
+		printf("Failed to get the first key in cgroup storage\n");
+		goto err;
+	}
+
+	if (bpf_map_lookup_elem(map_fd, &key, &value)) {
+		printf("Failed to lookup cgroup storage\n");
+		goto err;
+	}
+
+	/* Every second packet should be dropped */
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null") == 0);
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null"));
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null") == 0);
+
+	/* Check the counter in the cgroup local storage */
+	if (bpf_map_lookup_elem(map_fd, &key, &value)) {
+		printf("Failed to lookup cgroup storage\n");
+		goto err;
+	}
+
+	if (value != 3) {
+		printf("Unexpected data in the cgroup storage: %llu\n", value);
+		goto err;
+	}
+
+	/* Bump the counter in the cgroup local storage */
+	value++;
+	if (bpf_map_update_elem(map_fd, &key, &value, 0)) {
+		printf("Failed to update the data in the cgroup storage\n");
+		goto err;
+	}
+
+	/* Every second packet should be dropped */
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null") == 0);
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null"));
+	assert(system("ping localhost -c 1 -W 1 -q > /dev/null") == 0);
+
+	/* Check the final value of the counter in the cgroup local storage */
+	if (bpf_map_lookup_elem(map_fd, &key, &value)) {
+		printf("Failed to lookup the cgroup storage\n");
+		goto err;
+	}
+
+	if (value != 7) {
+		printf("Unexpected data in the cgroup storage: %llu\n", value);
+		goto err;
+	}
+
+	error = 0;
+	printf("test_cgroup_storage:PASS\n");
+
+err:
+	cleanup_cgroup_environment();
+
+out:
+	return error;
+}
-- 
2.14.4

[PATCH v2 bpf-next 14/14] samples/bpf: extend test_cgrp2_attach2 test to use cgroup storage

[Roman Gushchin]

The test_cgrp2_attach test covers bpf cgroup attachment code well,
so let's re-use it for testing allocation/releasing of cgroup storage.

The extension is pretty straightforward: the bpf program will use
the cgroup storage to save the number of transmitted bytes.

Expected output:
  $ ./test_cgrp2_attach2
  Attached DROP prog. This ping in cgroup /foo should fail...
  ping: sendmsg: Operation not permitted
  Attached DROP prog. This ping in cgroup /foo/bar should fail...
  ping: sendmsg: Operation not permitted
  Attached PASS prog. This ping in cgroup /foo/bar should pass...
  Detached PASS from /foo/bar while DROP is attached to /foo.
  This ping in cgroup /foo/bar should fail...
  ping: sendmsg: Operation not permitted
  Attached PASS from /foo/bar and detached DROP from /foo.
  This ping in cgroup /foo/bar should pass...
  ### override:PASS
  ### multi:PASS

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
 samples/bpf/test_cgrp2_attach2.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/samples/bpf/test_cgrp2_attach2.c b/samples/bpf/test_cgrp2_attach2.c
index b453e6a161be..180f9d813bca 100644
--- a/samples/bpf/test_cgrp2_attach2.c
+++ b/samples/bpf/test_cgrp2_attach2.c
@@ -8,7 +8,8 @@
  *   information. The number of invocations of the program, which maps
  *   to the number of packets received, is stored to key 0. Key 1 is
  *   incremented on each iteration by the number of bytes stored in
- *   the skb.
+ *   the skb. The program also stores the number of received bytes
+ *   in the cgroup storage.
  *
  * - Attaches the new program to a cgroup using BPF_PROG_ATTACH
  *
@@ -21,12 +22,15 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <assert.h>
+#include <sys/resource.h>
+#include <sys/time.h>
 #include <unistd.h>
 
 #include <linux/bpf.h>
 #include <bpf/bpf.h>
 
 #include "bpf_insn.h"
+#include "bpf_rlimit.h"
 #include "cgroup_helpers.h"
 
 #define FOO		"/foo"
@@ -205,6 +209,8 @@ static int map_fd = -1;
 
 static int prog_load_cnt(int verdict, int val)
 {
+	int cgroup_storage_fd;
+
 	if (map_fd < 0)
 		map_fd = bpf_create_map(BPF_MAP_TYPE_ARRAY, 4, 8, 1, 0);
 	if (map_fd < 0) {
@@ -212,6 +218,13 @@ static int prog_load_cnt(int verdict, int val)
 		return -1;
 	}
 
+	cgroup_storage_fd = bpf_create_map(BPF_MAP_TYPE_CGROUP_STORAGE,
+				sizeof(struct bpf_cgroup_storage_key), 8, 0, 0);
+	if (cgroup_storage_fd < 0) {
+		printf("failed to create map '%s'\n", strerror(errno));
+		return -1;
+	}
+
 	struct bpf_insn prog[] = {
 		BPF_MOV32_IMM(BPF_REG_0, 0),
 		BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_0, -4), /* *(u32 *)(fp - 4) = r0 */
@@ -222,6 +235,11 @@ static int prog_load_cnt(int verdict, int val)
 		BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
 		BPF_MOV64_IMM(BPF_REG_1, val), /* r1 = 1 */
 		BPF_RAW_INSN(BPF_STX | BPF_XADD | BPF_DW, BPF_REG_0, BPF_REG_1, 0, 0), /* xadd r0 += r1 */
+		BPF_LD_MAP_FD(BPF_REG_1, cgroup_storage_fd),
+		BPF_MOV64_IMM(BPF_REG_2, 0),
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_get_local_storage),
+		BPF_MOV64_IMM(BPF_REG_1, val),
+		BPF_RAW_INSN(BPF_STX | BPF_XADD | BPF_W, BPF_REG_0, BPF_REG_1, 0, 0),
 		BPF_MOV64_IMM(BPF_REG_0, verdict), /* r0 = verdict */
 		BPF_EXIT_INSN(),
 	};
@@ -237,6 +255,7 @@ static int prog_load_cnt(int verdict, int val)
 		printf("Output from verifier:\n%s\n-------\n", bpf_log_buf);
 		return 0;
 	}
+	close(cgroup_storage_fd);
 	return ret;
 }
 
-- 
2.14.4

Re: [PATCH v2 bpf-next 05/14] bpf: extend bpf_prog_array to store pointers to the cgroup storage

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2304 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: x86_64-randconfig-u0-07061335 (attached as .config)
compiler: gcc-5 (Debian 5.5.0-3) 5.4.1 20171010
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   drivers/media//rc/bpf-lirc.c: In function 'lirc_bpf_free':
>> drivers/media//rc/bpf-lirc.c:203:44: error: 'struct bpf_prog_array' has no member named 'progs'
     progs = rcu_dereference(rcdev->raw->progs)->progs;
                                               ^

vim +203 drivers/media//rc/bpf-lirc.c

f4364dcf Sean Young 2018-05-27  191  
f4364dcf Sean Young 2018-05-27  192  /*
f4364dcf Sean Young 2018-05-27  193   * This should be called once the rc thread has been stopped, so there can be
f4364dcf Sean Young 2018-05-27  194   * no concurrent bpf execution.
f4364dcf Sean Young 2018-05-27  195   */
f4364dcf Sean Young 2018-05-27  196  void lirc_bpf_free(struct rc_dev *rcdev)
f4364dcf Sean Young 2018-05-27  197  {
f4364dcf Sean Young 2018-05-27  198  	struct bpf_prog **progs;
f4364dcf Sean Young 2018-05-27  199  
f4364dcf Sean Young 2018-05-27  200  	if (!rcdev->raw->progs)
f4364dcf Sean Young 2018-05-27  201  		return;
f4364dcf Sean Young 2018-05-27  202  
f4364dcf Sean Young 2018-05-27 @203  	progs = rcu_dereference(rcdev->raw->progs)->progs;
f4364dcf Sean Young 2018-05-27  204  	while (*progs)
f4364dcf Sean Young 2018-05-27  205  		bpf_prog_put(*progs++);
f4364dcf Sean Young 2018-05-27  206  
f4364dcf Sean Young 2018-05-27  207  	bpf_prog_array_free(rcdev->raw->progs);
f4364dcf Sean Young 2018-05-27  208  }
f4364dcf Sean Young 2018-05-27  209  

:::::: The code at line 203 was first introduced by commit
:::::: f4364dcfc86df7c1ca47b256eaf6b6d0cdd0d936 media: rc: introduce BPF_PROG_LIRC_MODE2

:::::: TO: Sean Young <sean@mess.org>
:::::: CC: Daniel Borkmann <daniel@iogearbox.net>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 35548 bytes --]

Re: [PATCH v2 bpf-next 03/14] bpf: pass a pointer to a cgroup storage using pcpu variable

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 10623 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: powerpc-defconfig (attached as .config)
compiler: powerpc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.2.0 make.cross ARCH=powerpc 

All errors (new ones prefixed by >>):

   In file included from include/linux/bpf-cgroup.h:6:0,
                    from kernel/bpf/local_storage.c:2:
   include/linux/percpu-defs.h:49:34: error: 'PER_CPU_BASE_SECTION' undeclared here (not in a function); did you mean 'PER_CPU_FIRST_SECTION'?
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                     ^
   include/linux/percpu-defs.h:101:9: note: in expansion of macro '__PCPU_ATTRS'
     extern __PCPU_ATTRS(sec) __typeof__(type) name
            ^~~~~~~~~~~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
   include/linux/percpu-defs.h:113:38: error: expected ')' before string constant
     DECLARE_PER_CPU_SECTION(type, name, "")
                                         ^
   include/linux/percpu-defs.h:49:55: note: in definition of macro '__PCPU_ATTRS'
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                                          ^~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
   include/linux/percpu-defs.h:49:59: error: expected identifier or '(' before ')' token
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                                              ^
   include/linux/percpu-defs.h:101:9: note: in expansion of macro '__PCPU_ATTRS'
     extern __PCPU_ATTRS(sec) __typeof__(type) name
            ^~~~~~~~~~~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h: In function 'bpf_cgroup_storage_set':
>> include/linux/bpf-cgroup.h:110:17: error: 'bpf_cgroup_storage' undeclared (first use in this function)
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
                    ^
   include/linux/percpu-defs.h:221:47: note: in definition of macro '__verify_pcpu_ptr'
     const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
                                                  ^~~
   include/linux/percpu-defs.h:510:34: note: in expansion of macro '__pcpu_size_call'
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                     ^~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:17: note: each undeclared identifier is reported only once for each function it appears in
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
                    ^
   include/linux/percpu-defs.h:221:47: note: in definition of macro '__verify_pcpu_ptr'
     const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
                                                  ^~~
   include/linux/percpu-defs.h:510:34: note: in expansion of macro '__pcpu_size_call'
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                     ^~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_1'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:379:11: note: in definition of macro '__pcpu_size_call'
      case 1: stem##1(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_2'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:380:11: note: in definition of macro '__pcpu_size_call'
      case 2: stem##2(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_4'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:381:11: note: in definition of macro '__pcpu_size_call'
      case 4: stem##4(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_8'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:382:11: note: in definition of macro '__pcpu_size_call'
      case 8: stem##8(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   cc1: some warnings being treated as errors

vim +/bpf_cgroup_storage +110 include/linux/bpf-cgroup.h

    23	
  > 24	DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    25	
    26	struct bpf_cgroup_storage_map;
    27	
    28	struct bpf_storage_buffer {
    29		struct rcu_head rcu;
    30		char data[0];
    31	};
    32	
    33	struct bpf_cgroup_storage {
    34		struct bpf_storage_buffer *buf;
    35		struct bpf_cgroup_storage_map *map;
    36		struct bpf_cgroup_storage_key key;
    37		struct list_head list;
    38		struct rb_node node;
    39		struct rcu_head rcu;
    40	};
    41	
    42	struct bpf_prog_list {
    43		struct list_head node;
    44		struct bpf_prog *prog;
    45	};
    46	
    47	struct bpf_prog_array;
    48	
    49	struct cgroup_bpf {
    50		/* array of effective progs in this cgroup */
    51		struct bpf_prog_array __rcu *effective[MAX_BPF_ATTACH_TYPE];
    52	
    53		/* attached progs to this cgroup and attach flags
    54		 * when flags == 0 or BPF_F_ALLOW_OVERRIDE the progs list will
    55		 * have either zero or one element
    56		 * when BPF_F_ALLOW_MULTI the list can have up to BPF_CGROUP_MAX_PROGS
    57		 */
    58		struct list_head progs[MAX_BPF_ATTACH_TYPE];
    59		u32 flags[MAX_BPF_ATTACH_TYPE];
    60	
    61		/* temp storage for effective prog array used by prog_attach/detach */
    62		struct bpf_prog_array __rcu *inactive;
    63	};
    64	
    65	void cgroup_bpf_put(struct cgroup *cgrp);
    66	int cgroup_bpf_inherit(struct cgroup *cgrp);
    67	
    68	int __cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
    69				enum bpf_attach_type type, u32 flags);
    70	int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
    71				enum bpf_attach_type type, u32 flags);
    72	int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
    73			       union bpf_attr __user *uattr);
    74	
    75	/* Wrapper for __cgroup_bpf_*() protected by cgroup_mutex */
    76	int cgroup_bpf_attach(struct cgroup *cgrp, struct bpf_prog *prog,
    77			      enum bpf_attach_type type, u32 flags);
    78	int cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
    79			      enum bpf_attach_type type, u32 flags);
    80	int cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
    81			     union bpf_attr __user *uattr);
    82	
    83	int __cgroup_bpf_run_filter_skb(struct sock *sk,
    84					struct sk_buff *skb,
    85					enum bpf_attach_type type);
    86	
    87	int __cgroup_bpf_run_filter_sk(struct sock *sk,
    88				       enum bpf_attach_type type);
    89	
    90	int __cgroup_bpf_run_filter_sock_addr(struct sock *sk,
    91					      struct sockaddr *uaddr,
    92					      enum bpf_attach_type type,
    93					      void *t_ctx);
    94	
    95	int __cgroup_bpf_run_filter_sock_ops(struct sock *sk,
    96					     struct bpf_sock_ops_kern *sock_ops,
    97					     enum bpf_attach_type type);
    98	
    99	int __cgroup_bpf_check_dev_permission(short dev_type, u32 major, u32 minor,
   100					      short access, enum bpf_attach_type type);
   101	
   102	static inline void bpf_cgroup_storage_set(struct bpf_cgroup_storage *storage)
   103	{
   104		struct bpf_storage_buffer *buf;
   105	
   106		if (!storage)
   107			return;
   108	
   109		buf = rcu_dereference(storage->buf);
 > 110		this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
   111	}
   112	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 23809 bytes --]

Re: [PATCH v2 bpf-next 03/14] bpf: pass a pointer to a cgroup storage using pcpu variable

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 23155 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: um-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=um 

All error/warnings (new ones prefixed by >>):

   In file included from include/linux/bpf-cgroup.h:6:0,
                    from kernel/bpf/local_storage.c:2:
>> include/linux/percpu-defs.h:49:34: error: 'PER_CPU_BASE_SECTION' undeclared here (not in a function); did you mean 'PER_CPU_FIRST_SECTION'?
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                     ^
>> include/linux/percpu-defs.h:87:9: note: in expansion of macro '__PCPU_ATTRS'
     extern __PCPU_ATTRS(sec) __typeof__(type) name
            ^~~~~~~~~~~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
>> include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:113:38: error: expected ')' before string constant
     DECLARE_PER_CPU_SECTION(type, name, "")
                                         ^
   include/linux/percpu-defs.h:49:55: note: in definition of macro '__PCPU_ATTRS'
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                                          ^~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
>> include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:49:59: error: expected identifier or '(' before ')' token
     __percpu __attribute__((section(PER_CPU_BASE_SECTION sec))) \
                                                              ^
>> include/linux/percpu-defs.h:87:9: note: in expansion of macro '__PCPU_ATTRS'
     extern __PCPU_ATTRS(sec) __typeof__(type) name
            ^~~~~~~~~~~~
   include/linux/percpu-defs.h:113:2: note: in expansion of macro 'DECLARE_PER_CPU_SECTION'
     DECLARE_PER_CPU_SECTION(type, name, "")
     ^~~~~~~~~~~~~~~~~~~~~~~
>> include/linux/bpf-cgroup.h:24:1: note: in expansion of macro 'DECLARE_PER_CPU'
    DECLARE_PER_CPU(void*, bpf_cgroup_storage);
    ^~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h: In function 'bpf_cgroup_storage_set':
   include/linux/bpf-cgroup.h:110:17: error: 'bpf_cgroup_storage' undeclared (first use in this function)
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
                    ^
   include/linux/percpu-defs.h:221:47: note: in definition of macro '__verify_pcpu_ptr'
     const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
                                                  ^~~
   include/linux/percpu-defs.h:510:34: note: in expansion of macro '__pcpu_size_call'
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                     ^~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:17: note: each undeclared identifier is reported only once for each function it appears in
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
                    ^
   include/linux/percpu-defs.h:221:47: note: in definition of macro '__verify_pcpu_ptr'
     const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
                                                  ^~~
   include/linux/percpu-defs.h:510:34: note: in expansion of macro '__pcpu_size_call'
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                     ^~~~~~~~~~~~~~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_1'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:379:11: note: in definition of macro '__pcpu_size_call'
      case 1: stem##1(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_2'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:380:11: note: in definition of macro '__pcpu_size_call'
      case 2: stem##2(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_4'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:381:11: note: in definition of macro '__pcpu_size_call'
      case 4: stem##4(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
>> include/linux/percpu-defs.h:510:51: error: implicit declaration of function 'this_cpu_write_8'; did you mean 'this_cpu_write'? [-Werror=implicit-function-declaration]
    #define this_cpu_write(pcp, val) __pcpu_size_call(this_cpu_write_, pcp, val)
                                                      ^
   include/linux/percpu-defs.h:382:11: note: in definition of macro '__pcpu_size_call'
      case 8: stem##8(variable, __VA_ARGS__);break;  \
              ^~~~
   include/linux/bpf-cgroup.h:110:2: note: in expansion of macro 'this_cpu_write'
     this_cpu_write(bpf_cgroup_storage, &buf->data[0]);
     ^~~~~~~~~~~~~~
   cc1: some warnings being treated as errors

vim +49 include/linux/percpu-defs.h

62fde54123 Tejun Heo           2014-06-17   37  
5028eaa97d David Howells       2009-04-21   38  /*
5028eaa97d David Howells       2009-04-21   39   * Base implementations of per-CPU variable declarations and definitions, where
5028eaa97d David Howells       2009-04-21   40   * the section in which the variable is to be placed is provided by the
7c756e6e19 Tejun Heo           2009-06-24   41   * 'sec' argument.  This may be used to affect the parameters governing the
5028eaa97d David Howells       2009-04-21   42   * variable's storage.
5028eaa97d David Howells       2009-04-21   43   *
5028eaa97d David Howells       2009-04-21   44   * NOTE!  The sections for the DECLARE and for the DEFINE must match, lest
5028eaa97d David Howells       2009-04-21   45   * linkage errors occur due the compiler generating the wrong code to access
5028eaa97d David Howells       2009-04-21   46   * that section.
5028eaa97d David Howells       2009-04-21   47   */
7c756e6e19 Tejun Heo           2009-06-24   48  #define __PCPU_ATTRS(sec)						\
e0fdb0e050 Rusty Russell       2009-10-29  @49  	__percpu __attribute__((section(PER_CPU_BASE_SECTION sec)))	\
7c756e6e19 Tejun Heo           2009-06-24   50  	PER_CPU_ATTRIBUTES
7c756e6e19 Tejun Heo           2009-06-24   51  
7c756e6e19 Tejun Heo           2009-06-24   52  #define __PCPU_DUMMY_ATTRS						\
7c756e6e19 Tejun Heo           2009-06-24   53  	__attribute__((section(".discard"), unused))
7c756e6e19 Tejun Heo           2009-06-24   54  
7c756e6e19 Tejun Heo           2009-06-24   55  /*
7c756e6e19 Tejun Heo           2009-06-24   56   * s390 and alpha modules require percpu variables to be defined as
7c756e6e19 Tejun Heo           2009-06-24   57   * weak to force the compiler to generate GOT based external
7c756e6e19 Tejun Heo           2009-06-24   58   * references for them.  This is necessary because percpu sections
7c756e6e19 Tejun Heo           2009-06-24   59   * will be located outside of the usually addressable area.
7c756e6e19 Tejun Heo           2009-06-24   60   *
7c756e6e19 Tejun Heo           2009-06-24   61   * This definition puts the following two extra restrictions when
7c756e6e19 Tejun Heo           2009-06-24   62   * defining percpu variables.
7c756e6e19 Tejun Heo           2009-06-24   63   *
7c756e6e19 Tejun Heo           2009-06-24   64   * 1. The symbol must be globally unique, even the static ones.
7c756e6e19 Tejun Heo           2009-06-24   65   * 2. Static percpu variables cannot be defined inside a function.
7c756e6e19 Tejun Heo           2009-06-24   66   *
7c756e6e19 Tejun Heo           2009-06-24   67   * Archs which need weak percpu definitions should define
7c756e6e19 Tejun Heo           2009-06-24   68   * ARCH_NEEDS_WEAK_PER_CPU in asm/percpu.h when necessary.
7c756e6e19 Tejun Heo           2009-06-24   69   *
7c756e6e19 Tejun Heo           2009-06-24   70   * To ensure that the generic code observes the above two
7c756e6e19 Tejun Heo           2009-06-24   71   * restrictions, if CONFIG_DEBUG_FORCE_WEAK_PER_CPU is set weak
7c756e6e19 Tejun Heo           2009-06-24   72   * definition is used for all cases.
7c756e6e19 Tejun Heo           2009-06-24   73   */
7c756e6e19 Tejun Heo           2009-06-24   74  #if defined(ARCH_NEEDS_WEAK_PER_CPU) || defined(CONFIG_DEBUG_FORCE_WEAK_PER_CPU)
7c756e6e19 Tejun Heo           2009-06-24   75  /*
7c756e6e19 Tejun Heo           2009-06-24   76   * __pcpu_scope_* dummy variable is used to enforce scope.  It
7c756e6e19 Tejun Heo           2009-06-24   77   * receives the static modifier when it's used in front of
7c756e6e19 Tejun Heo           2009-06-24   78   * DEFINE_PER_CPU() and will trigger build failure if
7c756e6e19 Tejun Heo           2009-06-24   79   * DECLARE_PER_CPU() is used for the same variable.
7c756e6e19 Tejun Heo           2009-06-24   80   *
7c756e6e19 Tejun Heo           2009-06-24   81   * __pcpu_unique_* dummy variable is used to enforce symbol uniqueness
7c756e6e19 Tejun Heo           2009-06-24   82   * such that hidden weak symbol collision, which will cause unrelated
7c756e6e19 Tejun Heo           2009-06-24   83   * variables to share the same address, can be detected during build.
7c756e6e19 Tejun Heo           2009-06-24   84   */
7c756e6e19 Tejun Heo           2009-06-24   85  #define DECLARE_PER_CPU_SECTION(type, name, sec)			\
7c756e6e19 Tejun Heo           2009-06-24   86  	extern __PCPU_DUMMY_ATTRS char __pcpu_scope_##name;		\
dd17c8f729 Rusty Russell       2009-10-29  @87  	extern __PCPU_ATTRS(sec) __typeof__(type) name
7c756e6e19 Tejun Heo           2009-06-24   88  
7c756e6e19 Tejun Heo           2009-06-24   89  #define DEFINE_PER_CPU_SECTION(type, name, sec)				\
7c756e6e19 Tejun Heo           2009-06-24   90  	__PCPU_DUMMY_ATTRS char __pcpu_scope_##name;			\
0f5e4816db Tejun Heo           2009-10-29   91  	extern __PCPU_DUMMY_ATTRS char __pcpu_unique_##name;		\
7c756e6e19 Tejun Heo           2009-06-24   92  	__PCPU_DUMMY_ATTRS char __pcpu_unique_##name;			\
b1a0fbfdde Tejun Heo           2013-12-04   93  	extern __PCPU_ATTRS(sec) __typeof__(type) name;			\
c43768cbb7 Tejun Heo           2009-07-04   94  	__PCPU_ATTRS(sec) PER_CPU_DEF_ATTRIBUTES __weak			\
dd17c8f729 Rusty Russell       2009-10-29   95  	__typeof__(type) name
7c756e6e19 Tejun Heo           2009-06-24   96  #else
7c756e6e19 Tejun Heo           2009-06-24   97  /*
7c756e6e19 Tejun Heo           2009-06-24   98   * Normal declaration and definition macros.
7c756e6e19 Tejun Heo           2009-06-24   99   */
7c756e6e19 Tejun Heo           2009-06-24  100  #define DECLARE_PER_CPU_SECTION(type, name, sec)			\
dd17c8f729 Rusty Russell       2009-10-29  101  	extern __PCPU_ATTRS(sec) __typeof__(type) name
7c756e6e19 Tejun Heo           2009-06-24  102  
7c756e6e19 Tejun Heo           2009-06-24  103  #define DEFINE_PER_CPU_SECTION(type, name, sec)				\
c43768cbb7 Tejun Heo           2009-07-04  104  	__PCPU_ATTRS(sec) PER_CPU_DEF_ATTRIBUTES			\
dd17c8f729 Rusty Russell       2009-10-29  105  	__typeof__(type) name
7c756e6e19 Tejun Heo           2009-06-24  106  #endif
5028eaa97d David Howells       2009-04-21  107  
5028eaa97d David Howells       2009-04-21  108  /*
5028eaa97d David Howells       2009-04-21  109   * Variant on the per-CPU variable declaration/definition theme used for
5028eaa97d David Howells       2009-04-21  110   * ordinary per-CPU variables.
5028eaa97d David Howells       2009-04-21  111   */
5028eaa97d David Howells       2009-04-21  112  #define DECLARE_PER_CPU(type, name)					\
5028eaa97d David Howells       2009-04-21 @113  	DECLARE_PER_CPU_SECTION(type, name, "")
5028eaa97d David Howells       2009-04-21  114  
5028eaa97d David Howells       2009-04-21  115  #define DEFINE_PER_CPU(type, name)					\
5028eaa97d David Howells       2009-04-21  116  	DEFINE_PER_CPU_SECTION(type, name, "")
5028eaa97d David Howells       2009-04-21  117  
5028eaa97d David Howells       2009-04-21  118  /*
5028eaa97d David Howells       2009-04-21  119   * Declaration/definition used for per-CPU variables that must come first in
5028eaa97d David Howells       2009-04-21  120   * the set of variables.
5028eaa97d David Howells       2009-04-21  121   */
5028eaa97d David Howells       2009-04-21  122  #define DECLARE_PER_CPU_FIRST(type, name)				\
5028eaa97d David Howells       2009-04-21  123  	DECLARE_PER_CPU_SECTION(type, name, PER_CPU_FIRST_SECTION)
5028eaa97d David Howells       2009-04-21  124  
5028eaa97d David Howells       2009-04-21  125  #define DEFINE_PER_CPU_FIRST(type, name)				\
5028eaa97d David Howells       2009-04-21  126  	DEFINE_PER_CPU_SECTION(type, name, PER_CPU_FIRST_SECTION)
5028eaa97d David Howells       2009-04-21  127  
5028eaa97d David Howells       2009-04-21  128  /*
5028eaa97d David Howells       2009-04-21  129   * Declaration/definition used for per-CPU variables that must be cacheline
5028eaa97d David Howells       2009-04-21  130   * aligned under SMP conditions so that, whilst a particular instance of the
5028eaa97d David Howells       2009-04-21  131   * data corresponds to a particular CPU, inefficiencies due to direct access by
5028eaa97d David Howells       2009-04-21  132   * other CPUs are reduced by preventing the data from unnecessarily spanning
5028eaa97d David Howells       2009-04-21  133   * cachelines.
5028eaa97d David Howells       2009-04-21  134   *
5028eaa97d David Howells       2009-04-21  135   * An example of this would be statistical data, where each CPU's set of data
5028eaa97d David Howells       2009-04-21  136   * is updated by that CPU alone, but the data from across all CPUs is collated
5028eaa97d David Howells       2009-04-21  137   * by a CPU processing a read from a proc file.
5028eaa97d David Howells       2009-04-21  138   */
5028eaa97d David Howells       2009-04-21  139  #define DECLARE_PER_CPU_SHARED_ALIGNED(type, name)			\
5028eaa97d David Howells       2009-04-21  140  	DECLARE_PER_CPU_SECTION(type, name, PER_CPU_SHARED_ALIGNED_SECTION) \
5028eaa97d David Howells       2009-04-21  141  	____cacheline_aligned_in_smp
5028eaa97d David Howells       2009-04-21  142  
5028eaa97d David Howells       2009-04-21  143  #define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)			\
5028eaa97d David Howells       2009-04-21  144  	DEFINE_PER_CPU_SECTION(type, name, PER_CPU_SHARED_ALIGNED_SECTION) \
5028eaa97d David Howells       2009-04-21  145  	____cacheline_aligned_in_smp
5028eaa97d David Howells       2009-04-21  146  
53f824520b Jeremy Fitzhardinge 2009-09-03  147  #define DECLARE_PER_CPU_ALIGNED(type, name)				\
53f824520b Jeremy Fitzhardinge 2009-09-03  148  	DECLARE_PER_CPU_SECTION(type, name, PER_CPU_ALIGNED_SECTION)	\
53f824520b Jeremy Fitzhardinge 2009-09-03  149  	____cacheline_aligned
53f824520b Jeremy Fitzhardinge 2009-09-03  150  
53f824520b Jeremy Fitzhardinge 2009-09-03  151  #define DEFINE_PER_CPU_ALIGNED(type, name)				\
53f824520b Jeremy Fitzhardinge 2009-09-03  152  	DEFINE_PER_CPU_SECTION(type, name, PER_CPU_ALIGNED_SECTION)	\
53f824520b Jeremy Fitzhardinge 2009-09-03  153  	____cacheline_aligned
53f824520b Jeremy Fitzhardinge 2009-09-03  154  
5028eaa97d David Howells       2009-04-21  155  /*
5028eaa97d David Howells       2009-04-21  156   * Declaration/definition used for per-CPU variables that must be page aligned.
5028eaa97d David Howells       2009-04-21  157   */
5028eaa97d David Howells       2009-04-21  158  #define DECLARE_PER_CPU_PAGE_ALIGNED(type, name)			\
3d9a854c2d Denys Vlasenko      2010-02-20  159  	DECLARE_PER_CPU_SECTION(type, name, "..page_aligned")		\
3e352aa8ee Tejun Heo           2009-08-03  160  	__aligned(PAGE_SIZE)
5028eaa97d David Howells       2009-04-21  161  
5028eaa97d David Howells       2009-04-21  162  #define DEFINE_PER_CPU_PAGE_ALIGNED(type, name)				\
3d9a854c2d Denys Vlasenko      2010-02-20  163  	DEFINE_PER_CPU_SECTION(type, name, "..page_aligned")		\
3e352aa8ee Tejun Heo           2009-08-03  164  	__aligned(PAGE_SIZE)
5028eaa97d David Howells       2009-04-21  165  
5028eaa97d David Howells       2009-04-21  166  /*
c957ef2c59 Shaohua Li          2010-10-20  167   * Declaration/definition used for per-CPU variables that must be read mostly.
c957ef2c59 Shaohua Li          2010-10-20  168   */
c957ef2c59 Shaohua Li          2010-10-20  169  #define DECLARE_PER_CPU_READ_MOSTLY(type, name)			\
330d282216 Zhengyu He          2014-07-01  170  	DECLARE_PER_CPU_SECTION(type, name, "..read_mostly")
c957ef2c59 Shaohua Li          2010-10-20  171  
c957ef2c59 Shaohua Li          2010-10-20  172  #define DEFINE_PER_CPU_READ_MOSTLY(type, name)				\
330d282216 Zhengyu He          2014-07-01  173  	DEFINE_PER_CPU_SECTION(type, name, "..read_mostly")
c957ef2c59 Shaohua Li          2010-10-20  174  
c957ef2c59 Shaohua Li          2010-10-20  175  /*
ac26963a11 Brijesh Singh       2017-10-20  176   * Declaration/definition used for per-CPU variables that should be accessed
ac26963a11 Brijesh Singh       2017-10-20  177   * as decrypted when memory encryption is enabled in the guest.
ac26963a11 Brijesh Singh       2017-10-20  178   */
ac26963a11 Brijesh Singh       2017-10-20  179  #if defined(CONFIG_VIRTUALIZATION) && defined(CONFIG_AMD_MEM_ENCRYPT)
ac26963a11 Brijesh Singh       2017-10-20  180  
ac26963a11 Brijesh Singh       2017-10-20  181  #define DECLARE_PER_CPU_DECRYPTED(type, name)				\
ac26963a11 Brijesh Singh       2017-10-20  182  	DECLARE_PER_CPU_SECTION(type, name, "..decrypted")
ac26963a11 Brijesh Singh       2017-10-20  183  
ac26963a11 Brijesh Singh       2017-10-20  184  #define DEFINE_PER_CPU_DECRYPTED(type, name)				\
ac26963a11 Brijesh Singh       2017-10-20  185  	DEFINE_PER_CPU_SECTION(type, name, "..decrypted")
ac26963a11 Brijesh Singh       2017-10-20  186  #else
ac26963a11 Brijesh Singh       2017-10-20  187  #define DEFINE_PER_CPU_DECRYPTED(type, name)	DEFINE_PER_CPU(type, name)
ac26963a11 Brijesh Singh       2017-10-20  188  #endif
ac26963a11 Brijesh Singh       2017-10-20  189  
ac26963a11 Brijesh Singh       2017-10-20  190  /*
545695fb41 Tejun Heo           2009-10-29  191   * Intermodule exports for per-CPU variables.  sparse forgets about
545695fb41 Tejun Heo           2009-10-29  192   * address space across EXPORT_SYMBOL(), change EXPORT_SYMBOL() to
545695fb41 Tejun Heo           2009-10-29  193   * noop if __CHECKER__.
5028eaa97d David Howells       2009-04-21  194   */
545695fb41 Tejun Heo           2009-10-29  195  #ifndef __CHECKER__
dd17c8f729 Rusty Russell       2009-10-29  196  #define EXPORT_PER_CPU_SYMBOL(var) EXPORT_SYMBOL(var)
dd17c8f729 Rusty Russell       2009-10-29  197  #define EXPORT_PER_CPU_SYMBOL_GPL(var) EXPORT_SYMBOL_GPL(var)
545695fb41 Tejun Heo           2009-10-29  198  #else
545695fb41 Tejun Heo           2009-10-29  199  #define EXPORT_PER_CPU_SYMBOL(var)
545695fb41 Tejun Heo           2009-10-29  200  #define EXPORT_PER_CPU_SYMBOL_GPL(var)
545695fb41 Tejun Heo           2009-10-29  201  #endif
5028eaa97d David Howells       2009-04-21  202  
62fde54123 Tejun Heo           2014-06-17  203  /*
62fde54123 Tejun Heo           2014-06-17  204   * Accessors and operations.
62fde54123 Tejun Heo           2014-06-17  205   */
62fde54123 Tejun Heo           2014-06-17  206  #ifndef __ASSEMBLY__
62fde54123 Tejun Heo           2014-06-17  207  
9c28278a24 Tejun Heo           2014-06-17  208  /*
6fbc07bbe2 Tejun Heo           2014-06-17  209   * __verify_pcpu_ptr() verifies @ptr is a percpu pointer without evaluating
6fbc07bbe2 Tejun Heo           2014-06-17  210   * @ptr and is invoked once before a percpu area is accessed by all
6fbc07bbe2 Tejun Heo           2014-06-17  211   * accessors and operations.  This is performed in the generic part of
6fbc07bbe2 Tejun Heo           2014-06-17  212   * percpu and arch overrides don't need to worry about it; however, if an
6fbc07bbe2 Tejun Heo           2014-06-17  213   * arch wants to implement an arch-specific percpu accessor or operation,
6fbc07bbe2 Tejun Heo           2014-06-17  214   * it may use __verify_pcpu_ptr() to verify the parameters.
9c28278a24 Tejun Heo           2014-06-17  215   *
9c28278a24 Tejun Heo           2014-06-17  216   * + 0 is required in order to convert the pointer type from a
9c28278a24 Tejun Heo           2014-06-17  217   * potential array type to a pointer to a single item of the array.
9c28278a24 Tejun Heo           2014-06-17  218   */
eba117889a Tejun Heo           2014-06-17  219  #define __verify_pcpu_ptr(ptr)						\
eba117889a Tejun Heo           2014-06-17  220  do {									\
9c28278a24 Tejun Heo           2014-06-17 @221  	const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL;	\
9c28278a24 Tejun Heo           2014-06-17  222  	(void)__vpp_verify;						\
9c28278a24 Tejun Heo           2014-06-17  223  } while (0)
9c28278a24 Tejun Heo           2014-06-17  224  

:::::: The code at line 49 was first introduced by commit
:::::: e0fdb0e050eae331046385643618f12452aa7e73 percpu: add __percpu for sparse.

:::::: TO: Rusty Russell <rusty@rustcorp.com.au>
:::::: CC: Tejun Heo <tj@kernel.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 19734 bytes --]

Re: [PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2423 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: s390-performance_defconfig (attached as .config)
compiler: s390x-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.2.0 make.cross ARCH=s390 

All errors (new ones prefixed by >>):

   In file included from kernel//bpf/local_storage.c:2:0:
   include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_attach':
>> include/linux/bpf-cgroup.h:237:10: error: 'EINVAL' undeclared (first use in this function)
     return -EINVAL;
             ^~~~~~
   include/linux/bpf-cgroup.h:237:10: note: each undeclared identifier is reported only once for each function it appears in
   include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_detach':
   include/linux/bpf-cgroup.h:243:10: error: 'EINVAL' undeclared (first use in this function)
     return -EINVAL;
             ^~~~~~
   include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_query':
   include/linux/bpf-cgroup.h:249:10: error: 'EINVAL' undeclared (first use in this function)
     return -EINVAL;
             ^~~~~~

vim +/EINVAL +237 include/linux/bpf-cgroup.h

30070984 Daniel Mack 2016-11-23  232  
fdb5c453 Sean Young  2018-06-19  233  static inline int cgroup_bpf_prog_attach(const union bpf_attr *attr,
fdb5c453 Sean Young  2018-06-19  234  					 enum bpf_prog_type ptype,
fdb5c453 Sean Young  2018-06-19  235  					 struct bpf_prog *prog)
fdb5c453 Sean Young  2018-06-19  236  {
fdb5c453 Sean Young  2018-06-19 @237  	return -EINVAL;
fdb5c453 Sean Young  2018-06-19  238  }
fdb5c453 Sean Young  2018-06-19  239  

:::::: The code at line 237 was first introduced by commit
:::::: fdb5c4531c1e0e50e609df83f736b6f3a02896e2 bpf: fix attach type BPF_LIRC_MODE2 dependency wrt CONFIG_CGROUP_BPF

:::::: TO: Sean Young <sean@mess.org>
:::::: CC: Daniel Borkmann <daniel@iogearbox.net>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 19173 bytes --]

Re: [PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[Roman Gushchin]

On Fri, Jul 06, 2018 at 10:42:39PM +0800, kbuild test robot wrote:
> Hi Roman,
> 
> Thank you for the patch! Yet something to improve:
> 
> [auto build test ERROR on bpf-next/master]
> 
> url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
> config: s390-performance_defconfig (attached as .config)
> compiler: s390x-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
> reproduce:
>         wget https://urldefense.proofpoint.com/v2/url?u=https-3A__raw.githubusercontent.com_intel_lkp-2Dtests_master_sbin_make.cross&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=ImwiAAR6DVyf5JEcIc5VLF9xY4lfFxVuk8_4BN6I03g&s=r0S7gFQPOZVZagshsK0pL3eU8_GRpD1ywrvYfXJ4yBQ&e= -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         GCC_VERSION=7.2.0 make.cross ARCH=s390 
> 
> All errors (new ones prefixed by >>):
> 
>    In file included from kernel//bpf/local_storage.c:2:0:
>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_attach':
> >> include/linux/bpf-cgroup.h:237:10: error: 'EINVAL' undeclared (first use in this function)
>      return -EINVAL;
>              ^~~~~~
>    include/linux/bpf-cgroup.h:237:10: note: each undeclared identifier is reported only once for each function it appears in
>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_detach':
>    include/linux/bpf-cgroup.h:243:10: error: 'EINVAL' undeclared (first use in this function)
>      return -EINVAL;
>              ^~~~~~
>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_query':
>    include/linux/bpf-cgroup.h:249:10: error: 'EINVAL' undeclared (first use in this function)
>      return -EINVAL;
>              ^~~~~~
> 
> vim +/EINVAL +237 include/linux/bpf-cgroup.h
> 
> 30070984 Daniel Mack 2016-11-23  232  
> fdb5c453 Sean Young  2018-06-19  233  static inline int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> fdb5c453 Sean Young  2018-06-19  234  					 enum bpf_prog_type ptype,
> fdb5c453 Sean Young  2018-06-19  235  					 struct bpf_prog *prog)
> fdb5c453 Sean Young  2018-06-19  236  {
> fdb5c453 Sean Young  2018-06-19 @237  	return -EINVAL;
> fdb5c453 Sean Young  2018-06-19  238  }
> fdb5c453 Sean Young  2018-06-19  239  
> 
> :::::: The code at line 237 was first introduced by commit
> :::::: fdb5c4531c1e0e50e609df83f736b6f3a02896e2 bpf: fix attach type BPF_LIRC_MODE2 dependency wrt CONFIG_CGROUP_BPF
> 
> :::::: TO: Sean Young <sean@mess.org>
> :::::: CC: Daniel Borkmann <daniel@iogearbox.net>

These errors have nothing to do with the cgroup local storage patchset.
They do exist in the current bpf-next tree.
Here is the fix.

Thanks!

--

From c0cd59b969e060765514224e31595763213e40f9 Mon Sep 17 00:00:00 2001
From: Roman Gushchin <guro@fb.com>
Date: Fri, 6 Jul 2018 14:34:29 -0700
Subject: [PATCH bpf-next] bpf: include errno.h from bpf-cgroup.h

Commit fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
wrt CONFIG_CGROUP_BPF") caused some build issues, detected by 0-DAY
kernel test infrastructure.

The problem is that cgroup_bpf_prog_attach/detach/query() functions
can return -EINVAL error code, which is not defined. Fix this adding
errno.h to includes.

Fixes: fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
wrt CONFIG_CGROUP_BPF")
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Sean Young <sean@mess.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@kernel.org>
---
 include/linux/bpf-cgroup.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 79795c5fa7c3..d50c2f0a655a 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -2,6 +2,7 @@
 #ifndef _BPF_CGROUP_H
 #define _BPF_CGROUP_H
 
+#include <linux/errno.h>
 #include <linux/jump_label.h>
 #include <uapi/linux/bpf.h>
 
-- 
2.14.4

Re: [PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[Alexei Starovoitov]

On Fri, Jul 06, 2018 at 02:52:32PM -0700, Roman Gushchin wrote:
> On Fri, Jul 06, 2018 at 10:42:39PM +0800, kbuild test robot wrote:
> > Hi Roman,
> > 
> > Thank you for the patch! Yet something to improve:
> > 
> > [auto build test ERROR on bpf-next/master]
> > 
> > url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
> > base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
> > config: s390-performance_defconfig (attached as .config)
> > compiler: s390x-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
> > reproduce:
> >         wget https://urldefense.proofpoint.com/v2/url?u=https-3A__raw.githubusercontent.com_intel_lkp-2Dtests_master_sbin_make.cross&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=ImwiAAR6DVyf5JEcIc5VLF9xY4lfFxVuk8_4BN6I03g&s=r0S7gFQPOZVZagshsK0pL3eU8_GRpD1ywrvYfXJ4yBQ&e= -O ~/bin/make.cross
> >         chmod +x ~/bin/make.cross
> >         # save the attached .config to linux build tree
> >         GCC_VERSION=7.2.0 make.cross ARCH=s390 
> > 
> > All errors (new ones prefixed by >>):
> > 
> >    In file included from kernel//bpf/local_storage.c:2:0:
> >    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_attach':
> > >> include/linux/bpf-cgroup.h:237:10: error: 'EINVAL' undeclared (first use in this function)
> >      return -EINVAL;
> >              ^~~~~~
> >    include/linux/bpf-cgroup.h:237:10: note: each undeclared identifier is reported only once for each function it appears in
> >    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_detach':
> >    include/linux/bpf-cgroup.h:243:10: error: 'EINVAL' undeclared (first use in this function)
> >      return -EINVAL;
> >              ^~~~~~
> >    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_query':
> >    include/linux/bpf-cgroup.h:249:10: error: 'EINVAL' undeclared (first use in this function)
> >      return -EINVAL;
> >              ^~~~~~
> > 
> > vim +/EINVAL +237 include/linux/bpf-cgroup.h
> > 
> > 30070984 Daniel Mack 2016-11-23  232  
> > fdb5c453 Sean Young  2018-06-19  233  static inline int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> > fdb5c453 Sean Young  2018-06-19  234  					 enum bpf_prog_type ptype,
> > fdb5c453 Sean Young  2018-06-19  235  					 struct bpf_prog *prog)
> > fdb5c453 Sean Young  2018-06-19  236  {
> > fdb5c453 Sean Young  2018-06-19 @237  	return -EINVAL;
> > fdb5c453 Sean Young  2018-06-19  238  }
> > fdb5c453 Sean Young  2018-06-19  239  
> > 
> > :::::: The code at line 237 was first introduced by commit
> > :::::: fdb5c4531c1e0e50e609df83f736b6f3a02896e2 bpf: fix attach type BPF_LIRC_MODE2 dependency wrt CONFIG_CGROUP_BPF
> > 
> > :::::: TO: Sean Young <sean@mess.org>
> > :::::: CC: Daniel Borkmann <daniel@iogearbox.net>
> 
> These errors have nothing to do with the cgroup local storage patchset.
> They do exist in the current bpf-next tree.
> Here is the fix.
> 
> Thanks!
> 
> --
> 
> From c0cd59b969e060765514224e31595763213e40f9 Mon Sep 17 00:00:00 2001
> From: Roman Gushchin <guro@fb.com>
> Date: Fri, 6 Jul 2018 14:34:29 -0700
> Subject: [PATCH bpf-next] bpf: include errno.h from bpf-cgroup.h
> 
> Commit fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
> wrt CONFIG_CGROUP_BPF") caused some build issues, detected by 0-DAY
> kernel test infrastructure.
> 
> The problem is that cgroup_bpf_prog_attach/detach/query() functions
> can return -EINVAL error code, which is not defined. Fix this adding
> errno.h to includes.
> 
> Fixes: fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
> wrt CONFIG_CGROUP_BPF")
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Sean Young <sean@mess.org>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: Alexei Starovoitov <ast@kernel.org>
> ---
>  include/linux/bpf-cgroup.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> index 79795c5fa7c3..d50c2f0a655a 100644
> --- a/include/linux/bpf-cgroup.h
> +++ b/include/linux/bpf-cgroup.h
> @@ -2,6 +2,7 @@
>  #ifndef _BPF_CGROUP_H
>  #define _BPF_CGROUP_H
>  
> +#include <linux/errno.h>
>  #include <linux/jump_label.h>
>  #include <uapi/linux/bpf.h>

good catch, but please resend it as a proper patch,
since patchwork didn't pick it up.
Also the subj should say 'bpf' instead of 'bpf-next', right?

Re: [PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[Daniel Borkmann]

On 07/08/2018 12:35 AM, Alexei Starovoitov wrote:
> On Fri, Jul 06, 2018 at 02:52:32PM -0700, Roman Gushchin wrote:
>> On Fri, Jul 06, 2018 at 10:42:39PM +0800, kbuild test robot wrote:
>>> Hi Roman,
>>>
>>> Thank you for the patch! Yet something to improve:
>>>
>>> [auto build test ERROR on bpf-next/master]
>>>
>>> url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
>>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
>>> config: s390-performance_defconfig (attached as .config)
>>> compiler: s390x-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
>>> reproduce:
>>>         wget https://urldefense.proofpoint.com/v2/url?u=https-3A__raw.githubusercontent.com_intel_lkp-2Dtests_master_sbin_make.cross&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=ImwiAAR6DVyf5JEcIc5VLF9xY4lfFxVuk8_4BN6I03g&s=r0S7gFQPOZVZagshsK0pL3eU8_GRpD1ywrvYfXJ4yBQ&e= -O ~/bin/make.cross
>>>         chmod +x ~/bin/make.cross
>>>         # save the attached .config to linux build tree
>>>         GCC_VERSION=7.2.0 make.cross ARCH=s390 
>>>
>>> All errors (new ones prefixed by >>):
>>>
>>>    In file included from kernel//bpf/local_storage.c:2:0:
>>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_attach':
>>>>> include/linux/bpf-cgroup.h:237:10: error: 'EINVAL' undeclared (first use in this function)
>>>      return -EINVAL;
>>>              ^~~~~~
>>>    include/linux/bpf-cgroup.h:237:10: note: each undeclared identifier is reported only once for each function it appears in
>>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_detach':
>>>    include/linux/bpf-cgroup.h:243:10: error: 'EINVAL' undeclared (first use in this function)
>>>      return -EINVAL;
>>>              ^~~~~~
>>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_query':
>>>    include/linux/bpf-cgroup.h:249:10: error: 'EINVAL' undeclared (first use in this function)
>>>      return -EINVAL;
>>>              ^~~~~~
>>>
>>> vim +/EINVAL +237 include/linux/bpf-cgroup.h
>>>
>>> 30070984 Daniel Mack 2016-11-23  232  
>>> fdb5c453 Sean Young  2018-06-19  233  static inline int cgroup_bpf_prog_attach(const union bpf_attr *attr,
>>> fdb5c453 Sean Young  2018-06-19  234  					 enum bpf_prog_type ptype,
>>> fdb5c453 Sean Young  2018-06-19  235  					 struct bpf_prog *prog)
>>> fdb5c453 Sean Young  2018-06-19  236  {
>>> fdb5c453 Sean Young  2018-06-19 @237  	return -EINVAL;
>>> fdb5c453 Sean Young  2018-06-19  238  }
>>> fdb5c453 Sean Young  2018-06-19  239  
>>>
>>> :::::: The code at line 237 was first introduced by commit
>>> :::::: fdb5c4531c1e0e50e609df83f736b6f3a02896e2 bpf: fix attach type BPF_LIRC_MODE2 dependency wrt CONFIG_CGROUP_BPF
>>>
>>> :::::: TO: Sean Young <sean@mess.org>
>>> :::::: CC: Daniel Borkmann <daniel@iogearbox.net>
>>
>> These errors have nothing to do with the cgroup local storage patchset.
>> They do exist in the current bpf-next tree.
>> Here is the fix.
>>
>> Thanks!
>>
>> --
>>
>> From c0cd59b969e060765514224e31595763213e40f9 Mon Sep 17 00:00:00 2001
>> From: Roman Gushchin <guro@fb.com>
>> Date: Fri, 6 Jul 2018 14:34:29 -0700
>> Subject: [PATCH bpf-next] bpf: include errno.h from bpf-cgroup.h
>>
>> Commit fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
>> wrt CONFIG_CGROUP_BPF") caused some build issues, detected by 0-DAY
>> kernel test infrastructure.
>>
>> The problem is that cgroup_bpf_prog_attach/detach/query() functions
>> can return -EINVAL error code, which is not defined. Fix this adding
>> errno.h to includes.
>>
>> Fixes: fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
>> wrt CONFIG_CGROUP_BPF")
>> Signed-off-by: Roman Gushchin <guro@fb.com>
>> Cc: Sean Young <sean@mess.org>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: Alexei Starovoitov <ast@kernel.org>
>> ---
>>  include/linux/bpf-cgroup.h | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
>> index 79795c5fa7c3..d50c2f0a655a 100644
>> --- a/include/linux/bpf-cgroup.h
>> +++ b/include/linux/bpf-cgroup.h
>> @@ -2,6 +2,7 @@
>>  #ifndef _BPF_CGROUP_H
>>  #define _BPF_CGROUP_H
>>  
>> +#include <linux/errno.h>
>>  #include <linux/jump_label.h>
>>  #include <uapi/linux/bpf.h>
> 
> good catch, but please resend it as a proper patch,
> since patchwork didn't pick it up.
> Also the subj should say 'bpf' instead of 'bpf-next', right?

Yes, this needs to go to bpf tree, since the affected commit went
there as well.

I just applied the above fix into bpf tree, thanks.

Cheers,
Daniel

Re: [PATCH v2 bpf-next 02/14] bpf: introduce cgroup storage maps

[Roman Gushchin]

On Mon, Jul 09, 2018 at 10:37:45AM +0200, Daniel Borkmann wrote:
> On 07/08/2018 12:35 AM, Alexei Starovoitov wrote:
> > On Fri, Jul 06, 2018 at 02:52:32PM -0700, Roman Gushchin wrote:
> >> On Fri, Jul 06, 2018 at 10:42:39PM +0800, kbuild test robot wrote:
> >>> Hi Roman,
> >>>
> >>> Thank you for the patch! Yet something to improve:
> >>>
> >>> [auto build test ERROR on bpf-next/master]
> >>>
> >>> url:    https://github.com/0day-ci/linux/commits/Roman-Gushchin/bpf-cgroup-local-storage/20180706-055938
> >>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
> >>> config: s390-performance_defconfig (attached as .config)
> >>> compiler: s390x-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
> >>> reproduce:
> >>>         wget https://urldefense.proofpoint.com/v2/url?u=https-3A__raw.githubusercontent.com_intel_lkp-2Dtests_master_sbin_make.cross&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=ImwiAAR6DVyf5JEcIc5VLF9xY4lfFxVuk8_4BN6I03g&s=r0S7gFQPOZVZagshsK0pL3eU8_GRpD1ywrvYfXJ4yBQ&e= -O ~/bin/make.cross
> >>>         chmod +x ~/bin/make.cross
> >>>         # save the attached .config to linux build tree
> >>>         GCC_VERSION=7.2.0 make.cross ARCH=s390 
> >>>
> >>> All errors (new ones prefixed by >>):
> >>>
> >>>    In file included from kernel//bpf/local_storage.c:2:0:
> >>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_attach':
> >>>>> include/linux/bpf-cgroup.h:237:10: error: 'EINVAL' undeclared (first use in this function)
> >>>      return -EINVAL;
> >>>              ^~~~~~
> >>>    include/linux/bpf-cgroup.h:237:10: note: each undeclared identifier is reported only once for each function it appears in
> >>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_detach':
> >>>    include/linux/bpf-cgroup.h:243:10: error: 'EINVAL' undeclared (first use in this function)
> >>>      return -EINVAL;
> >>>              ^~~~~~
> >>>    include/linux/bpf-cgroup.h: In function 'cgroup_bpf_prog_query':
> >>>    include/linux/bpf-cgroup.h:249:10: error: 'EINVAL' undeclared (first use in this function)
> >>>      return -EINVAL;
> >>>              ^~~~~~
> >>>
> >>> vim +/EINVAL +237 include/linux/bpf-cgroup.h
> >>>
> >>> 30070984 Daniel Mack 2016-11-23  232  
> >>> fdb5c453 Sean Young  2018-06-19  233  static inline int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> >>> fdb5c453 Sean Young  2018-06-19  234  					 enum bpf_prog_type ptype,
> >>> fdb5c453 Sean Young  2018-06-19  235  					 struct bpf_prog *prog)
> >>> fdb5c453 Sean Young  2018-06-19  236  {
> >>> fdb5c453 Sean Young  2018-06-19 @237  	return -EINVAL;
> >>> fdb5c453 Sean Young  2018-06-19  238  }
> >>> fdb5c453 Sean Young  2018-06-19  239  
> >>>
> >>> :::::: The code at line 237 was first introduced by commit
> >>> :::::: fdb5c4531c1e0e50e609df83f736b6f3a02896e2 bpf: fix attach type BPF_LIRC_MODE2 dependency wrt CONFIG_CGROUP_BPF
> >>>
> >>> :::::: TO: Sean Young <sean@mess.org>
> >>> :::::: CC: Daniel Borkmann <daniel@iogearbox.net>
> >>
> >> These errors have nothing to do with the cgroup local storage patchset.
> >> They do exist in the current bpf-next tree.
> >> Here is the fix.
> >>
> >> Thanks!
> >>
> >> --
> >>
> >> From c0cd59b969e060765514224e31595763213e40f9 Mon Sep 17 00:00:00 2001
> >> From: Roman Gushchin <guro@fb.com>
> >> Date: Fri, 6 Jul 2018 14:34:29 -0700
> >> Subject: [PATCH bpf-next] bpf: include errno.h from bpf-cgroup.h
> >>
> >> Commit fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
> >> wrt CONFIG_CGROUP_BPF") caused some build issues, detected by 0-DAY
> >> kernel test infrastructure.
> >>
> >> The problem is that cgroup_bpf_prog_attach/detach/query() functions
> >> can return -EINVAL error code, which is not defined. Fix this adding
> >> errno.h to includes.
> >>
> >> Fixes: fdb5c4531c1e ("bpf: fix attach type BPF_LIRC_MODE2 dependency
> >> wrt CONFIG_CGROUP_BPF")
> >> Signed-off-by: Roman Gushchin <guro@fb.com>
> >> Cc: Sean Young <sean@mess.org>
> >> Cc: Daniel Borkmann <daniel@iogearbox.net>
> >> Cc: Alexei Starovoitov <ast@kernel.org>
> >> ---
> >>  include/linux/bpf-cgroup.h | 1 +
> >>  1 file changed, 1 insertion(+)
> >>
> >> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> >> index 79795c5fa7c3..d50c2f0a655a 100644
> >> --- a/include/linux/bpf-cgroup.h
> >> +++ b/include/linux/bpf-cgroup.h
> >> @@ -2,6 +2,7 @@
> >>  #ifndef _BPF_CGROUP_H
> >>  #define _BPF_CGROUP_H
> >>  
> >> +#include <linux/errno.h>
> >>  #include <linux/jump_label.h>
> >>  #include <uapi/linux/bpf.h>
> > 
> > good catch, but please resend it as a proper patch,
> > since patchwork didn't pick it up.
> > Also the subj should say 'bpf' instead of 'bpf-next', right?
> 
> Yes, this needs to go to bpf tree, since the affected commit went
> there as well.
> 
> I just applied the above fix into bpf tree, thanks.

Thanks, Daniel!