[PATCH bpf-next v2 0/8] bpf: getsockopt and setsockopt hooks

[PATCH bpf-next v2 0/8] bpf: getsockopt and setsockopt hooks

[Stanislav Fomichev]

This series implements two new per-cgroup hooks: getsockopt and
setsockopt along with a new sockopt program type. The idea is pretty
similar to recently introduced cgroup sysctl hooks, but
implementation is simpler (no need to convert to/from strings).

What this can be applied to:
* move business logic of what tos/priority/etc can be set by
  containers (either pass or reject)
* handle existing options (or introduce new ones) differently by
  propagating some information in cgroup/socket local storage

Compared to a simple syscall/{g,s}etsockopt tracepoint, those
hooks are context aware. Meaning, they can access underlying socket
and use cgroup and socket local storage.

Stanislav Fomichev (8):
  bpf: implement getsockopt and setsockopt hooks
  bpf: sync bpf.h to tools/
  libbpf: support sockopt hooks
  selftests/bpf: test sockopt section name
  selftests/bpf: add sockopt test
  selftests/bpf: add sockopt test that exercises sk helpers
  bpf: add sockopt documentation
  bpftool: support cgroup sockopt

 Documentation/bpf/index.rst                   |   1 +
 Documentation/bpf/prog_cgroup_sockopt.rst     |  39 +
 include/linux/bpf-cgroup.h                    |  29 +
 include/linux/bpf.h                           |  46 ++
 include/linux/bpf_types.h                     |   1 +
 include/linux/filter.h                        |  13 +
 include/uapi/linux/bpf.h                      |  14 +
 kernel/bpf/cgroup.c                           | 277 +++++++
 kernel/bpf/core.c                             |   9 +
 kernel/bpf/syscall.c                          |  19 +
 kernel/bpf/verifier.c                         |  15 +
 net/core/filter.c                             |   4 +-
 net/socket.c                                  |  18 +
 .../bpftool/Documentation/bpftool-cgroup.rst  |   7 +-
 .../bpftool/Documentation/bpftool-prog.rst    |   2 +-
 tools/bpf/bpftool/bash-completion/bpftool     |   8 +-
 tools/bpf/bpftool/cgroup.c                    |   5 +-
 tools/bpf/bpftool/main.h                      |   1 +
 tools/bpf/bpftool/prog.c                      |   3 +-
 tools/include/uapi/linux/bpf.h                |  14 +
 tools/lib/bpf/libbpf.c                        |   5 +
 tools/lib/bpf/libbpf_probes.c                 |   1 +
 tools/testing/selftests/bpf/.gitignore        |   2 +
 tools/testing/selftests/bpf/Makefile          |   4 +-
 .../testing/selftests/bpf/progs/sockopt_sk.c  |  77 ++
 .../selftests/bpf/test_section_names.c        |  10 +
 tools/testing/selftests/bpf/test_sockopt.c    | 773 ++++++++++++++++++
 tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++
 28 files changed, 1542 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst
 create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c
 create mode 100644 tools/testing/selftests/bpf/test_sockopt.c
 create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c

-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Stanislav Fomichev]

Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.

BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.

The buffer memory is pre-allocated (because I don't think there is
a precedent for working with __user memory from bpf). This might be
slow to do for each {s,g}etsockopt call, that's why I've added
__cgroup_bpf_prog_array_is_empty that exits early if there is nothing
attached to a cgroup. Note, however, that there is a race between
__cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
program layout might have changed; this should not be a problem
because in general there is a race between multiple calls to
{s,g}etsocktop and user adding/removing bpf progs from a cgroup.

The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
     prog exits

v2:
* moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
* aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
* bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
* dropped bpf_target_off from ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 include/linux/bpf-cgroup.h |  29 ++++
 include/linux/bpf.h        |  46 ++++++
 include/linux/bpf_types.h  |   1 +
 include/linux/filter.h     |  13 ++
 include/uapi/linux/bpf.h   |  14 ++
 kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
 kernel/bpf/core.c          |   9 ++
 kernel/bpf/syscall.c       |  19 +++
 kernel/bpf/verifier.c      |  15 ++
 net/core/filter.c          |   4 +-
 net/socket.c               |  18 +++
 11 files changed, 443 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index b631ee75762d..406f1ba82531 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
 				   loff_t *ppos, void **new_buf,
 				   enum bpf_attach_type type);
 
+int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
+				       int optname, char __user *optval,
+				       unsigned int optlen);
+int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
+				       int optname, char __user *optval,
+				       int __user *optlen);
+
 static inline enum bpf_cgroup_storage_type cgroup_storage_type(
 	struct bpf_map *map)
 {
@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
 	__ret;								       \
 })
 
+#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
+({									       \
+	int __ret = 0;							       \
+	if (cgroup_bpf_enabled)						       \
+		__ret = __cgroup_bpf_run_filter_setsockopt(sock, level,	       \
+							   optname, optval,    \
+							   optlen);	       \
+	__ret;								       \
+})
+
+#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
+({									       \
+	int __ret = 0;							       \
+	if (cgroup_bpf_enabled)						       \
+		__ret = __cgroup_bpf_run_filter_getsockopt(sock, level,	       \
+							   optname, optval,    \
+							   optlen);	       \
+	__ret;								       \
+})
+
 int cgroup_bpf_prog_attach(const union bpf_attr *attr,
 			   enum bpf_prog_type ptype, struct bpf_prog *prog);
 int cgroup_bpf_prog_detach(const union bpf_attr *attr,
@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
 #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
 #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
 #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
+#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
+#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
 
 #define for_each_cgroup_storage_type(stype) for (; false; )
 
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e5a309e6a400..883a190bc0b8 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -520,6 +520,7 @@ struct bpf_prog_array {
 struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
 void bpf_prog_array_free(struct bpf_prog_array *progs);
 int bpf_prog_array_length(struct bpf_prog_array *progs);
+bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
 int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
 				__u32 __user *prog_ids, u32 cnt);
 
@@ -606,6 +607,49 @@ _out:							\
 		_ret;					\
 	})
 
+/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
+ *
+ * Expected BPF program return values are:
+ *   0: return -EPERM to the userspace
+ *   1: sockopt was not handled by BPF, kernel should do it
+ *   2: sockopt was handled by BPF, kernel not should do it and return
+ *      to the userspace instead
+ *
+ * Note, that return '0' takes precedence over everything else. In other
+ * words, if any single program in the prog array has returned 0,
+ * the userspace will get -EPERM (regardless of what other programs
+ * return).
+ *
+ * The macro itself returns:
+ *        0: sockopt was not handled by BPF, kernel should do it
+ *        1: sockopt was handled by BPF, kernel snot hould do it
+ *   -EPERM: return error back to userspace
+ */
+#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)		\
+	({								\
+		struct bpf_prog_array_item *_item;			\
+		struct bpf_prog *_prog;					\
+		struct bpf_prog_array *_array;				\
+		u32 ret;						\
+		u32 _success = 1;					\
+		u32 _bypass = 0;					\
+		preempt_disable();					\
+		rcu_read_lock();					\
+		_array = rcu_dereference(array);			\
+		_item = &_array->items[0];				\
+		while ((_prog = READ_ONCE(_item->prog))) {		\
+			bpf_cgroup_storage_set(_item->cgroup_storage);	\
+			ret = func(_prog, ctx);				\
+			_success &= (ret > 0);				\
+			_bypass |= (ret == 2);				\
+			_item++;					\
+		}							\
+		rcu_read_unlock();					\
+		preempt_enable();					\
+		ret = _success ? _bypass : -EPERM;			\
+		ret;							\
+	})
+
 #define BPF_PROG_RUN_ARRAY(array, ctx, func)		\
 	__BPF_PROG_RUN_ARRAY(array, ctx, func, false)
 
@@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
 extern const struct bpf_func_proto bpf_get_local_storage_proto;
 extern const struct bpf_func_proto bpf_strtol_proto;
 extern const struct bpf_func_proto bpf_strtoul_proto;
+extern const struct bpf_func_proto bpf_sk_fullsock_proto;
+extern const struct bpf_func_proto bpf_tcp_sock_proto;
 
 /* Shared helpers among cBPF and eBPF. */
 void bpf_user_rnd_init_once(void);
diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
index 5a9975678d6f..eec5aeeeaf92 100644
--- a/include/linux/bpf_types.h
+++ b/include/linux/bpf_types.h
@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
 #ifdef CONFIG_CGROUP_BPF
 BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
 BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
+BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
 #endif
 #ifdef CONFIG_BPF_LIRC_MODE2
 BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 43b45d6db36d..6e64d01e4e36 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
 	u64 tmp_reg;
 };
 
+struct bpf_sockopt_kern {
+	struct sock	*sk;
+	u8		*optval;
+	u8		*optval_end;
+	s32		level;
+	s32		optname;
+	u32		optlen;
+
+	/* Small on-stack optval buffer to avoid small allocations.
+	 */
+	u8 buf[64] __aligned(8);
+};
+
 #endif /* __LINUX_FILTER_H__ */
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 7c6aef253173..310b6bbfded8 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -170,6 +170,7 @@ enum bpf_prog_type {
 	BPF_PROG_TYPE_FLOW_DISSECTOR,
 	BPF_PROG_TYPE_CGROUP_SYSCTL,
 	BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
+	BPF_PROG_TYPE_CGROUP_SOCKOPT,
 };
 
 enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {
 	BPF_LIRC_MODE2,
 	BPF_FLOW_DISSECTOR,
 	BPF_CGROUP_SYSCTL,
+	BPF_CGROUP_GETSOCKOPT,
+	BPF_CGROUP_SETSOCKOPT,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -3533,4 +3536,15 @@ struct bpf_sysctl {
 				 */
 };
 
+struct bpf_sockopt {
+	__bpf_md_ptr(struct bpf_sock *, sk);
+
+	__s32	level;
+	__s32	optname;
+
+	__u32	optlen;
+	__u32	optval;
+	__u32	optval_end;
+};
+
 #endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 1b65ab0df457..04bc1a09464e 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -18,6 +18,7 @@
 #include <linux/bpf.h>
 #include <linux/bpf-cgroup.h>
 #include <net/sock.h>
+#include <net/bpf_sk_storage.h>
 
 DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
 EXPORT_SYMBOL(cgroup_bpf_enabled_key);
@@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
 }
 EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
 
+static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
+					     enum bpf_attach_type attach_type)
+{
+	struct bpf_prog_array *prog_array;
+	bool empty;
+
+	rcu_read_lock();
+	prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
+	empty = bpf_prog_array_is_empty(prog_array);
+	rcu_read_unlock();
+
+	return empty;
+}
+
+static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
+{
+	if (unlikely(max_optlen > PAGE_SIZE))
+		return -EINVAL;
+
+	if (likely(max_optlen <= sizeof(ctx->buf))) {
+		ctx->optval = ctx->buf;
+	} else {
+		ctx->optval = kzalloc(max_optlen, GFP_USER);
+		if (!ctx->optval)
+			return -ENOMEM;
+	}
+
+	ctx->optval_end = ctx->optval + max_optlen;
+	ctx->optlen = max_optlen;
+
+	return 0;
+}
+
+static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
+{
+	if (unlikely(ctx->optval != ctx->buf))
+		kfree(ctx->optval);
+}
+
+int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
+				       int optname, char __user *optval,
+				       unsigned int optlen)
+{
+	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
+	struct bpf_sockopt_kern ctx = {
+		.sk = sk,
+		.level = level,
+		.optname = optname,
+	};
+	int ret;
+
+	/* Opportunistic check to see whether we have any BPF program
+	 * attached to the hook so we don't waste time allocating
+	 * memory and locking the socket.
+	 */
+	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
+		return 0;
+
+	ret = sockopt_alloc_buf(&ctx, optlen);
+	if (ret)
+		return ret;
+
+	if (copy_from_user(ctx.optval, optval, optlen) != 0) {
+		sockopt_free_buf(&ctx);
+		return -EFAULT;
+	}
+
+	lock_sock(sk);
+	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
+		cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
+		&ctx, BPF_PROG_RUN);
+	release_sock(sk);
+
+	sockopt_free_buf(&ctx);
+
+	return ret;
+}
+EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
+
+int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
+				       int optname, char __user *optval,
+				       int __user *optlen)
+{
+	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
+	struct bpf_sockopt_kern ctx = {
+		.sk = sk,
+		.level = level,
+		.optname = optname,
+	};
+	int max_optlen;
+	int ret;
+
+	/* Opportunistic check to see whether we have any BPF program
+	 * attached to the hook so we don't waste time allocating
+	 * memory and locking the socket.
+	 */
+	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
+		return 0;
+
+	if (get_user(max_optlen, optlen))
+		return -EFAULT;
+
+	ret = sockopt_alloc_buf(&ctx, max_optlen);
+	if (ret)
+		return ret;
+
+	lock_sock(sk);
+	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
+		cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
+		&ctx, BPF_PROG_RUN);
+	release_sock(sk);
+
+	if (ret < 0) {
+		sockopt_free_buf(&ctx);
+		return ret;
+	}
+
+	if (ctx.optlen > max_optlen) {
+		sockopt_free_buf(&ctx);
+		return -EFAULT;
+	}
+
+	if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
+		sockopt_free_buf(&ctx);
+		return -EFAULT;
+	}
+
+	sockopt_free_buf(&ctx);
+
+	if (put_user(ctx.optlen, optlen))
+		return -EFAULT;
+
+	return ret;
+}
+EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
+
 static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
 			      size_t *lenp)
 {
@@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
 
 const struct bpf_prog_ops cg_sysctl_prog_ops = {
 };
+
+static const struct bpf_func_proto *
+cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
+{
+	switch (func_id) {
+	case BPF_FUNC_sk_fullsock:
+		return &bpf_sk_fullsock_proto;
+	case BPF_FUNC_sk_storage_get:
+		return &bpf_sk_storage_get_proto;
+	case BPF_FUNC_sk_storage_delete:
+		return &bpf_sk_storage_delete_proto;
+#ifdef CONFIG_INET
+	case BPF_FUNC_tcp_sock:
+		return &bpf_tcp_sock_proto;
+#endif
+	default:
+		return cgroup_base_func_proto(func_id, prog);
+	}
+}
+
+static bool cg_sockopt_is_valid_access(int off, int size,
+				       enum bpf_access_type type,
+				       const struct bpf_prog *prog,
+				       struct bpf_insn_access_aux *info)
+{
+	const int size_default = sizeof(__u32);
+
+	if (off < 0 || off >= sizeof(struct bpf_sockopt))
+		return false;
+
+	if (off % size != 0)
+		return false;
+
+	if (type == BPF_WRITE) {
+		switch (off) {
+		case offsetof(struct bpf_sockopt, optlen):
+			if (size != size_default)
+				return false;
+			return prog->expected_attach_type ==
+				BPF_CGROUP_GETSOCKOPT;
+		default:
+			return false;
+		}
+	}
+
+	switch (off) {
+	case offsetof(struct bpf_sockopt, sk):
+		if (size != sizeof(__u64))
+			return false;
+		info->reg_type = PTR_TO_SOCKET;
+		break;
+	case bpf_ctx_range(struct bpf_sockopt, optval):
+		if (size != size_default)
+			return false;
+		info->reg_type = PTR_TO_PACKET;
+		break;
+	case bpf_ctx_range(struct bpf_sockopt, optval_end):
+		if (size != size_default)
+			return false;
+		info->reg_type = PTR_TO_PACKET_END;
+		break;
+	default:
+		if (size != size_default)
+			return false;
+		break;
+	}
+	return true;
+}
+
+static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
+					 const struct bpf_insn *si,
+					 struct bpf_insn *insn_buf,
+					 struct bpf_prog *prog,
+					 u32 *target_size)
+{
+	struct bpf_insn *insn = insn_buf;
+
+	switch (si->off) {
+	case offsetof(struct bpf_sockopt, sk):
+		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
+						       bpf_sockopt_kern, sk),
+				      si->dst_reg, si->src_reg,
+				      offsetof(struct bpf_sockopt_kern, sk));
+		break;
+	case offsetof(struct bpf_sockopt, level):
+		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
+				      offsetof(struct bpf_sockopt_kern, level));
+		break;
+	case offsetof(struct bpf_sockopt, optname):
+		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
+				      offsetof(struct bpf_sockopt_kern,
+					       optname));
+		break;
+	case offsetof(struct bpf_sockopt, optlen):
+		if (type == BPF_WRITE)
+			*insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
+					      offsetof(struct bpf_sockopt_kern,
+						       optlen));
+		else
+			*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
+					      offsetof(struct bpf_sockopt_kern,
+						       optlen));
+		break;
+	case offsetof(struct bpf_sockopt, optval):
+		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
+						       optval),
+				      si->dst_reg, si->src_reg,
+				      offsetof(struct bpf_sockopt_kern,
+					       optval));
+		break;
+	case offsetof(struct bpf_sockopt, optval_end):
+		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
+						       optval_end),
+				      si->dst_reg, si->src_reg,
+				      offsetof(struct bpf_sockopt_kern,
+					       optval_end));
+		break;
+	}
+
+	return insn - insn_buf;
+}
+
+static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
+				   bool direct_write,
+				   const struct bpf_prog *prog)
+{
+	/* Nothing to do for sockopt argument. The data is kzalloc'ated.
+	 */
+	return 0;
+}
+
+const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
+	.get_func_proto		= cg_sockopt_func_proto,
+	.is_valid_access	= cg_sockopt_is_valid_access,
+	.convert_ctx_access	= cg_sockopt_convert_ctx_access,
+	.gen_prologue		= cg_sockopt_get_prologue,
+};
+
+const struct bpf_prog_ops cg_sockopt_prog_ops = {
+};
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 33fb292f2e30..e9152ebd66bc 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)
 	return cnt;
 }
 
+bool bpf_prog_array_is_empty(struct bpf_prog_array *array)
+{
+	struct bpf_prog_array_item *item;
+
+	for (item = array->items; item->prog; item++)
+		if (item->prog != &dummy_bpf_prog.prog)
+			return false;
+	return true;
+}
 
 static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
 				     u32 *prog_ids,
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 4c53cbd3329d..4ad2b5f1905f 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,
 		default:
 			return -EINVAL;
 		}
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+		switch (expected_attach_type) {
+		case BPF_CGROUP_SETSOCKOPT:
+		case BPF_CGROUP_GETSOCKOPT:
+			return 0;
+		default:
+			return -EINVAL;
+		}
 	default:
 		return 0;
 	}
@@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
 	switch (prog->type) {
 	case BPF_PROG_TYPE_CGROUP_SOCK:
 	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
 		return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
 	case BPF_PROG_TYPE_CGROUP_SKB:
 		return prog->enforce_expected_attach_type &&
@@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)
 	case BPF_CGROUP_SYSCTL:
 		ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
 		break;
+	case BPF_CGROUP_GETSOCKOPT:
+	case BPF_CGROUP_SETSOCKOPT:
+		ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)
 	case BPF_CGROUP_SYSCTL:
 		ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
 		break;
+	case BPF_CGROUP_GETSOCKOPT:
+	case BPF_CGROUP_SETSOCKOPT:
+		ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,
 	case BPF_CGROUP_SOCK_OPS:
 	case BPF_CGROUP_DEVICE:
 	case BPF_CGROUP_SYSCTL:
+	case BPF_CGROUP_GETSOCKOPT:
+	case BPF_CGROUP_SETSOCKOPT:
 		break;
 	case BPF_LIRC_MODE2:
 		return lirc_prog_query(attr, uattr);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 5c2cb5bd84ce..fffc668ef536 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env,
 
 		env->seen_direct_write = true;
 		return true;
+
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+		if (t == BPF_WRITE) {
+			if (env->prog->expected_attach_type ==
+			    BPF_CGROUP_GETSOCKOPT) {
+				env->seen_direct_write = true;
+				return true;
+			}
+			return false;
+		}
+		return true;
+
 	default:
 		return false;
 	}
@@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)
 	case BPF_PROG_TYPE_CGROUP_DEVICE:
 	case BPF_PROG_TYPE_CGROUP_SYSCTL:
 		break;
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+		range = tnum_range(0, 2);
+		break;
 	default:
 		return 0;
 	}
diff --git a/net/core/filter.c b/net/core/filter.c
index 55bfc941d17a..4652c0a005ca 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
 	return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
 }
 
-static const struct bpf_func_proto bpf_sk_fullsock_proto = {
+const struct bpf_func_proto bpf_sk_fullsock_proto = {
 	.func		= bpf_sk_fullsock,
 	.gpl_only	= false,
 	.ret_type	= RET_PTR_TO_SOCKET_OR_NULL,
@@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
 	return (unsigned long)NULL;
 }
 
-static const struct bpf_func_proto bpf_tcp_sock_proto = {
+const struct bpf_func_proto bpf_tcp_sock_proto = {
 	.func		= bpf_tcp_sock,
 	.gpl_only	= false,
 	.ret_type	= RET_PTR_TO_TCP_SOCK_OR_NULL,
diff --git a/net/socket.c b/net/socket.c
index 72372dc5dd70..e8654f1f70e6 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,
 		if (err)
 			goto out_put;
 
+		err = BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock->sk, level, optname,
+						     optval, optlen);
+		if (err < 0) {
+			goto out_put;
+		} else if (err > 0) {
+			err = 0;
+			goto out_put;
+		}
+
 		if (level == SOL_SOCKET)
 			err =
 			    sock_setsockopt(sock, level, optname, optval,
@@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,
 		if (err)
 			goto out_put;
 
+		err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname,
+						     optval, optlen);
+		if (err < 0) {
+			goto out_put;
+		} else if (err > 0) {
+			err = 0;
+			goto out_put;
+		}
+
 		if (level == SOL_SOCKET)
 			err =
 			    sock_getsockopt(sock, level, optname, optval,
-- 
2.22.0.rc1.311.g5d7573a151-goog

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Andrii Nakryiko]

On Thu, Jun 6, 2019 at 12:03 PM Stanislav Fomichev <sdf@google.com> wrote:
>
> Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
>
> BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
>
> The buffer memory is pre-allocated (because I don't think there is
> a precedent for working with __user memory from bpf). This might be
> slow to do for each {s,g}etsockopt call, that's why I've added
> __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> attached to a cgroup. Note, however, that there is a race between
> __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> program layout might have changed; this should not be a problem
> because in general there is a race between multiple calls to
> {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
>
> The return code of the BPF program is handled as follows:
> * 0: EPERM
> * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
>      prog exits
>
> v2:
> * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> * added [0,2] return code check to verifier (Martin Lau)
> * dropped unused buf[64] from the stack (Martin Lau)
> * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> * dropped bpf_target_off from ctx rewrites (Martin Lau)
> * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
>
> Signed-off-by: Stanislav Fomichev <sdf@google.com>
> ---
>  include/linux/bpf-cgroup.h |  29 ++++
>  include/linux/bpf.h        |  46 ++++++
>  include/linux/bpf_types.h  |   1 +
>  include/linux/filter.h     |  13 ++
>  include/uapi/linux/bpf.h   |  14 ++
>  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
>  kernel/bpf/core.c          |   9 ++
>  kernel/bpf/syscall.c       |  19 +++
>  kernel/bpf/verifier.c      |  15 ++
>  net/core/filter.c          |   4 +-
>  net/socket.c               |  18 +++
>  11 files changed, 443 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> index b631ee75762d..406f1ba82531 100644
> --- a/include/linux/bpf-cgroup.h
> +++ b/include/linux/bpf-cgroup.h
> @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
>                                    loff_t *ppos, void **new_buf,
>                                    enum bpf_attach_type type);
>
> +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> +                                      int optname, char __user *optval,
> +                                      unsigned int optlen);
> +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> +                                      int optname, char __user *optval,
> +                                      int __user *optlen);
> +
>  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
>         struct bpf_map *map)
>  {
> @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
>         __ret;                                                                 \
>  })
>
> +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> +({                                                                            \
> +       int __ret = 0;                                                         \
> +       if (cgroup_bpf_enabled)                                                \
> +               __ret = __cgroup_bpf_run_filter_setsockopt(sock, level,        \
> +                                                          optname, optval,    \
> +                                                          optlen);            \
> +       __ret;                                                                 \
> +})
> +
> +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> +({                                                                            \
> +       int __ret = 0;                                                         \
> +       if (cgroup_bpf_enabled)                                                \
> +               __ret = __cgroup_bpf_run_filter_getsockopt(sock, level,        \
> +                                                          optname, optval,    \
> +                                                          optlen);            \
> +       __ret;                                                                 \
> +})
> +
>  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
>                            enum bpf_prog_type ptype, struct bpf_prog *prog);
>  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
>  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
>  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
>  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
>
>  #define for_each_cgroup_storage_type(stype) for (; false; )
>
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index e5a309e6a400..883a190bc0b8 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -520,6 +520,7 @@ struct bpf_prog_array {
>  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
>  void bpf_prog_array_free(struct bpf_prog_array *progs);
>  int bpf_prog_array_length(struct bpf_prog_array *progs);
> +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
>  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
>                                 __u32 __user *prog_ids, u32 cnt);
>
> @@ -606,6 +607,49 @@ _out:                                                      \
>                 _ret;                                   \
>         })
>
> +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> + *
> + * Expected BPF program return values are:
> + *   0: return -EPERM to the userspace
> + *   1: sockopt was not handled by BPF, kernel should do it
> + *   2: sockopt was handled by BPF, kernel not should do it and return

typo: should not do it?

> + *      to the userspace instead
> + *
> + * Note, that return '0' takes precedence over everything else. In other
> + * words, if any single program in the prog array has returned 0,
> + * the userspace will get -EPERM (regardless of what other programs
> + * return).
> + *
> + * The macro itself returns:
> + *        0: sockopt was not handled by BPF, kernel should do it
> + *        1: sockopt was handled by BPF, kernel snot hould do it

typo: "snot hould do it" -> "shouldn't do it"?

> + *   -EPERM: return error back to userspace
> + */
> +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)            \
> +       ({                                                              \
> +               struct bpf_prog_array_item *_item;                      \
> +               struct bpf_prog *_prog;                                 \
> +               struct bpf_prog_array *_array;                          \
> +               u32 ret;                                                \
> +               u32 _success = 1;                                       \
> +               u32 _bypass = 0;                                        \

reverse Christmas tree? or it's not enforced in a macro?

> +               preempt_disable();                                      \
> +               rcu_read_lock();                                        \
> +               _array = rcu_dereference(array);                        \
> +               _item = &_array->items[0];                              \
> +               while ((_prog = READ_ONCE(_item->prog))) {              \
> +                       bpf_cgroup_storage_set(_item->cgroup_storage);  \
> +                       ret = func(_prog, ctx);                         \
> +                       _success &= (ret > 0);                          \
> +                       _bypass |= (ret == 2);                          \
> +                       _item++;                                        \
> +               }                                                       \
> +               rcu_read_unlock();                                      \
> +               preempt_enable();                                       \
> +               ret = _success ? _bypass : -EPERM;                      \
> +               ret;                                                    \
> +       })
> +
>  #define BPF_PROG_RUN_ARRAY(array, ctx, func)           \
>         __BPF_PROG_RUN_ARRAY(array, ctx, func, false)
>
> @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
>  extern const struct bpf_func_proto bpf_get_local_storage_proto;
>  extern const struct bpf_func_proto bpf_strtol_proto;
>  extern const struct bpf_func_proto bpf_strtoul_proto;
> +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> +extern const struct bpf_func_proto bpf_tcp_sock_proto;
>
>  /* Shared helpers among cBPF and eBPF. */
>  void bpf_user_rnd_init_once(void);
> diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> index 5a9975678d6f..eec5aeeeaf92 100644
> --- a/include/linux/bpf_types.h
> +++ b/include/linux/bpf_types.h
> @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
>  #ifdef CONFIG_CGROUP_BPF
>  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
>  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
>  #endif
>  #ifdef CONFIG_BPF_LIRC_MODE2
>  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> diff --git a/include/linux/filter.h b/include/linux/filter.h
> index 43b45d6db36d..6e64d01e4e36 100644
> --- a/include/linux/filter.h
> +++ b/include/linux/filter.h
> @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
>         u64 tmp_reg;
>  };
>
> +struct bpf_sockopt_kern {
> +       struct sock     *sk;
> +       u8              *optval;
> +       u8              *optval_end;
> +       s32             level;
> +       s32             optname;
> +       u32             optlen;
> +
> +       /* Small on-stack optval buffer to avoid small allocations.
> +        */
> +       u8 buf[64] __aligned(8);
> +};
> +
>  #endif /* __LINUX_FILTER_H__ */
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 7c6aef253173..310b6bbfded8 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -170,6 +170,7 @@ enum bpf_prog_type {
>         BPF_PROG_TYPE_FLOW_DISSECTOR,
>         BPF_PROG_TYPE_CGROUP_SYSCTL,
>         BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> +       BPF_PROG_TYPE_CGROUP_SOCKOPT,
>  };
>
>  enum bpf_attach_type {
> @@ -192,6 +193,8 @@ enum bpf_attach_type {
>         BPF_LIRC_MODE2,
>         BPF_FLOW_DISSECTOR,
>         BPF_CGROUP_SYSCTL,
> +       BPF_CGROUP_GETSOCKOPT,
> +       BPF_CGROUP_SETSOCKOPT,
>         __MAX_BPF_ATTACH_TYPE
>  };
>
> @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
>                                  */
>  };
>
> +struct bpf_sockopt {
> +       __bpf_md_ptr(struct bpf_sock *, sk);
> +
> +       __s32   level;
> +       __s32   optname;
> +
> +       __u32   optlen;
> +       __u32   optval;
> +       __u32   optval_end;
> +};
> +
>  #endif /* _UAPI__LINUX_BPF_H__ */
> diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> index 1b65ab0df457..04bc1a09464e 100644
> --- a/kernel/bpf/cgroup.c
> +++ b/kernel/bpf/cgroup.c
> @@ -18,6 +18,7 @@
>  #include <linux/bpf.h>
>  #include <linux/bpf-cgroup.h>
>  #include <net/sock.h>
> +#include <net/bpf_sk_storage.h>
>
>  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
>  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
>  }
>  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
>
> +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> +                                            enum bpf_attach_type attach_type)
> +{
> +       struct bpf_prog_array *prog_array;
> +       bool empty;
> +
> +       rcu_read_lock();
> +       prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> +       empty = bpf_prog_array_is_empty(prog_array);
> +       rcu_read_unlock();
> +
> +       return empty;
> +}
> +
> +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> +{
> +       if (unlikely(max_optlen > PAGE_SIZE))
> +               return -EINVAL;
> +
> +       if (likely(max_optlen <= sizeof(ctx->buf))) {
> +               ctx->optval = ctx->buf;
> +       } else {
> +               ctx->optval = kzalloc(max_optlen, GFP_USER);
> +               if (!ctx->optval)
> +                       return -ENOMEM;
> +       }
> +
> +       ctx->optval_end = ctx->optval + max_optlen;
> +       ctx->optlen = max_optlen;
> +
> +       return 0;
> +}
> +
> +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> +{
> +       if (unlikely(ctx->optval != ctx->buf))
> +               kfree(ctx->optval);
> +}
> +
> +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> +                                      int optname, char __user *optval,
> +                                      unsigned int optlen)
> +{
> +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> +       struct bpf_sockopt_kern ctx = {
> +               .sk = sk,
> +               .level = level,
> +               .optname = optname,
> +       };
> +       int ret;
> +
> +       /* Opportunistic check to see whether we have any BPF program
> +        * attached to the hook so we don't waste time allocating
> +        * memory and locking the socket.
> +        */
> +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> +               return 0;
> +
> +       ret = sockopt_alloc_buf(&ctx, optlen);
> +       if (ret)
> +               return ret;
> +
> +       if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> +               sockopt_free_buf(&ctx);
> +               return -EFAULT;
> +       }
> +
> +       lock_sock(sk);
> +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> +               cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> +               &ctx, BPF_PROG_RUN);
> +       release_sock(sk);
> +
> +       sockopt_free_buf(&ctx);
> +
> +       return ret;
> +}
> +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> +
> +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> +                                      int optname, char __user *optval,
> +                                      int __user *optlen)
> +{
> +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> +       struct bpf_sockopt_kern ctx = {
> +               .sk = sk,
> +               .level = level,
> +               .optname = optname,
> +       };
> +       int max_optlen;
> +       int ret;
> +
> +       /* Opportunistic check to see whether we have any BPF program
> +        * attached to the hook so we don't waste time allocating
> +        * memory and locking the socket.
> +        */
> +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> +               return 0;
> +
> +       if (get_user(max_optlen, optlen))
> +               return -EFAULT;
> +
> +       ret = sockopt_alloc_buf(&ctx, max_optlen);
> +       if (ret)
> +               return ret;
> +
> +       lock_sock(sk);
> +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> +               cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> +               &ctx, BPF_PROG_RUN);
> +       release_sock(sk);
> +
> +       if (ret < 0) {
> +               sockopt_free_buf(&ctx);
> +               return ret;
> +       }
> +
> +       if (ctx.optlen > max_optlen) {
> +               sockopt_free_buf(&ctx);
> +               return -EFAULT;
> +       }

So this is the case where BPF program returns value that's bigger than
a buffer provided by users? Existing code in sock_getsockopt handles
that by filling out only first N bytes, instead of failing. Should the
behavior be the same here?

> +
> +       if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> +               sockopt_free_buf(&ctx);
> +               return -EFAULT;
> +       }
> +
> +       sockopt_free_buf(&ctx);
> +
> +       if (put_user(ctx.optlen, optlen))
> +               return -EFAULT;
> +
> +       return ret;
> +}
> +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> +
>  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
>                               size_t *lenp)
>  {
> @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
>
>  const struct bpf_prog_ops cg_sysctl_prog_ops = {
>  };
> +
> +static const struct bpf_func_proto *
> +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> +{
> +       switch (func_id) {
> +       case BPF_FUNC_sk_fullsock:
> +               return &bpf_sk_fullsock_proto;
> +       case BPF_FUNC_sk_storage_get:
> +               return &bpf_sk_storage_get_proto;
> +       case BPF_FUNC_sk_storage_delete:
> +               return &bpf_sk_storage_delete_proto;
> +#ifdef CONFIG_INET
> +       case BPF_FUNC_tcp_sock:
> +               return &bpf_tcp_sock_proto;
> +#endif
> +       default:
> +               return cgroup_base_func_proto(func_id, prog);
> +       }
> +}
> +
> +static bool cg_sockopt_is_valid_access(int off, int size,
> +                                      enum bpf_access_type type,
> +                                      const struct bpf_prog *prog,
> +                                      struct bpf_insn_access_aux *info)
> +{
> +       const int size_default = sizeof(__u32);
> +
> +       if (off < 0 || off >= sizeof(struct bpf_sockopt))
> +               return false;
> +
> +       if (off % size != 0)
> +               return false;
> +
> +       if (type == BPF_WRITE) {
> +               switch (off) {
> +               case offsetof(struct bpf_sockopt, optlen):
> +                       if (size != size_default)
> +                               return false;
> +                       return prog->expected_attach_type ==
> +                               BPF_CGROUP_GETSOCKOPT;
> +               default:
> +                       return false;
> +               }
> +       }
> +
> +       switch (off) {
> +       case offsetof(struct bpf_sockopt, sk):
> +               if (size != sizeof(__u64))
> +                       return false;
> +               info->reg_type = PTR_TO_SOCKET;
> +               break;
> +       case bpf_ctx_range(struct bpf_sockopt, optval):
> +               if (size != size_default)
> +                       return false;
> +               info->reg_type = PTR_TO_PACKET;
> +               break;
> +       case bpf_ctx_range(struct bpf_sockopt, optval_end):
> +               if (size != size_default)
> +                       return false;
> +               info->reg_type = PTR_TO_PACKET_END;
> +               break;
> +       default:
> +               if (size != size_default)
> +                       return false;
> +               break;
> +       }
> +       return true;
> +}
> +
> +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> +                                        const struct bpf_insn *si,
> +                                        struct bpf_insn *insn_buf,
> +                                        struct bpf_prog *prog,
> +                                        u32 *target_size)
> +{
> +       struct bpf_insn *insn = insn_buf;
> +
> +       switch (si->off) {
> +       case offsetof(struct bpf_sockopt, sk):
> +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> +                                                      bpf_sockopt_kern, sk),
> +                                     si->dst_reg, si->src_reg,
> +                                     offsetof(struct bpf_sockopt_kern, sk));
> +               break;
> +       case offsetof(struct bpf_sockopt, level):
> +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +                                     offsetof(struct bpf_sockopt_kern, level));
> +               break;
> +       case offsetof(struct bpf_sockopt, optname):
> +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +                                     offsetof(struct bpf_sockopt_kern,
> +                                              optname));
> +               break;
> +       case offsetof(struct bpf_sockopt, optlen):
> +               if (type == BPF_WRITE)
> +                       *insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +                                             offsetof(struct bpf_sockopt_kern,
> +                                                      optlen));
> +               else
> +                       *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +                                             offsetof(struct bpf_sockopt_kern,
> +                                                      optlen));
> +               break;
> +       case offsetof(struct bpf_sockopt, optval):
> +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> +                                                      optval),
> +                                     si->dst_reg, si->src_reg,
> +                                     offsetof(struct bpf_sockopt_kern,
> +                                              optval));
> +               break;
> +       case offsetof(struct bpf_sockopt, optval_end):
> +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> +                                                      optval_end),
> +                                     si->dst_reg, si->src_reg,
> +                                     offsetof(struct bpf_sockopt_kern,
> +                                              optval_end));
> +               break;
> +       }
> +
> +       return insn - insn_buf;
> +}
> +
> +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> +                                  bool direct_write,
> +                                  const struct bpf_prog *prog)
> +{
> +       /* Nothing to do for sockopt argument. The data is kzalloc'ated.
> +        */
> +       return 0;
> +}
> +
> +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> +       .get_func_proto         = cg_sockopt_func_proto,
> +       .is_valid_access        = cg_sockopt_is_valid_access,
> +       .convert_ctx_access     = cg_sockopt_convert_ctx_access,
> +       .gen_prologue           = cg_sockopt_get_prologue,
> +};
> +
> +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> +};
> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index 33fb292f2e30..e9152ebd66bc 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c
> @@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)
>         return cnt;
>  }
>
> +bool bpf_prog_array_is_empty(struct bpf_prog_array *array)
> +{
> +       struct bpf_prog_array_item *item;
> +
> +       for (item = array->items; item->prog; item++)
> +               if (item->prog != &dummy_bpf_prog.prog)
> +                       return false;
> +       return true;
> +}
>
>  static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
>                                      u32 *prog_ids,
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 4c53cbd3329d..4ad2b5f1905f 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,
>                 default:
>                         return -EINVAL;
>                 }
> +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> +               switch (expected_attach_type) {
> +               case BPF_CGROUP_SETSOCKOPT:
> +               case BPF_CGROUP_GETSOCKOPT:
> +                       return 0;
> +               default:
> +                       return -EINVAL;
> +               }
>         default:
>                 return 0;
>         }
> @@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
>         switch (prog->type) {
>         case BPF_PROG_TYPE_CGROUP_SOCK:
>         case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
>                 return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
>         case BPF_PROG_TYPE_CGROUP_SKB:
>                 return prog->enforce_expected_attach_type &&
> @@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)
>         case BPF_CGROUP_SYSCTL:
>                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
>                 break;
> +       case BPF_CGROUP_GETSOCKOPT:
> +       case BPF_CGROUP_SETSOCKOPT:
> +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> +               break;
>         default:
>                 return -EINVAL;
>         }
> @@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)
>         case BPF_CGROUP_SYSCTL:
>                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
>                 break;
> +       case BPF_CGROUP_GETSOCKOPT:
> +       case BPF_CGROUP_SETSOCKOPT:
> +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> +               break;
>         default:
>                 return -EINVAL;
>         }
> @@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,
>         case BPF_CGROUP_SOCK_OPS:
>         case BPF_CGROUP_DEVICE:
>         case BPF_CGROUP_SYSCTL:
> +       case BPF_CGROUP_GETSOCKOPT:
> +       case BPF_CGROUP_SETSOCKOPT:
>                 break;
>         case BPF_LIRC_MODE2:
>                 return lirc_prog_query(attr, uattr);
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 5c2cb5bd84ce..fffc668ef536 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env,
>
>                 env->seen_direct_write = true;
>                 return true;
> +
> +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> +               if (t == BPF_WRITE) {
> +                       if (env->prog->expected_attach_type ==
> +                           BPF_CGROUP_GETSOCKOPT) {
> +                               env->seen_direct_write = true;
> +                               return true;
> +                       }
> +                       return false;
> +               }
> +               return true;
> +
>         default:
>                 return false;
>         }
> @@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)
>         case BPF_PROG_TYPE_CGROUP_DEVICE:
>         case BPF_PROG_TYPE_CGROUP_SYSCTL:
>                 break;
> +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> +               range = tnum_range(0, 2);
> +               break;
>         default:
>                 return 0;
>         }
> diff --git a/net/core/filter.c b/net/core/filter.c
> index 55bfc941d17a..4652c0a005ca 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
>         return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
>  }
>
> -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> +const struct bpf_func_proto bpf_sk_fullsock_proto = {
>         .func           = bpf_sk_fullsock,
>         .gpl_only       = false,
>         .ret_type       = RET_PTR_TO_SOCKET_OR_NULL,
> @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
>         return (unsigned long)NULL;
>  }
>
> -static const struct bpf_func_proto bpf_tcp_sock_proto = {
> +const struct bpf_func_proto bpf_tcp_sock_proto = {
>         .func           = bpf_tcp_sock,
>         .gpl_only       = false,
>         .ret_type       = RET_PTR_TO_TCP_SOCK_OR_NULL,
> diff --git a/net/socket.c b/net/socket.c
> index 72372dc5dd70..e8654f1f70e6 100644
> --- a/net/socket.c
> +++ b/net/socket.c
> @@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,
>                 if (err)
>                         goto out_put;
>
> +               err = BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock->sk, level, optname,
> +                                                    optval, optlen);
> +               if (err < 0) {
> +                       goto out_put;
> +               } else if (err > 0) {
> +                       err = 0;
> +                       goto out_put;
> +               }
> +
>                 if (level == SOL_SOCKET)
>                         err =
>                             sock_setsockopt(sock, level, optname, optval,
> @@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,
>                 if (err)
>                         goto out_put;
>
> +               err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname,
> +                                                    optval, optlen);
> +               if (err < 0) {
> +                       goto out_put;
> +               } else if (err > 0) {
> +                       err = 0;
> +                       goto out_put;
> +               }
> +
>                 if (level == SOL_SOCKET)
>                         err =
>                             sock_getsockopt(sock, level, optname, optval,
> --
> 2.22.0.rc1.311.g5d7573a151-goog
>

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Stanislav Fomichev]

On 06/06, Andrii Nakryiko wrote:
> On Thu, Jun 6, 2019 at 12:03 PM Stanislav Fomichev <sdf@google.com> wrote:
> >
> > Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> > BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
> >
> > BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> > BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> > Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
> >
> > The buffer memory is pre-allocated (because I don't think there is
> > a precedent for working with __user memory from bpf). This might be
> > slow to do for each {s,g}etsockopt call, that's why I've added
> > __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> > attached to a cgroup. Note, however, that there is a race between
> > __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> > program layout might have changed; this should not be a problem
> > because in general there is a race between multiple calls to
> > {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
> >
> > The return code of the BPF program is handled as follows:
> > * 0: EPERM
> > * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> > * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
> >      prog exits
> >
> > v2:
> > * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> > * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> > * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> > * added [0,2] return code check to verifier (Martin Lau)
> > * dropped unused buf[64] from the stack (Martin Lau)
> > * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> > * dropped bpf_target_off from ctx rewrites (Martin Lau)
> > * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
> >
> > Signed-off-by: Stanislav Fomichev <sdf@google.com>
> > ---
> >  include/linux/bpf-cgroup.h |  29 ++++
> >  include/linux/bpf.h        |  46 ++++++
> >  include/linux/bpf_types.h  |   1 +
> >  include/linux/filter.h     |  13 ++
> >  include/uapi/linux/bpf.h   |  14 ++
> >  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
> >  kernel/bpf/core.c          |   9 ++
> >  kernel/bpf/syscall.c       |  19 +++
> >  kernel/bpf/verifier.c      |  15 ++
> >  net/core/filter.c          |   4 +-
> >  net/socket.c               |  18 +++
> >  11 files changed, 443 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> > index b631ee75762d..406f1ba82531 100644
> > --- a/include/linux/bpf-cgroup.h
> > +++ b/include/linux/bpf-cgroup.h
> > @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> >                                    loff_t *ppos, void **new_buf,
> >                                    enum bpf_attach_type type);
> >
> > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> > +                                      int optname, char __user *optval,
> > +                                      unsigned int optlen);
> > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> > +                                      int optname, char __user *optval,
> > +                                      int __user *optlen);
> > +
> >  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
> >         struct bpf_map *map)
> >  {
> > @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
> >         __ret;                                                                 \
> >  })
> >
> > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> > +({                                                                            \
> > +       int __ret = 0;                                                         \
> > +       if (cgroup_bpf_enabled)                                                \
> > +               __ret = __cgroup_bpf_run_filter_setsockopt(sock, level,        \
> > +                                                          optname, optval,    \
> > +                                                          optlen);            \
> > +       __ret;                                                                 \
> > +})
> > +
> > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> > +({                                                                            \
> > +       int __ret = 0;                                                         \
> > +       if (cgroup_bpf_enabled)                                                \
> > +               __ret = __cgroup_bpf_run_filter_getsockopt(sock, level,        \
> > +                                                          optname, optval,    \
> > +                                                          optlen);            \
> > +       __ret;                                                                 \
> > +})
> > +
> >  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> >                            enum bpf_prog_type ptype, struct bpf_prog *prog);
> >  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> > @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
> >  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
> >  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
> >  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> >
> >  #define for_each_cgroup_storage_type(stype) for (; false; )
> >
> > diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> > index e5a309e6a400..883a190bc0b8 100644
> > --- a/include/linux/bpf.h
> > +++ b/include/linux/bpf.h
> > @@ -520,6 +520,7 @@ struct bpf_prog_array {
> >  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
> >  void bpf_prog_array_free(struct bpf_prog_array *progs);
> >  int bpf_prog_array_length(struct bpf_prog_array *progs);
> > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
> >  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
> >                                 __u32 __user *prog_ids, u32 cnt);
> >
> > @@ -606,6 +607,49 @@ _out:                                                      \
> >                 _ret;                                   \
> >         })
> >
> > +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> > + *
> > + * Expected BPF program return values are:
> > + *   0: return -EPERM to the userspace
> > + *   1: sockopt was not handled by BPF, kernel should do it
> > + *   2: sockopt was handled by BPF, kernel not should do it and return
> 
> typo: should not do it?
Sigh, good catch, thanks!

> > + *      to the userspace instead
> > + *
> > + * Note, that return '0' takes precedence over everything else. In other
> > + * words, if any single program in the prog array has returned 0,
> > + * the userspace will get -EPERM (regardless of what other programs
> > + * return).
> > + *
> > + * The macro itself returns:
> > + *        0: sockopt was not handled by BPF, kernel should do it
> > + *        1: sockopt was handled by BPF, kernel snot hould do it
> 
> typo: "snot hould do it" -> "shouldn't do it"?
Ditto.

> > + *   -EPERM: return error back to userspace
> > + */
> > +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)            \
> > +       ({                                                              \
> > +               struct bpf_prog_array_item *_item;                      \
> > +               struct bpf_prog *_prog;                                 \
> > +               struct bpf_prog_array *_array;                          \
> > +               u32 ret;                                                \
> > +               u32 _success = 1;                                       \
> > +               u32 _bypass = 0;                                        \
> 
> reverse Christmas tree? or it's not enforced in a macro?
I don't think we do that for other BPF_PROG_XXX_RUN_ARRAY
handlers. But sure, why not? :-)

> > +               preempt_disable();                                      \
> > +               rcu_read_lock();                                        \
> > +               _array = rcu_dereference(array);                        \
> > +               _item = &_array->items[0];                              \
> > +               while ((_prog = READ_ONCE(_item->prog))) {              \
> > +                       bpf_cgroup_storage_set(_item->cgroup_storage);  \
> > +                       ret = func(_prog, ctx);                         \
> > +                       _success &= (ret > 0);                          \
> > +                       _bypass |= (ret == 2);                          \
> > +                       _item++;                                        \
> > +               }                                                       \
> > +               rcu_read_unlock();                                      \
> > +               preempt_enable();                                       \
> > +               ret = _success ? _bypass : -EPERM;                      \
> > +               ret;                                                    \
> > +       })
> > +
> >  #define BPF_PROG_RUN_ARRAY(array, ctx, func)           \
> >         __BPF_PROG_RUN_ARRAY(array, ctx, func, false)
> >
> > @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
> >  extern const struct bpf_func_proto bpf_get_local_storage_proto;
> >  extern const struct bpf_func_proto bpf_strtol_proto;
> >  extern const struct bpf_func_proto bpf_strtoul_proto;
> > +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> > +extern const struct bpf_func_proto bpf_tcp_sock_proto;
> >
> >  /* Shared helpers among cBPF and eBPF. */
> >  void bpf_user_rnd_init_once(void);
> > diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> > index 5a9975678d6f..eec5aeeeaf92 100644
> > --- a/include/linux/bpf_types.h
> > +++ b/include/linux/bpf_types.h
> > @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
> >  #ifdef CONFIG_CGROUP_BPF
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> > +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
> >  #endif
> >  #ifdef CONFIG_BPF_LIRC_MODE2
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> > diff --git a/include/linux/filter.h b/include/linux/filter.h
> > index 43b45d6db36d..6e64d01e4e36 100644
> > --- a/include/linux/filter.h
> > +++ b/include/linux/filter.h
> > @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
> >         u64 tmp_reg;
> >  };
> >
> > +struct bpf_sockopt_kern {
> > +       struct sock     *sk;
> > +       u8              *optval;
> > +       u8              *optval_end;
> > +       s32             level;
> > +       s32             optname;
> > +       u32             optlen;
> > +
> > +       /* Small on-stack optval buffer to avoid small allocations.
> > +        */
> > +       u8 buf[64] __aligned(8);
> > +};
> > +
> >  #endif /* __LINUX_FILTER_H__ */
> > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > index 7c6aef253173..310b6bbfded8 100644
> > --- a/include/uapi/linux/bpf.h
> > +++ b/include/uapi/linux/bpf.h
> > @@ -170,6 +170,7 @@ enum bpf_prog_type {
> >         BPF_PROG_TYPE_FLOW_DISSECTOR,
> >         BPF_PROG_TYPE_CGROUP_SYSCTL,
> >         BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> > +       BPF_PROG_TYPE_CGROUP_SOCKOPT,
> >  };
> >
> >  enum bpf_attach_type {
> > @@ -192,6 +193,8 @@ enum bpf_attach_type {
> >         BPF_LIRC_MODE2,
> >         BPF_FLOW_DISSECTOR,
> >         BPF_CGROUP_SYSCTL,
> > +       BPF_CGROUP_GETSOCKOPT,
> > +       BPF_CGROUP_SETSOCKOPT,
> >         __MAX_BPF_ATTACH_TYPE
> >  };
> >
> > @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
> >                                  */
> >  };
> >
> > +struct bpf_sockopt {
> > +       __bpf_md_ptr(struct bpf_sock *, sk);
> > +
> > +       __s32   level;
> > +       __s32   optname;
> > +
> > +       __u32   optlen;
> > +       __u32   optval;
> > +       __u32   optval_end;
> > +};
> > +
> >  #endif /* _UAPI__LINUX_BPF_H__ */
> > diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> > index 1b65ab0df457..04bc1a09464e 100644
> > --- a/kernel/bpf/cgroup.c
> > +++ b/kernel/bpf/cgroup.c
> > @@ -18,6 +18,7 @@
> >  #include <linux/bpf.h>
> >  #include <linux/bpf-cgroup.h>
> >  #include <net/sock.h>
> > +#include <net/bpf_sk_storage.h>
> >
> >  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
> >  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> > @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> >  }
> >  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
> >
> > +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> > +                                            enum bpf_attach_type attach_type)
> > +{
> > +       struct bpf_prog_array *prog_array;
> > +       bool empty;
> > +
> > +       rcu_read_lock();
> > +       prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> > +       empty = bpf_prog_array_is_empty(prog_array);
> > +       rcu_read_unlock();
> > +
> > +       return empty;
> > +}
> > +
> > +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> > +{
> > +       if (unlikely(max_optlen > PAGE_SIZE))
> > +               return -EINVAL;
> > +
> > +       if (likely(max_optlen <= sizeof(ctx->buf))) {
> > +               ctx->optval = ctx->buf;
> > +       } else {
> > +               ctx->optval = kzalloc(max_optlen, GFP_USER);
> > +               if (!ctx->optval)
> > +                       return -ENOMEM;
> > +       }
> > +
> > +       ctx->optval_end = ctx->optval + max_optlen;
> > +       ctx->optlen = max_optlen;
> > +
> > +       return 0;
> > +}
> > +
> > +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> > +{
> > +       if (unlikely(ctx->optval != ctx->buf))
> > +               kfree(ctx->optval);
> > +}
> > +
> > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> > +                                      int optname, char __user *optval,
> > +                                      unsigned int optlen)
> > +{
> > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > +       struct bpf_sockopt_kern ctx = {
> > +               .sk = sk,
> > +               .level = level,
> > +               .optname = optname,
> > +       };
> > +       int ret;
> > +
> > +       /* Opportunistic check to see whether we have any BPF program
> > +        * attached to the hook so we don't waste time allocating
> > +        * memory and locking the socket.
> > +        */
> > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> > +               return 0;
> > +
> > +       ret = sockopt_alloc_buf(&ctx, optlen);
> > +       if (ret)
> > +               return ret;
> > +
> > +       if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> > +               sockopt_free_buf(&ctx);
> > +               return -EFAULT;
> > +       }
> > +
> > +       lock_sock(sk);
> > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > +               cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> > +               &ctx, BPF_PROG_RUN);
> > +       release_sock(sk);
> > +
> > +       sockopt_free_buf(&ctx);
> > +
> > +       return ret;
> > +}
> > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> > +
> > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> > +                                      int optname, char __user *optval,
> > +                                      int __user *optlen)
> > +{
> > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > +       struct bpf_sockopt_kern ctx = {
> > +               .sk = sk,
> > +               .level = level,
> > +               .optname = optname,
> > +       };
> > +       int max_optlen;
> > +       int ret;
> > +
> > +       /* Opportunistic check to see whether we have any BPF program
> > +        * attached to the hook so we don't waste time allocating
> > +        * memory and locking the socket.
> > +        */
> > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> > +               return 0;
> > +
> > +       if (get_user(max_optlen, optlen))
> > +               return -EFAULT;
> > +
> > +       ret = sockopt_alloc_buf(&ctx, max_optlen);
> > +       if (ret)
> > +               return ret;
> > +
> > +       lock_sock(sk);
> > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > +               cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> > +               &ctx, BPF_PROG_RUN);
> > +       release_sock(sk);
> > +
> > +       if (ret < 0) {
> > +               sockopt_free_buf(&ctx);
> > +               return ret;
> > +       }
> > +
> > +       if (ctx.optlen > max_optlen) {
> > +               sockopt_free_buf(&ctx);
> > +               return -EFAULT;
> > +       }
> 
> So this is the case where BPF program returns value that's bigger than
> a buffer provided by users? Existing code in sock_getsockopt handles
> that by filling out only first N bytes, instead of failing. Should the
> behavior be the same here?
I'm not sure. BPF program would need to handle this case anyway, right?
Verifier would not let it put more bytes in the buffer than available,
so increasing optlen in the BPF program doesn't really make sense.
It feels like returning a EPERM (and assuming BPF program is buggy)
is safer. Am I missing something?

> > +       if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> > +               sockopt_free_buf(&ctx);
> > +               return -EFAULT;
> > +       }
> > +
> > +       sockopt_free_buf(&ctx);
> > +
> > +       if (put_user(ctx.optlen, optlen))
> > +               return -EFAULT;
> > +
> > +       return ret;
> > +}
> > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> > +
> >  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
> >                               size_t *lenp)
> >  {
> > @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
> >
> >  const struct bpf_prog_ops cg_sysctl_prog_ops = {
> >  };
> > +
> > +static const struct bpf_func_proto *
> > +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> > +{
> > +       switch (func_id) {
> > +       case BPF_FUNC_sk_fullsock:
> > +               return &bpf_sk_fullsock_proto;
> > +       case BPF_FUNC_sk_storage_get:
> > +               return &bpf_sk_storage_get_proto;
> > +       case BPF_FUNC_sk_storage_delete:
> > +               return &bpf_sk_storage_delete_proto;
> > +#ifdef CONFIG_INET
> > +       case BPF_FUNC_tcp_sock:
> > +               return &bpf_tcp_sock_proto;
> > +#endif
> > +       default:
> > +               return cgroup_base_func_proto(func_id, prog);
> > +       }
> > +}
> > +
> > +static bool cg_sockopt_is_valid_access(int off, int size,
> > +                                      enum bpf_access_type type,
> > +                                      const struct bpf_prog *prog,
> > +                                      struct bpf_insn_access_aux *info)
> > +{
> > +       const int size_default = sizeof(__u32);
> > +
> > +       if (off < 0 || off >= sizeof(struct bpf_sockopt))
> > +               return false;
> > +
> > +       if (off % size != 0)
> > +               return false;
> > +
> > +       if (type == BPF_WRITE) {
> > +               switch (off) {
> > +               case offsetof(struct bpf_sockopt, optlen):
> > +                       if (size != size_default)
> > +                               return false;
> > +                       return prog->expected_attach_type ==
> > +                               BPF_CGROUP_GETSOCKOPT;
> > +               default:
> > +                       return false;
> > +               }
> > +       }
> > +
> > +       switch (off) {
> > +       case offsetof(struct bpf_sockopt, sk):
> > +               if (size != sizeof(__u64))
> > +                       return false;
> > +               info->reg_type = PTR_TO_SOCKET;
> > +               break;
> > +       case bpf_ctx_range(struct bpf_sockopt, optval):
> > +               if (size != size_default)
> > +                       return false;
> > +               info->reg_type = PTR_TO_PACKET;
> > +               break;
> > +       case bpf_ctx_range(struct bpf_sockopt, optval_end):
> > +               if (size != size_default)
> > +                       return false;
> > +               info->reg_type = PTR_TO_PACKET_END;
> > +               break;
> > +       default:
> > +               if (size != size_default)
> > +                       return false;
> > +               break;
> > +       }
> > +       return true;
> > +}
> > +
> > +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> > +                                        const struct bpf_insn *si,
> > +                                        struct bpf_insn *insn_buf,
> > +                                        struct bpf_prog *prog,
> > +                                        u32 *target_size)
> > +{
> > +       struct bpf_insn *insn = insn_buf;
> > +
> > +       switch (si->off) {
> > +       case offsetof(struct bpf_sockopt, sk):
> > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> > +                                                      bpf_sockopt_kern, sk),
> > +                                     si->dst_reg, si->src_reg,
> > +                                     offsetof(struct bpf_sockopt_kern, sk));
> > +               break;
> > +       case offsetof(struct bpf_sockopt, level):
> > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +                                     offsetof(struct bpf_sockopt_kern, level));
> > +               break;
> > +       case offsetof(struct bpf_sockopt, optname):
> > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +                                     offsetof(struct bpf_sockopt_kern,
> > +                                              optname));
> > +               break;
> > +       case offsetof(struct bpf_sockopt, optlen):
> > +               if (type == BPF_WRITE)
> > +                       *insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +                                             offsetof(struct bpf_sockopt_kern,
> > +                                                      optlen));
> > +               else
> > +                       *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +                                             offsetof(struct bpf_sockopt_kern,
> > +                                                      optlen));
> > +               break;
> > +       case offsetof(struct bpf_sockopt, optval):
> > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > +                                                      optval),
> > +                                     si->dst_reg, si->src_reg,
> > +                                     offsetof(struct bpf_sockopt_kern,
> > +                                              optval));
> > +               break;
> > +       case offsetof(struct bpf_sockopt, optval_end):
> > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > +                                                      optval_end),
> > +                                     si->dst_reg, si->src_reg,
> > +                                     offsetof(struct bpf_sockopt_kern,
> > +                                              optval_end));
> > +               break;
> > +       }
> > +
> > +       return insn - insn_buf;
> > +}
> > +
> > +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> > +                                  bool direct_write,
> > +                                  const struct bpf_prog *prog)
> > +{
> > +       /* Nothing to do for sockopt argument. The data is kzalloc'ated.
> > +        */
> > +       return 0;
> > +}
> > +
> > +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> > +       .get_func_proto         = cg_sockopt_func_proto,
> > +       .is_valid_access        = cg_sockopt_is_valid_access,
> > +       .convert_ctx_access     = cg_sockopt_convert_ctx_access,
> > +       .gen_prologue           = cg_sockopt_get_prologue,
> > +};
> > +
> > +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> > +};
> > diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> > index 33fb292f2e30..e9152ebd66bc 100644
> > --- a/kernel/bpf/core.c
> > +++ b/kernel/bpf/core.c
> > @@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)
> >         return cnt;
> >  }
> >
> > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array)
> > +{
> > +       struct bpf_prog_array_item *item;
> > +
> > +       for (item = array->items; item->prog; item++)
> > +               if (item->prog != &dummy_bpf_prog.prog)
> > +                       return false;
> > +       return true;
> > +}
> >
> >  static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
> >                                      u32 *prog_ids,
> > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> > index 4c53cbd3329d..4ad2b5f1905f 100644
> > --- a/kernel/bpf/syscall.c
> > +++ b/kernel/bpf/syscall.c
> > @@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,
> >                 default:
> >                         return -EINVAL;
> >                 }
> > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > +               switch (expected_attach_type) {
> > +               case BPF_CGROUP_SETSOCKOPT:
> > +               case BPF_CGROUP_GETSOCKOPT:
> > +                       return 0;
> > +               default:
> > +                       return -EINVAL;
> > +               }
> >         default:
> >                 return 0;
> >         }
> > @@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> >         switch (prog->type) {
> >         case BPF_PROG_TYPE_CGROUP_SOCK:
> >         case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> >                 return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
> >         case BPF_PROG_TYPE_CGROUP_SKB:
> >                 return prog->enforce_expected_attach_type &&
> > @@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)
> >         case BPF_CGROUP_SYSCTL:
> >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> >                 break;
> > +       case BPF_CGROUP_GETSOCKOPT:
> > +       case BPF_CGROUP_SETSOCKOPT:
> > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > +               break;
> >         default:
> >                 return -EINVAL;
> >         }
> > @@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)
> >         case BPF_CGROUP_SYSCTL:
> >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> >                 break;
> > +       case BPF_CGROUP_GETSOCKOPT:
> > +       case BPF_CGROUP_SETSOCKOPT:
> > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > +               break;
> >         default:
> >                 return -EINVAL;
> >         }
> > @@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,
> >         case BPF_CGROUP_SOCK_OPS:
> >         case BPF_CGROUP_DEVICE:
> >         case BPF_CGROUP_SYSCTL:
> > +       case BPF_CGROUP_GETSOCKOPT:
> > +       case BPF_CGROUP_SETSOCKOPT:
> >                 break;
> >         case BPF_LIRC_MODE2:
> >                 return lirc_prog_query(attr, uattr);
> > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > index 5c2cb5bd84ce..fffc668ef536 100644
> > --- a/kernel/bpf/verifier.c
> > +++ b/kernel/bpf/verifier.c
> > @@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env,
> >
> >                 env->seen_direct_write = true;
> >                 return true;
> > +
> > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > +               if (t == BPF_WRITE) {
> > +                       if (env->prog->expected_attach_type ==
> > +                           BPF_CGROUP_GETSOCKOPT) {
> > +                               env->seen_direct_write = true;
> > +                               return true;
> > +                       }
> > +                       return false;
> > +               }
> > +               return true;
> > +
> >         default:
> >                 return false;
> >         }
> > @@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)
> >         case BPF_PROG_TYPE_CGROUP_DEVICE:
> >         case BPF_PROG_TYPE_CGROUP_SYSCTL:
> >                 break;
> > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > +               range = tnum_range(0, 2);
> > +               break;
> >         default:
> >                 return 0;
> >         }
> > diff --git a/net/core/filter.c b/net/core/filter.c
> > index 55bfc941d17a..4652c0a005ca 100644
> > --- a/net/core/filter.c
> > +++ b/net/core/filter.c
> > @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
> >         return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
> >  }
> >
> > -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > +const struct bpf_func_proto bpf_sk_fullsock_proto = {
> >         .func           = bpf_sk_fullsock,
> >         .gpl_only       = false,
> >         .ret_type       = RET_PTR_TO_SOCKET_OR_NULL,
> > @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
> >         return (unsigned long)NULL;
> >  }
> >
> > -static const struct bpf_func_proto bpf_tcp_sock_proto = {
> > +const struct bpf_func_proto bpf_tcp_sock_proto = {
> >         .func           = bpf_tcp_sock,
> >         .gpl_only       = false,
> >         .ret_type       = RET_PTR_TO_TCP_SOCK_OR_NULL,
> > diff --git a/net/socket.c b/net/socket.c
> > index 72372dc5dd70..e8654f1f70e6 100644
> > --- a/net/socket.c
> > +++ b/net/socket.c
> > @@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,
> >                 if (err)
> >                         goto out_put;
> >
> > +               err = BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock->sk, level, optname,
> > +                                                    optval, optlen);
> > +               if (err < 0) {
> > +                       goto out_put;
> > +               } else if (err > 0) {
> > +                       err = 0;
> > +                       goto out_put;
> > +               }
> > +
> >                 if (level == SOL_SOCKET)
> >                         err =
> >                             sock_setsockopt(sock, level, optname, optval,
> > @@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,
> >                 if (err)
> >                         goto out_put;
> >
> > +               err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname,
> > +                                                    optval, optlen);
> > +               if (err < 0) {
> > +                       goto out_put;
> > +               } else if (err > 0) {
> > +                       err = 0;
> > +                       goto out_put;
> > +               }
> > +
> >                 if (level == SOL_SOCKET)
> >                         err =
> >                             sock_getsockopt(sock, level, optname, optval,
> > --
> > 2.22.0.rc1.311.g5d7573a151-goog
> >

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Andrii Nakryiko]

On Thu, Jun 6, 2019 at 3:27 PM Stanislav Fomichev <sdf@fomichev.me> wrote:
>
> On 06/06, Andrii Nakryiko wrote:
> > On Thu, Jun 6, 2019 at 12:03 PM Stanislav Fomichev <sdf@google.com> wrote:
> > >
> > > Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> > > BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
> > >
> > > BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> > > BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> > > Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
> > >
> > > The buffer memory is pre-allocated (because I don't think there is
> > > a precedent for working with __user memory from bpf). This might be
> > > slow to do for each {s,g}etsockopt call, that's why I've added
> > > __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> > > attached to a cgroup. Note, however, that there is a race between
> > > __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> > > program layout might have changed; this should not be a problem
> > > because in general there is a race between multiple calls to
> > > {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
> > >
> > > The return code of the BPF program is handled as follows:
> > > * 0: EPERM
> > > * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> > > * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
> > >      prog exits
> > >
> > > v2:
> > > * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> > > * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> > > * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> > > * added [0,2] return code check to verifier (Martin Lau)
> > > * dropped unused buf[64] from the stack (Martin Lau)
> > > * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> > > * dropped bpf_target_off from ctx rewrites (Martin Lau)
> > > * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
> > >
> > > Signed-off-by: Stanislav Fomichev <sdf@google.com>
> > > ---
> > >  include/linux/bpf-cgroup.h |  29 ++++
> > >  include/linux/bpf.h        |  46 ++++++
> > >  include/linux/bpf_types.h  |   1 +
> > >  include/linux/filter.h     |  13 ++
> > >  include/uapi/linux/bpf.h   |  14 ++
> > >  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
> > >  kernel/bpf/core.c          |   9 ++
> > >  kernel/bpf/syscall.c       |  19 +++
> > >  kernel/bpf/verifier.c      |  15 ++
> > >  net/core/filter.c          |   4 +-
> > >  net/socket.c               |  18 +++
> > >  11 files changed, 443 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> > > index b631ee75762d..406f1ba82531 100644
> > > --- a/include/linux/bpf-cgroup.h
> > > +++ b/include/linux/bpf-cgroup.h
> > > @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> > >                                    loff_t *ppos, void **new_buf,
> > >                                    enum bpf_attach_type type);
> > >
> > > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> > > +                                      int optname, char __user *optval,
> > > +                                      unsigned int optlen);
> > > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> > > +                                      int optname, char __user *optval,
> > > +                                      int __user *optlen);
> > > +
> > >  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
> > >         struct bpf_map *map)
> > >  {
> > > @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
> > >         __ret;                                                                 \
> > >  })
> > >
> > > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> > > +({                                                                            \
> > > +       int __ret = 0;                                                         \
> > > +       if (cgroup_bpf_enabled)                                                \
> > > +               __ret = __cgroup_bpf_run_filter_setsockopt(sock, level,        \
> > > +                                                          optname, optval,    \
> > > +                                                          optlen);            \
> > > +       __ret;                                                                 \
> > > +})
> > > +
> > > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> > > +({                                                                            \
> > > +       int __ret = 0;                                                         \
> > > +       if (cgroup_bpf_enabled)                                                \
> > > +               __ret = __cgroup_bpf_run_filter_getsockopt(sock, level,        \
> > > +                                                          optname, optval,    \
> > > +                                                          optlen);            \
> > > +       __ret;                                                                 \
> > > +})
> > > +
> > >  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> > >                            enum bpf_prog_type ptype, struct bpf_prog *prog);
> > >  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> > > @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
> > >  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
> > >  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
> > >  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> > > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > >
> > >  #define for_each_cgroup_storage_type(stype) for (; false; )
> > >
> > > diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> > > index e5a309e6a400..883a190bc0b8 100644
> > > --- a/include/linux/bpf.h
> > > +++ b/include/linux/bpf.h
> > > @@ -520,6 +520,7 @@ struct bpf_prog_array {
> > >  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
> > >  void bpf_prog_array_free(struct bpf_prog_array *progs);
> > >  int bpf_prog_array_length(struct bpf_prog_array *progs);
> > > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
> > >  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
> > >                                 __u32 __user *prog_ids, u32 cnt);
> > >
> > > @@ -606,6 +607,49 @@ _out:                                                      \
> > >                 _ret;                                   \
> > >         })
> > >
> > > +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> > > + *
> > > + * Expected BPF program return values are:
> > > + *   0: return -EPERM to the userspace
> > > + *   1: sockopt was not handled by BPF, kernel should do it
> > > + *   2: sockopt was handled by BPF, kernel not should do it and return
> >
> > typo: should not do it?
> Sigh, good catch, thanks!
>
> > > + *      to the userspace instead
> > > + *
> > > + * Note, that return '0' takes precedence over everything else. In other
> > > + * words, if any single program in the prog array has returned 0,
> > > + * the userspace will get -EPERM (regardless of what other programs
> > > + * return).
> > > + *
> > > + * The macro itself returns:
> > > + *        0: sockopt was not handled by BPF, kernel should do it
> > > + *        1: sockopt was handled by BPF, kernel snot hould do it
> >
> > typo: "snot hould do it" -> "shouldn't do it"?
> Ditto.
>
> > > + *   -EPERM: return error back to userspace
> > > + */
> > > +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)            \
> > > +       ({                                                              \
> > > +               struct bpf_prog_array_item *_item;                      \
> > > +               struct bpf_prog *_prog;                                 \
> > > +               struct bpf_prog_array *_array;                          \
> > > +               u32 ret;                                                \
> > > +               u32 _success = 1;                                       \
> > > +               u32 _bypass = 0;                                        \
> >
> > reverse Christmas tree? or it's not enforced in a macro?
> I don't think we do that for other BPF_PROG_XXX_RUN_ARRAY
> handlers. But sure, why not? :-)
>
> > > +               preempt_disable();                                      \
> > > +               rcu_read_lock();                                        \
> > > +               _array = rcu_dereference(array);                        \
> > > +               _item = &_array->items[0];                              \
> > > +               while ((_prog = READ_ONCE(_item->prog))) {              \
> > > +                       bpf_cgroup_storage_set(_item->cgroup_storage);  \
> > > +                       ret = func(_prog, ctx);                         \
> > > +                       _success &= (ret > 0);                          \
> > > +                       _bypass |= (ret == 2);                          \
> > > +                       _item++;                                        \
> > > +               }                                                       \
> > > +               rcu_read_unlock();                                      \
> > > +               preempt_enable();                                       \
> > > +               ret = _success ? _bypass : -EPERM;                      \
> > > +               ret;                                                    \
> > > +       })
> > > +
> > >  #define BPF_PROG_RUN_ARRAY(array, ctx, func)           \
> > >         __BPF_PROG_RUN_ARRAY(array, ctx, func, false)
> > >
> > > @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
> > >  extern const struct bpf_func_proto bpf_get_local_storage_proto;
> > >  extern const struct bpf_func_proto bpf_strtol_proto;
> > >  extern const struct bpf_func_proto bpf_strtoul_proto;
> > > +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> > > +extern const struct bpf_func_proto bpf_tcp_sock_proto;
> > >
> > >  /* Shared helpers among cBPF and eBPF. */
> > >  void bpf_user_rnd_init_once(void);
> > > diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> > > index 5a9975678d6f..eec5aeeeaf92 100644
> > > --- a/include/linux/bpf_types.h
> > > +++ b/include/linux/bpf_types.h
> > > @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
> > >  #ifdef CONFIG_CGROUP_BPF
> > >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
> > >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> > > +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
> > >  #endif
> > >  #ifdef CONFIG_BPF_LIRC_MODE2
> > >  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> > > diff --git a/include/linux/filter.h b/include/linux/filter.h
> > > index 43b45d6db36d..6e64d01e4e36 100644
> > > --- a/include/linux/filter.h
> > > +++ b/include/linux/filter.h
> > > @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
> > >         u64 tmp_reg;
> > >  };
> > >
> > > +struct bpf_sockopt_kern {
> > > +       struct sock     *sk;
> > > +       u8              *optval;
> > > +       u8              *optval_end;
> > > +       s32             level;
> > > +       s32             optname;
> > > +       u32             optlen;
> > > +
> > > +       /* Small on-stack optval buffer to avoid small allocations.
> > > +        */
> > > +       u8 buf[64] __aligned(8);
> > > +};
> > > +
> > >  #endif /* __LINUX_FILTER_H__ */
> > > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > > index 7c6aef253173..310b6bbfded8 100644
> > > --- a/include/uapi/linux/bpf.h
> > > +++ b/include/uapi/linux/bpf.h
> > > @@ -170,6 +170,7 @@ enum bpf_prog_type {
> > >         BPF_PROG_TYPE_FLOW_DISSECTOR,
> > >         BPF_PROG_TYPE_CGROUP_SYSCTL,
> > >         BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> > > +       BPF_PROG_TYPE_CGROUP_SOCKOPT,
> > >  };
> > >
> > >  enum bpf_attach_type {
> > > @@ -192,6 +193,8 @@ enum bpf_attach_type {
> > >         BPF_LIRC_MODE2,
> > >         BPF_FLOW_DISSECTOR,
> > >         BPF_CGROUP_SYSCTL,
> > > +       BPF_CGROUP_GETSOCKOPT,
> > > +       BPF_CGROUP_SETSOCKOPT,
> > >         __MAX_BPF_ATTACH_TYPE
> > >  };
> > >
> > > @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
> > >                                  */
> > >  };
> > >
> > > +struct bpf_sockopt {
> > > +       __bpf_md_ptr(struct bpf_sock *, sk);
> > > +
> > > +       __s32   level;
> > > +       __s32   optname;
> > > +
> > > +       __u32   optlen;
> > > +       __u32   optval;
> > > +       __u32   optval_end;
> > > +};
> > > +
> > >  #endif /* _UAPI__LINUX_BPF_H__ */
> > > diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> > > index 1b65ab0df457..04bc1a09464e 100644
> > > --- a/kernel/bpf/cgroup.c
> > > +++ b/kernel/bpf/cgroup.c
> > > @@ -18,6 +18,7 @@
> > >  #include <linux/bpf.h>
> > >  #include <linux/bpf-cgroup.h>
> > >  #include <net/sock.h>
> > > +#include <net/bpf_sk_storage.h>
> > >
> > >  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
> > >  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> > > @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> > >  }
> > >  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
> > >
> > > +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> > > +                                            enum bpf_attach_type attach_type)
> > > +{
> > > +       struct bpf_prog_array *prog_array;
> > > +       bool empty;
> > > +
> > > +       rcu_read_lock();
> > > +       prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> > > +       empty = bpf_prog_array_is_empty(prog_array);
> > > +       rcu_read_unlock();
> > > +
> > > +       return empty;
> > > +}
> > > +
> > > +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> > > +{
> > > +       if (unlikely(max_optlen > PAGE_SIZE))
> > > +               return -EINVAL;
> > > +
> > > +       if (likely(max_optlen <= sizeof(ctx->buf))) {
> > > +               ctx->optval = ctx->buf;
> > > +       } else {
> > > +               ctx->optval = kzalloc(max_optlen, GFP_USER);
> > > +               if (!ctx->optval)
> > > +                       return -ENOMEM;
> > > +       }
> > > +
> > > +       ctx->optval_end = ctx->optval + max_optlen;
> > > +       ctx->optlen = max_optlen;
> > > +
> > > +       return 0;
> > > +}
> > > +
> > > +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> > > +{
> > > +       if (unlikely(ctx->optval != ctx->buf))
> > > +               kfree(ctx->optval);
> > > +}
> > > +
> > > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> > > +                                      int optname, char __user *optval,
> > > +                                      unsigned int optlen)
> > > +{
> > > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > > +       struct bpf_sockopt_kern ctx = {
> > > +               .sk = sk,
> > > +               .level = level,
> > > +               .optname = optname,
> > > +       };
> > > +       int ret;
> > > +
> > > +       /* Opportunistic check to see whether we have any BPF program
> > > +        * attached to the hook so we don't waste time allocating
> > > +        * memory and locking the socket.
> > > +        */
> > > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> > > +               return 0;
> > > +
> > > +       ret = sockopt_alloc_buf(&ctx, optlen);
> > > +       if (ret)
> > > +               return ret;
> > > +
> > > +       if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> > > +               sockopt_free_buf(&ctx);
> > > +               return -EFAULT;
> > > +       }
> > > +
> > > +       lock_sock(sk);
> > > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > > +               cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> > > +               &ctx, BPF_PROG_RUN);
> > > +       release_sock(sk);
> > > +
> > > +       sockopt_free_buf(&ctx);
> > > +
> > > +       return ret;
> > > +}
> > > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> > > +
> > > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> > > +                                      int optname, char __user *optval,
> > > +                                      int __user *optlen)
> > > +{
> > > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > > +       struct bpf_sockopt_kern ctx = {
> > > +               .sk = sk,
> > > +               .level = level,
> > > +               .optname = optname,
> > > +       };
> > > +       int max_optlen;
> > > +       int ret;
> > > +
> > > +       /* Opportunistic check to see whether we have any BPF program
> > > +        * attached to the hook so we don't waste time allocating
> > > +        * memory and locking the socket.
> > > +        */
> > > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> > > +               return 0;
> > > +
> > > +       if (get_user(max_optlen, optlen))
> > > +               return -EFAULT;
> > > +
> > > +       ret = sockopt_alloc_buf(&ctx, max_optlen);
> > > +       if (ret)
> > > +               return ret;
> > > +
> > > +       lock_sock(sk);
> > > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > > +               cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> > > +               &ctx, BPF_PROG_RUN);
> > > +       release_sock(sk);
> > > +
> > > +       if (ret < 0) {
> > > +               sockopt_free_buf(&ctx);
> > > +               return ret;
> > > +       }
> > > +
> > > +       if (ctx.optlen > max_optlen) {
> > > +               sockopt_free_buf(&ctx);
> > > +               return -EFAULT;
> > > +       }
> >
> > So this is the case where BPF program returns value that's bigger than
> > a buffer provided by users? Existing code in sock_getsockopt handles
> > that by filling out only first N bytes, instead of failing. Should the
> > behavior be the same here?
> I'm not sure. BPF program would need to handle this case anyway, right?
> Verifier would not let it put more bytes in the buffer than available,
> so increasing optlen in the BPF program doesn't really make sense.
> It feels like returning a EPERM (and assuming BPF program is buggy)
> is safer. Am I missing something?

Nope, I think you are right. There is one case, where we could allow
bump. If we use internal buffer (64 bytes) and new optlen is still
<=64, but might not be worth it.

Please add to this patch:

Acked-by: Andrii Nakryiko <andriin@fb.com>

>
> > > +       if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> > > +               sockopt_free_buf(&ctx);
> > > +               return -EFAULT;
> > > +       }
> > > +
> > > +       sockopt_free_buf(&ctx);
> > > +
> > > +       if (put_user(ctx.optlen, optlen))
> > > +               return -EFAULT;
> > > +
> > > +       return ret;
> > > +}
> > > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> > > +
> > >  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
> > >                               size_t *lenp)
> > >  {
> > > @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
> > >
> > >  const struct bpf_prog_ops cg_sysctl_prog_ops = {
> > >  };
> > > +
> > > +static const struct bpf_func_proto *
> > > +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> > > +{
> > > +       switch (func_id) {
> > > +       case BPF_FUNC_sk_fullsock:
> > > +               return &bpf_sk_fullsock_proto;
> > > +       case BPF_FUNC_sk_storage_get:
> > > +               return &bpf_sk_storage_get_proto;
> > > +       case BPF_FUNC_sk_storage_delete:
> > > +               return &bpf_sk_storage_delete_proto;
> > > +#ifdef CONFIG_INET
> > > +       case BPF_FUNC_tcp_sock:
> > > +               return &bpf_tcp_sock_proto;
> > > +#endif
> > > +       default:
> > > +               return cgroup_base_func_proto(func_id, prog);
> > > +       }
> > > +}
> > > +
> > > +static bool cg_sockopt_is_valid_access(int off, int size,
> > > +                                      enum bpf_access_type type,
> > > +                                      const struct bpf_prog *prog,
> > > +                                      struct bpf_insn_access_aux *info)
> > > +{
> > > +       const int size_default = sizeof(__u32);
> > > +
> > > +       if (off < 0 || off >= sizeof(struct bpf_sockopt))
> > > +               return false;
> > > +
> > > +       if (off % size != 0)
> > > +               return false;
> > > +
> > > +       if (type == BPF_WRITE) {
> > > +               switch (off) {
> > > +               case offsetof(struct bpf_sockopt, optlen):
> > > +                       if (size != size_default)
> > > +                               return false;
> > > +                       return prog->expected_attach_type ==
> > > +                               BPF_CGROUP_GETSOCKOPT;
> > > +               default:
> > > +                       return false;
> > > +               }
> > > +       }
> > > +
> > > +       switch (off) {
> > > +       case offsetof(struct bpf_sockopt, sk):
> > > +               if (size != sizeof(__u64))
> > > +                       return false;
> > > +               info->reg_type = PTR_TO_SOCKET;
> > > +               break;
> > > +       case bpf_ctx_range(struct bpf_sockopt, optval):
> > > +               if (size != size_default)
> > > +                       return false;
> > > +               info->reg_type = PTR_TO_PACKET;
> > > +               break;
> > > +       case bpf_ctx_range(struct bpf_sockopt, optval_end):
> > > +               if (size != size_default)
> > > +                       return false;
> > > +               info->reg_type = PTR_TO_PACKET_END;
> > > +               break;
> > > +       default:
> > > +               if (size != size_default)
> > > +                       return false;
> > > +               break;
> > > +       }
> > > +       return true;
> > > +}
> > > +
> > > +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> > > +                                        const struct bpf_insn *si,
> > > +                                        struct bpf_insn *insn_buf,
> > > +                                        struct bpf_prog *prog,
> > > +                                        u32 *target_size)
> > > +{
> > > +       struct bpf_insn *insn = insn_buf;
> > > +
> > > +       switch (si->off) {
> > > +       case offsetof(struct bpf_sockopt, sk):
> > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> > > +                                                      bpf_sockopt_kern, sk),
> > > +                                     si->dst_reg, si->src_reg,
> > > +                                     offsetof(struct bpf_sockopt_kern, sk));
> > > +               break;
> > > +       case offsetof(struct bpf_sockopt, level):
> > > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > +                                     offsetof(struct bpf_sockopt_kern, level));
> > > +               break;
> > > +       case offsetof(struct bpf_sockopt, optname):
> > > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > +                                     offsetof(struct bpf_sockopt_kern,
> > > +                                              optname));
> > > +               break;
> > > +       case offsetof(struct bpf_sockopt, optlen):
> > > +               if (type == BPF_WRITE)
> > > +                       *insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > +                                             offsetof(struct bpf_sockopt_kern,
> > > +                                                      optlen));
> > > +               else
> > > +                       *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > +                                             offsetof(struct bpf_sockopt_kern,
> > > +                                                      optlen));
> > > +               break;
> > > +       case offsetof(struct bpf_sockopt, optval):
> > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > > +                                                      optval),
> > > +                                     si->dst_reg, si->src_reg,
> > > +                                     offsetof(struct bpf_sockopt_kern,
> > > +                                              optval));
> > > +               break;
> > > +       case offsetof(struct bpf_sockopt, optval_end):
> > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > > +                                                      optval_end),
> > > +                                     si->dst_reg, si->src_reg,
> > > +                                     offsetof(struct bpf_sockopt_kern,
> > > +                                              optval_end));
> > > +               break;
> > > +       }
> > > +
> > > +       return insn - insn_buf;
> > > +}
> > > +
> > > +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> > > +                                  bool direct_write,
> > > +                                  const struct bpf_prog *prog)
> > > +{
> > > +       /* Nothing to do for sockopt argument. The data is kzalloc'ated.
> > > +        */
> > > +       return 0;
> > > +}
> > > +
> > > +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> > > +       .get_func_proto         = cg_sockopt_func_proto,
> > > +       .is_valid_access        = cg_sockopt_is_valid_access,
> > > +       .convert_ctx_access     = cg_sockopt_convert_ctx_access,
> > > +       .gen_prologue           = cg_sockopt_get_prologue,
> > > +};
> > > +
> > > +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> > > +};
> > > diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> > > index 33fb292f2e30..e9152ebd66bc 100644
> > > --- a/kernel/bpf/core.c
> > > +++ b/kernel/bpf/core.c
> > > @@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)
> > >         return cnt;
> > >  }
> > >
> > > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array)
> > > +{
> > > +       struct bpf_prog_array_item *item;
> > > +
> > > +       for (item = array->items; item->prog; item++)
> > > +               if (item->prog != &dummy_bpf_prog.prog)
> > > +                       return false;
> > > +       return true;
> > > +}
> > >
> > >  static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
> > >                                      u32 *prog_ids,
> > > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> > > index 4c53cbd3329d..4ad2b5f1905f 100644
> > > --- a/kernel/bpf/syscall.c
> > > +++ b/kernel/bpf/syscall.c
> > > @@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,
> > >                 default:
> > >                         return -EINVAL;
> > >                 }
> > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > +               switch (expected_attach_type) {
> > > +               case BPF_CGROUP_SETSOCKOPT:
> > > +               case BPF_CGROUP_GETSOCKOPT:
> > > +                       return 0;
> > > +               default:
> > > +                       return -EINVAL;
> > > +               }
> > >         default:
> > >                 return 0;
> > >         }
> > > @@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> > >         switch (prog->type) {
> > >         case BPF_PROG_TYPE_CGROUP_SOCK:
> > >         case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > >                 return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
> > >         case BPF_PROG_TYPE_CGROUP_SKB:
> > >                 return prog->enforce_expected_attach_type &&
> > > @@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)
> > >         case BPF_CGROUP_SYSCTL:
> > >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> > >                 break;
> > > +       case BPF_CGROUP_GETSOCKOPT:
> > > +       case BPF_CGROUP_SETSOCKOPT:
> > > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > > +               break;
> > >         default:
> > >                 return -EINVAL;
> > >         }
> > > @@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)
> > >         case BPF_CGROUP_SYSCTL:
> > >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> > >                 break;
> > > +       case BPF_CGROUP_GETSOCKOPT:
> > > +       case BPF_CGROUP_SETSOCKOPT:
> > > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > > +               break;
> > >         default:
> > >                 return -EINVAL;
> > >         }
> > > @@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,
> > >         case BPF_CGROUP_SOCK_OPS:
> > >         case BPF_CGROUP_DEVICE:
> > >         case BPF_CGROUP_SYSCTL:
> > > +       case BPF_CGROUP_GETSOCKOPT:
> > > +       case BPF_CGROUP_SETSOCKOPT:
> > >                 break;
> > >         case BPF_LIRC_MODE2:
> > >                 return lirc_prog_query(attr, uattr);
> > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > index 5c2cb5bd84ce..fffc668ef536 100644
> > > --- a/kernel/bpf/verifier.c
> > > +++ b/kernel/bpf/verifier.c
> > > @@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env,
> > >
> > >                 env->seen_direct_write = true;
> > >                 return true;
> > > +
> > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > +               if (t == BPF_WRITE) {
> > > +                       if (env->prog->expected_attach_type ==
> > > +                           BPF_CGROUP_GETSOCKOPT) {
> > > +                               env->seen_direct_write = true;
> > > +                               return true;
> > > +                       }
> > > +                       return false;
> > > +               }
> > > +               return true;
> > > +
> > >         default:
> > >                 return false;
> > >         }
> > > @@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)
> > >         case BPF_PROG_TYPE_CGROUP_DEVICE:
> > >         case BPF_PROG_TYPE_CGROUP_SYSCTL:
> > >                 break;
> > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > +               range = tnum_range(0, 2);
> > > +               break;
> > >         default:
> > >                 return 0;
> > >         }
> > > diff --git a/net/core/filter.c b/net/core/filter.c
> > > index 55bfc941d17a..4652c0a005ca 100644
> > > --- a/net/core/filter.c
> > > +++ b/net/core/filter.c
> > > @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
> > >         return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
> > >  }
> > >
> > > -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > > +const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > >         .func           = bpf_sk_fullsock,
> > >         .gpl_only       = false,
> > >         .ret_type       = RET_PTR_TO_SOCKET_OR_NULL,
> > > @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
> > >         return (unsigned long)NULL;
> > >  }
> > >
> > > -static const struct bpf_func_proto bpf_tcp_sock_proto = {
> > > +const struct bpf_func_proto bpf_tcp_sock_proto = {
> > >         .func           = bpf_tcp_sock,
> > >         .gpl_only       = false,
> > >         .ret_type       = RET_PTR_TO_TCP_SOCK_OR_NULL,
> > > diff --git a/net/socket.c b/net/socket.c
> > > index 72372dc5dd70..e8654f1f70e6 100644
> > > --- a/net/socket.c
> > > +++ b/net/socket.c
> > > @@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,
> > >                 if (err)
> > >                         goto out_put;
> > >
> > > +               err = BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock->sk, level, optname,
> > > +                                                    optval, optlen);
> > > +               if (err < 0) {
> > > +                       goto out_put;
> > > +               } else if (err > 0) {
> > > +                       err = 0;
> > > +                       goto out_put;
> > > +               }
> > > +
> > >                 if (level == SOL_SOCKET)
> > >                         err =
> > >                             sock_setsockopt(sock, level, optname, optval,
> > > @@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,
> > >                 if (err)
> > >                         goto out_put;
> > >
> > > +               err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname,
> > > +                                                    optval, optlen);
> > > +               if (err < 0) {
> > > +                       goto out_put;
> > > +               } else if (err > 0) {
> > > +                       err = 0;
> > > +                       goto out_put;
> > > +               }
> > > +
> > >                 if (level == SOL_SOCKET)
> > >                         err =
> > >                             sock_getsockopt(sock, level, optname, optval,
> > > --
> > > 2.22.0.rc1.311.g5d7573a151-goog
> > >

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Stanislav Fomichev]

On 06/06, Andrii Nakryiko wrote:
> On Thu, Jun 6, 2019 at 3:27 PM Stanislav Fomichev <sdf@fomichev.me> wrote:
> >
> > On 06/06, Andrii Nakryiko wrote:
> > > On Thu, Jun 6, 2019 at 12:03 PM Stanislav Fomichev <sdf@google.com> wrote:
> > > >
> > > > Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> > > > BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
> > > >
> > > > BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> > > > BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> > > > Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
> > > >
> > > > The buffer memory is pre-allocated (because I don't think there is
> > > > a precedent for working with __user memory from bpf). This might be
> > > > slow to do for each {s,g}etsockopt call, that's why I've added
> > > > __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> > > > attached to a cgroup. Note, however, that there is a race between
> > > > __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> > > > program layout might have changed; this should not be a problem
> > > > because in general there is a race between multiple calls to
> > > > {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
> > > >
> > > > The return code of the BPF program is handled as follows:
> > > > * 0: EPERM
> > > > * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> > > > * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
> > > >      prog exits
> > > >
> > > > v2:
> > > > * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> > > > * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> > > > * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> > > > * added [0,2] return code check to verifier (Martin Lau)
> > > > * dropped unused buf[64] from the stack (Martin Lau)
> > > > * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> > > > * dropped bpf_target_off from ctx rewrites (Martin Lau)
> > > > * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
> > > >
> > > > Signed-off-by: Stanislav Fomichev <sdf@google.com>
> > > > ---
> > > >  include/linux/bpf-cgroup.h |  29 ++++
> > > >  include/linux/bpf.h        |  46 ++++++
> > > >  include/linux/bpf_types.h  |   1 +
> > > >  include/linux/filter.h     |  13 ++
> > > >  include/uapi/linux/bpf.h   |  14 ++
> > > >  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
> > > >  kernel/bpf/core.c          |   9 ++
> > > >  kernel/bpf/syscall.c       |  19 +++
> > > >  kernel/bpf/verifier.c      |  15 ++
> > > >  net/core/filter.c          |   4 +-
> > > >  net/socket.c               |  18 +++
> > > >  11 files changed, 443 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> > > > index b631ee75762d..406f1ba82531 100644
> > > > --- a/include/linux/bpf-cgroup.h
> > > > +++ b/include/linux/bpf-cgroup.h
> > > > @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> > > >                                    loff_t *ppos, void **new_buf,
> > > >                                    enum bpf_attach_type type);
> > > >
> > > > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> > > > +                                      int optname, char __user *optval,
> > > > +                                      unsigned int optlen);
> > > > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> > > > +                                      int optname, char __user *optval,
> > > > +                                      int __user *optlen);
> > > > +
> > > >  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
> > > >         struct bpf_map *map)
> > > >  {
> > > > @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
> > > >         __ret;                                                                 \
> > > >  })
> > > >
> > > > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> > > > +({                                                                            \
> > > > +       int __ret = 0;                                                         \
> > > > +       if (cgroup_bpf_enabled)                                                \
> > > > +               __ret = __cgroup_bpf_run_filter_setsockopt(sock, level,        \
> > > > +                                                          optname, optval,    \
> > > > +                                                          optlen);            \
> > > > +       __ret;                                                                 \
> > > > +})
> > > > +
> > > > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> > > > +({                                                                            \
> > > > +       int __ret = 0;                                                         \
> > > > +       if (cgroup_bpf_enabled)                                                \
> > > > +               __ret = __cgroup_bpf_run_filter_getsockopt(sock, level,        \
> > > > +                                                          optname, optval,    \
> > > > +                                                          optlen);            \
> > > > +       __ret;                                                                 \
> > > > +})
> > > > +
> > > >  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> > > >                            enum bpf_prog_type ptype, struct bpf_prog *prog);
> > > >  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> > > > @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
> > > >  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
> > > >  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
> > > >  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> > > > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > > > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > > >
> > > >  #define for_each_cgroup_storage_type(stype) for (; false; )
> > > >
> > > > diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> > > > index e5a309e6a400..883a190bc0b8 100644
> > > > --- a/include/linux/bpf.h
> > > > +++ b/include/linux/bpf.h
> > > > @@ -520,6 +520,7 @@ struct bpf_prog_array {
> > > >  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
> > > >  void bpf_prog_array_free(struct bpf_prog_array *progs);
> > > >  int bpf_prog_array_length(struct bpf_prog_array *progs);
> > > > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
> > > >  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
> > > >                                 __u32 __user *prog_ids, u32 cnt);
> > > >
> > > > @@ -606,6 +607,49 @@ _out:                                                      \
> > > >                 _ret;                                   \
> > > >         })
> > > >
> > > > +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> > > > + *
> > > > + * Expected BPF program return values are:
> > > > + *   0: return -EPERM to the userspace
> > > > + *   1: sockopt was not handled by BPF, kernel should do it
> > > > + *   2: sockopt was handled by BPF, kernel not should do it and return
> > >
> > > typo: should not do it?
> > Sigh, good catch, thanks!
> >
> > > > + *      to the userspace instead
> > > > + *
> > > > + * Note, that return '0' takes precedence over everything else. In other
> > > > + * words, if any single program in the prog array has returned 0,
> > > > + * the userspace will get -EPERM (regardless of what other programs
> > > > + * return).
> > > > + *
> > > > + * The macro itself returns:
> > > > + *        0: sockopt was not handled by BPF, kernel should do it
> > > > + *        1: sockopt was handled by BPF, kernel snot hould do it
> > >
> > > typo: "snot hould do it" -> "shouldn't do it"?
> > Ditto.
> >
> > > > + *   -EPERM: return error back to userspace
> > > > + */
> > > > +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)            \
> > > > +       ({                                                              \
> > > > +               struct bpf_prog_array_item *_item;                      \
> > > > +               struct bpf_prog *_prog;                                 \
> > > > +               struct bpf_prog_array *_array;                          \
> > > > +               u32 ret;                                                \
> > > > +               u32 _success = 1;                                       \
> > > > +               u32 _bypass = 0;                                        \
> > >
> > > reverse Christmas tree? or it's not enforced in a macro?
> > I don't think we do that for other BPF_PROG_XXX_RUN_ARRAY
> > handlers. But sure, why not? :-)
> >
> > > > +               preempt_disable();                                      \
> > > > +               rcu_read_lock();                                        \
> > > > +               _array = rcu_dereference(array);                        \
> > > > +               _item = &_array->items[0];                              \
> > > > +               while ((_prog = READ_ONCE(_item->prog))) {              \
> > > > +                       bpf_cgroup_storage_set(_item->cgroup_storage);  \
> > > > +                       ret = func(_prog, ctx);                         \
> > > > +                       _success &= (ret > 0);                          \
> > > > +                       _bypass |= (ret == 2);                          \
> > > > +                       _item++;                                        \
> > > > +               }                                                       \
> > > > +               rcu_read_unlock();                                      \
> > > > +               preempt_enable();                                       \
> > > > +               ret = _success ? _bypass : -EPERM;                      \
> > > > +               ret;                                                    \
> > > > +       })
> > > > +
> > > >  #define BPF_PROG_RUN_ARRAY(array, ctx, func)           \
> > > >         __BPF_PROG_RUN_ARRAY(array, ctx, func, false)
> > > >
> > > > @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
> > > >  extern const struct bpf_func_proto bpf_get_local_storage_proto;
> > > >  extern const struct bpf_func_proto bpf_strtol_proto;
> > > >  extern const struct bpf_func_proto bpf_strtoul_proto;
> > > > +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> > > > +extern const struct bpf_func_proto bpf_tcp_sock_proto;
> > > >
> > > >  /* Shared helpers among cBPF and eBPF. */
> > > >  void bpf_user_rnd_init_once(void);
> > > > diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> > > > index 5a9975678d6f..eec5aeeeaf92 100644
> > > > --- a/include/linux/bpf_types.h
> > > > +++ b/include/linux/bpf_types.h
> > > > @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
> > > >  #ifdef CONFIG_CGROUP_BPF
> > > >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
> > > >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> > > > +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
> > > >  #endif
> > > >  #ifdef CONFIG_BPF_LIRC_MODE2
> > > >  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> > > > diff --git a/include/linux/filter.h b/include/linux/filter.h
> > > > index 43b45d6db36d..6e64d01e4e36 100644
> > > > --- a/include/linux/filter.h
> > > > +++ b/include/linux/filter.h
> > > > @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
> > > >         u64 tmp_reg;
> > > >  };
> > > >
> > > > +struct bpf_sockopt_kern {
> > > > +       struct sock     *sk;
> > > > +       u8              *optval;
> > > > +       u8              *optval_end;
> > > > +       s32             level;
> > > > +       s32             optname;
> > > > +       u32             optlen;
> > > > +
> > > > +       /* Small on-stack optval buffer to avoid small allocations.
> > > > +        */
> > > > +       u8 buf[64] __aligned(8);
> > > > +};
> > > > +
> > > >  #endif /* __LINUX_FILTER_H__ */
> > > > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > > > index 7c6aef253173..310b6bbfded8 100644
> > > > --- a/include/uapi/linux/bpf.h
> > > > +++ b/include/uapi/linux/bpf.h
> > > > @@ -170,6 +170,7 @@ enum bpf_prog_type {
> > > >         BPF_PROG_TYPE_FLOW_DISSECTOR,
> > > >         BPF_PROG_TYPE_CGROUP_SYSCTL,
> > > >         BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> > > > +       BPF_PROG_TYPE_CGROUP_SOCKOPT,
> > > >  };
> > > >
> > > >  enum bpf_attach_type {
> > > > @@ -192,6 +193,8 @@ enum bpf_attach_type {
> > > >         BPF_LIRC_MODE2,
> > > >         BPF_FLOW_DISSECTOR,
> > > >         BPF_CGROUP_SYSCTL,
> > > > +       BPF_CGROUP_GETSOCKOPT,
> > > > +       BPF_CGROUP_SETSOCKOPT,
> > > >         __MAX_BPF_ATTACH_TYPE
> > > >  };
> > > >
> > > > @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
> > > >                                  */
> > > >  };
> > > >
> > > > +struct bpf_sockopt {
> > > > +       __bpf_md_ptr(struct bpf_sock *, sk);
> > > > +
> > > > +       __s32   level;
> > > > +       __s32   optname;
> > > > +
> > > > +       __u32   optlen;
> > > > +       __u32   optval;
> > > > +       __u32   optval_end;
> > > > +};
> > > > +
> > > >  #endif /* _UAPI__LINUX_BPF_H__ */
> > > > diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> > > > index 1b65ab0df457..04bc1a09464e 100644
> > > > --- a/kernel/bpf/cgroup.c
> > > > +++ b/kernel/bpf/cgroup.c
> > > > @@ -18,6 +18,7 @@
> > > >  #include <linux/bpf.h>
> > > >  #include <linux/bpf-cgroup.h>
> > > >  #include <net/sock.h>
> > > > +#include <net/bpf_sk_storage.h>
> > > >
> > > >  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
> > > >  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> > > > @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> > > >  }
> > > >  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
> > > >
> > > > +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> > > > +                                            enum bpf_attach_type attach_type)
> > > > +{
> > > > +       struct bpf_prog_array *prog_array;
> > > > +       bool empty;
> > > > +
> > > > +       rcu_read_lock();
> > > > +       prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> > > > +       empty = bpf_prog_array_is_empty(prog_array);
> > > > +       rcu_read_unlock();
> > > > +
> > > > +       return empty;
> > > > +}
> > > > +
> > > > +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> > > > +{
> > > > +       if (unlikely(max_optlen > PAGE_SIZE))
> > > > +               return -EINVAL;
> > > > +
> > > > +       if (likely(max_optlen <= sizeof(ctx->buf))) {
> > > > +               ctx->optval = ctx->buf;
> > > > +       } else {
> > > > +               ctx->optval = kzalloc(max_optlen, GFP_USER);
> > > > +               if (!ctx->optval)
> > > > +                       return -ENOMEM;
> > > > +       }
> > > > +
> > > > +       ctx->optval_end = ctx->optval + max_optlen;
> > > > +       ctx->optlen = max_optlen;
> > > > +
> > > > +       return 0;
> > > > +}
> > > > +
> > > > +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> > > > +{
> > > > +       if (unlikely(ctx->optval != ctx->buf))
> > > > +               kfree(ctx->optval);
> > > > +}
> > > > +
> > > > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> > > > +                                      int optname, char __user *optval,
> > > > +                                      unsigned int optlen)
> > > > +{
> > > > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > > > +       struct bpf_sockopt_kern ctx = {
> > > > +               .sk = sk,
> > > > +               .level = level,
> > > > +               .optname = optname,
> > > > +       };
> > > > +       int ret;
> > > > +
> > > > +       /* Opportunistic check to see whether we have any BPF program
> > > > +        * attached to the hook so we don't waste time allocating
> > > > +        * memory and locking the socket.
> > > > +        */
> > > > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> > > > +               return 0;
> > > > +
> > > > +       ret = sockopt_alloc_buf(&ctx, optlen);
> > > > +       if (ret)
> > > > +               return ret;
> > > > +
> > > > +       if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> > > > +               sockopt_free_buf(&ctx);
> > > > +               return -EFAULT;
> > > > +       }
> > > > +
> > > > +       lock_sock(sk);
> > > > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > > > +               cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> > > > +               &ctx, BPF_PROG_RUN);
> > > > +       release_sock(sk);
> > > > +
> > > > +       sockopt_free_buf(&ctx);
> > > > +
> > > > +       return ret;
> > > > +}
> > > > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> > > > +
> > > > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> > > > +                                      int optname, char __user *optval,
> > > > +                                      int __user *optlen)
> > > > +{
> > > > +       struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > > > +       struct bpf_sockopt_kern ctx = {
> > > > +               .sk = sk,
> > > > +               .level = level,
> > > > +               .optname = optname,
> > > > +       };
> > > > +       int max_optlen;
> > > > +       int ret;
> > > > +
> > > > +       /* Opportunistic check to see whether we have any BPF program
> > > > +        * attached to the hook so we don't waste time allocating
> > > > +        * memory and locking the socket.
> > > > +        */
> > > > +       if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> > > > +               return 0;
> > > > +
> > > > +       if (get_user(max_optlen, optlen))
> > > > +               return -EFAULT;
> > > > +
> > > > +       ret = sockopt_alloc_buf(&ctx, max_optlen);
> > > > +       if (ret)
> > > > +               return ret;
> > > > +
> > > > +       lock_sock(sk);
> > > > +       ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > > > +               cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> > > > +               &ctx, BPF_PROG_RUN);
> > > > +       release_sock(sk);
> > > > +
> > > > +       if (ret < 0) {
> > > > +               sockopt_free_buf(&ctx);
> > > > +               return ret;
> > > > +       }
> > > > +
> > > > +       if (ctx.optlen > max_optlen) {
> > > > +               sockopt_free_buf(&ctx);
> > > > +               return -EFAULT;
> > > > +       }
> > >
> > > So this is the case where BPF program returns value that's bigger than
> > > a buffer provided by users? Existing code in sock_getsockopt handles
> > > that by filling out only first N bytes, instead of failing. Should the
> > > behavior be the same here?
> > I'm not sure. BPF program would need to handle this case anyway, right?
> > Verifier would not let it put more bytes in the buffer than available,
> > so increasing optlen in the BPF program doesn't really make sense.
> > It feels like returning a EPERM (and assuming BPF program is buggy)
> > is safer. Am I missing something?
> 
> Nope, I think you are right. There is one case, where we could allow
> bump. If we use internal buffer (64 bytes) and new optlen is still
> <=64, but might not be worth it.
> 
> Please add to this patch:
> 
> Acked-by: Andrii Nakryiko <andriin@fb.com>
Thank you for the review! I'll wait for Martin to do another round and
will send out v3 with the typos addressed.

> > > > +       if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> > > > +               sockopt_free_buf(&ctx);
> > > > +               return -EFAULT;
> > > > +       }
> > > > +
> > > > +       sockopt_free_buf(&ctx);
> > > > +
> > > > +       if (put_user(ctx.optlen, optlen))
> > > > +               return -EFAULT;
> > > > +
> > > > +       return ret;
> > > > +}
> > > > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> > > > +
> > > >  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
> > > >                               size_t *lenp)
> > > >  {
> > > > @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
> > > >
> > > >  const struct bpf_prog_ops cg_sysctl_prog_ops = {
> > > >  };
> > > > +
> > > > +static const struct bpf_func_proto *
> > > > +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> > > > +{
> > > > +       switch (func_id) {
> > > > +       case BPF_FUNC_sk_fullsock:
> > > > +               return &bpf_sk_fullsock_proto;
> > > > +       case BPF_FUNC_sk_storage_get:
> > > > +               return &bpf_sk_storage_get_proto;
> > > > +       case BPF_FUNC_sk_storage_delete:
> > > > +               return &bpf_sk_storage_delete_proto;
> > > > +#ifdef CONFIG_INET
> > > > +       case BPF_FUNC_tcp_sock:
> > > > +               return &bpf_tcp_sock_proto;
> > > > +#endif
> > > > +       default:
> > > > +               return cgroup_base_func_proto(func_id, prog);
> > > > +       }
> > > > +}
> > > > +
> > > > +static bool cg_sockopt_is_valid_access(int off, int size,
> > > > +                                      enum bpf_access_type type,
> > > > +                                      const struct bpf_prog *prog,
> > > > +                                      struct bpf_insn_access_aux *info)
> > > > +{
> > > > +       const int size_default = sizeof(__u32);
> > > > +
> > > > +       if (off < 0 || off >= sizeof(struct bpf_sockopt))
> > > > +               return false;
> > > > +
> > > > +       if (off % size != 0)
> > > > +               return false;
> > > > +
> > > > +       if (type == BPF_WRITE) {
> > > > +               switch (off) {
> > > > +               case offsetof(struct bpf_sockopt, optlen):
> > > > +                       if (size != size_default)
> > > > +                               return false;
> > > > +                       return prog->expected_attach_type ==
> > > > +                               BPF_CGROUP_GETSOCKOPT;
> > > > +               default:
> > > > +                       return false;
> > > > +               }
> > > > +       }
> > > > +
> > > > +       switch (off) {
> > > > +       case offsetof(struct bpf_sockopt, sk):
> > > > +               if (size != sizeof(__u64))
> > > > +                       return false;
> > > > +               info->reg_type = PTR_TO_SOCKET;
> > > > +               break;
> > > > +       case bpf_ctx_range(struct bpf_sockopt, optval):
> > > > +               if (size != size_default)
> > > > +                       return false;
> > > > +               info->reg_type = PTR_TO_PACKET;
> > > > +               break;
> > > > +       case bpf_ctx_range(struct bpf_sockopt, optval_end):
> > > > +               if (size != size_default)
> > > > +                       return false;
> > > > +               info->reg_type = PTR_TO_PACKET_END;
> > > > +               break;
> > > > +       default:
> > > > +               if (size != size_default)
> > > > +                       return false;
> > > > +               break;
> > > > +       }
> > > > +       return true;
> > > > +}
> > > > +
> > > > +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> > > > +                                        const struct bpf_insn *si,
> > > > +                                        struct bpf_insn *insn_buf,
> > > > +                                        struct bpf_prog *prog,
> > > > +                                        u32 *target_size)
> > > > +{
> > > > +       struct bpf_insn *insn = insn_buf;
> > > > +
> > > > +       switch (si->off) {
> > > > +       case offsetof(struct bpf_sockopt, sk):
> > > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> > > > +                                                      bpf_sockopt_kern, sk),
> > > > +                                     si->dst_reg, si->src_reg,
> > > > +                                     offsetof(struct bpf_sockopt_kern, sk));
> > > > +               break;
> > > > +       case offsetof(struct bpf_sockopt, level):
> > > > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > > +                                     offsetof(struct bpf_sockopt_kern, level));
> > > > +               break;
> > > > +       case offsetof(struct bpf_sockopt, optname):
> > > > +               *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > > +                                     offsetof(struct bpf_sockopt_kern,
> > > > +                                              optname));
> > > > +               break;
> > > > +       case offsetof(struct bpf_sockopt, optlen):
> > > > +               if (type == BPF_WRITE)
> > > > +                       *insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > > +                                             offsetof(struct bpf_sockopt_kern,
> > > > +                                                      optlen));
> > > > +               else
> > > > +                       *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > > > +                                             offsetof(struct bpf_sockopt_kern,
> > > > +                                                      optlen));
> > > > +               break;
> > > > +       case offsetof(struct bpf_sockopt, optval):
> > > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > > > +                                                      optval),
> > > > +                                     si->dst_reg, si->src_reg,
> > > > +                                     offsetof(struct bpf_sockopt_kern,
> > > > +                                              optval));
> > > > +               break;
> > > > +       case offsetof(struct bpf_sockopt, optval_end):
> > > > +               *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > > > +                                                      optval_end),
> > > > +                                     si->dst_reg, si->src_reg,
> > > > +                                     offsetof(struct bpf_sockopt_kern,
> > > > +                                              optval_end));
> > > > +               break;
> > > > +       }
> > > > +
> > > > +       return insn - insn_buf;
> > > > +}
> > > > +
> > > > +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> > > > +                                  bool direct_write,
> > > > +                                  const struct bpf_prog *prog)
> > > > +{
> > > > +       /* Nothing to do for sockopt argument. The data is kzalloc'ated.
> > > > +        */
> > > > +       return 0;
> > > > +}
> > > > +
> > > > +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> > > > +       .get_func_proto         = cg_sockopt_func_proto,
> > > > +       .is_valid_access        = cg_sockopt_is_valid_access,
> > > > +       .convert_ctx_access     = cg_sockopt_convert_ctx_access,
> > > > +       .gen_prologue           = cg_sockopt_get_prologue,
> > > > +};
> > > > +
> > > > +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> > > > +};
> > > > diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> > > > index 33fb292f2e30..e9152ebd66bc 100644
> > > > --- a/kernel/bpf/core.c
> > > > +++ b/kernel/bpf/core.c
> > > > @@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)
> > > >         return cnt;
> > > >  }
> > > >
> > > > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array)
> > > > +{
> > > > +       struct bpf_prog_array_item *item;
> > > > +
> > > > +       for (item = array->items; item->prog; item++)
> > > > +               if (item->prog != &dummy_bpf_prog.prog)
> > > > +                       return false;
> > > > +       return true;
> > > > +}
> > > >
> > > >  static bool bpf_prog_array_copy_core(struct bpf_prog_array *array,
> > > >                                      u32 *prog_ids,
> > > > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> > > > index 4c53cbd3329d..4ad2b5f1905f 100644
> > > > --- a/kernel/bpf/syscall.c
> > > > +++ b/kernel/bpf/syscall.c
> > > > @@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,
> > > >                 default:
> > > >                         return -EINVAL;
> > > >                 }
> > > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > > +               switch (expected_attach_type) {
> > > > +               case BPF_CGROUP_SETSOCKOPT:
> > > > +               case BPF_CGROUP_GETSOCKOPT:
> > > > +                       return 0;
> > > > +               default:
> > > > +                       return -EINVAL;
> > > > +               }
> > > >         default:
> > > >                 return 0;
> > > >         }
> > > > @@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> > > >         switch (prog->type) {
> > > >         case BPF_PROG_TYPE_CGROUP_SOCK:
> > > >         case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> > > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > >                 return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
> > > >         case BPF_PROG_TYPE_CGROUP_SKB:
> > > >                 return prog->enforce_expected_attach_type &&
> > > > @@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)
> > > >         case BPF_CGROUP_SYSCTL:
> > > >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> > > >                 break;
> > > > +       case BPF_CGROUP_GETSOCKOPT:
> > > > +       case BPF_CGROUP_SETSOCKOPT:
> > > > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > > > +               break;
> > > >         default:
> > > >                 return -EINVAL;
> > > >         }
> > > > @@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)
> > > >         case BPF_CGROUP_SYSCTL:
> > > >                 ptype = BPF_PROG_TYPE_CGROUP_SYSCTL;
> > > >                 break;
> > > > +       case BPF_CGROUP_GETSOCKOPT:
> > > > +       case BPF_CGROUP_SETSOCKOPT:
> > > > +               ptype = BPF_PROG_TYPE_CGROUP_SOCKOPT;
> > > > +               break;
> > > >         default:
> > > >                 return -EINVAL;
> > > >         }
> > > > @@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,
> > > >         case BPF_CGROUP_SOCK_OPS:
> > > >         case BPF_CGROUP_DEVICE:
> > > >         case BPF_CGROUP_SYSCTL:
> > > > +       case BPF_CGROUP_GETSOCKOPT:
> > > > +       case BPF_CGROUP_SETSOCKOPT:
> > > >                 break;
> > > >         case BPF_LIRC_MODE2:
> > > >                 return lirc_prog_query(attr, uattr);
> > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> > > > index 5c2cb5bd84ce..fffc668ef536 100644
> > > > --- a/kernel/bpf/verifier.c
> > > > +++ b/kernel/bpf/verifier.c
> > > > @@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env,
> > > >
> > > >                 env->seen_direct_write = true;
> > > >                 return true;
> > > > +
> > > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > > +               if (t == BPF_WRITE) {
> > > > +                       if (env->prog->expected_attach_type ==
> > > > +                           BPF_CGROUP_GETSOCKOPT) {
> > > > +                               env->seen_direct_write = true;
> > > > +                               return true;
> > > > +                       }
> > > > +                       return false;
> > > > +               }
> > > > +               return true;
> > > > +
> > > >         default:
> > > >                 return false;
> > > >         }
> > > > @@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)
> > > >         case BPF_PROG_TYPE_CGROUP_DEVICE:
> > > >         case BPF_PROG_TYPE_CGROUP_SYSCTL:
> > > >                 break;
> > > > +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> > > > +               range = tnum_range(0, 2);
> > > > +               break;
> > > >         default:
> > > >                 return 0;
> > > >         }
> > > > diff --git a/net/core/filter.c b/net/core/filter.c
> > > > index 55bfc941d17a..4652c0a005ca 100644
> > > > --- a/net/core/filter.c
> > > > +++ b/net/core/filter.c
> > > > @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
> > > >         return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
> > > >  }
> > > >
> > > > -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > > > +const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > > >         .func           = bpf_sk_fullsock,
> > > >         .gpl_only       = false,
> > > >         .ret_type       = RET_PTR_TO_SOCKET_OR_NULL,
> > > > @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
> > > >         return (unsigned long)NULL;
> > > >  }
> > > >
> > > > -static const struct bpf_func_proto bpf_tcp_sock_proto = {
> > > > +const struct bpf_func_proto bpf_tcp_sock_proto = {
> > > >         .func           = bpf_tcp_sock,
> > > >         .gpl_only       = false,
> > > >         .ret_type       = RET_PTR_TO_TCP_SOCK_OR_NULL,
> > > > diff --git a/net/socket.c b/net/socket.c
> > > > index 72372dc5dd70..e8654f1f70e6 100644
> > > > --- a/net/socket.c
> > > > +++ b/net/socket.c
> > > > @@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,
> > > >                 if (err)
> > > >                         goto out_put;
> > > >
> > > > +               err = BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock->sk, level, optname,
> > > > +                                                    optval, optlen);
> > > > +               if (err < 0) {
> > > > +                       goto out_put;
> > > > +               } else if (err > 0) {
> > > > +                       err = 0;
> > > > +                       goto out_put;
> > > > +               }
> > > > +
> > > >                 if (level == SOL_SOCKET)
> > > >                         err =
> > > >                             sock_setsockopt(sock, level, optname, optval,
> > > > @@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,
> > > >                 if (err)
> > > >                         goto out_put;
> > > >
> > > > +               err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname,
> > > > +                                                    optval, optlen);
> > > > +               if (err < 0) {
> > > > +                       goto out_put;
> > > > +               } else if (err > 0) {
> > > > +                       err = 0;
> > > > +                       goto out_put;
> > > > +               }
> > > > +
> > > >                 if (level == SOL_SOCKET)
> > > >                         err =
> > > >                             sock_getsockopt(sock, level, optname, optval,
> > > > --
> > > > 2.22.0.rc1.311.g5d7573a151-goog
> > > >

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Martin Lau]

On Thu, Jun 06, 2019 at 10:51:39AM -0700, Stanislav Fomichev wrote:
> Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
> 
> BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
> 
> The buffer memory is pre-allocated (because I don't think there is
> a precedent for working with __user memory from bpf). This might be
> slow to do for each {s,g}etsockopt call, that's why I've added
> __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> attached to a cgroup. Note, however, that there is a race between
> __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> program layout might have changed; this should not be a problem
> because in general there is a race between multiple calls to
> {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
> 
> The return code of the BPF program is handled as follows:
> * 0: EPERM
> * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
>      prog exits
> 
> v2:
> * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> * added [0,2] return code check to verifier (Martin Lau)
> * dropped unused buf[64] from the stack (Martin Lau)
> * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> * dropped bpf_target_off from ctx rewrites (Martin Lau)
> * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
> 
> Signed-off-by: Stanislav Fomichev <sdf@google.com>
> ---
>  include/linux/bpf-cgroup.h |  29 ++++
>  include/linux/bpf.h        |  46 ++++++
>  include/linux/bpf_types.h  |   1 +
>  include/linux/filter.h     |  13 ++
>  include/uapi/linux/bpf.h   |  14 ++
>  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
>  kernel/bpf/core.c          |   9 ++
>  kernel/bpf/syscall.c       |  19 +++
>  kernel/bpf/verifier.c      |  15 ++
>  net/core/filter.c          |   4 +-
>  net/socket.c               |  18 +++
>  11 files changed, 443 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> index b631ee75762d..406f1ba82531 100644
> --- a/include/linux/bpf-cgroup.h
> +++ b/include/linux/bpf-cgroup.h
> @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
>  				   loff_t *ppos, void **new_buf,
>  				   enum bpf_attach_type type);
>  
> +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> +				       int optname, char __user *optval,
> +				       unsigned int optlen);
> +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> +				       int optname, char __user *optval,
> +				       int __user *optlen);
> +
>  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
>  	struct bpf_map *map)
>  {
> @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
>  	__ret;								       \
>  })
>  
> +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> +({									       \
> +	int __ret = 0;							       \
> +	if (cgroup_bpf_enabled)						       \
> +		__ret = __cgroup_bpf_run_filter_setsockopt(sock, level,	       \
> +							   optname, optval,    \
> +							   optlen);	       \
> +	__ret;								       \
> +})
> +
> +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> +({									       \
> +	int __ret = 0;							       \
> +	if (cgroup_bpf_enabled)						       \
> +		__ret = __cgroup_bpf_run_filter_getsockopt(sock, level,	       \
> +							   optname, optval,    \
> +							   optlen);	       \
> +	__ret;								       \
> +})
> +
>  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
>  			   enum bpf_prog_type ptype, struct bpf_prog *prog);
>  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
>  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
>  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
>  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
>  
>  #define for_each_cgroup_storage_type(stype) for (; false; )
>  
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index e5a309e6a400..883a190bc0b8 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -520,6 +520,7 @@ struct bpf_prog_array {
>  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
>  void bpf_prog_array_free(struct bpf_prog_array *progs);
>  int bpf_prog_array_length(struct bpf_prog_array *progs);
> +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
>  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
>  				__u32 __user *prog_ids, u32 cnt);
>  
> @@ -606,6 +607,49 @@ _out:							\
>  		_ret;					\
>  	})
>  
> +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> + *
> + * Expected BPF program return values are:
> + *   0: return -EPERM to the userspace
> + *   1: sockopt was not handled by BPF, kernel should do it
> + *   2: sockopt was handled by BPF, kernel not should do it and return
> + *      to the userspace instead
> + *
> + * Note, that return '0' takes precedence over everything else. In other
> + * words, if any single program in the prog array has returned 0,
> + * the userspace will get -EPERM (regardless of what other programs
> + * return).
> + *
> + * The macro itself returns:
> + *        0: sockopt was not handled by BPF, kernel should do it
> + *        1: sockopt was handled by BPF, kernel snot hould do it
> + *   -EPERM: return error back to userspace
> + */
> +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)		\
> +	({								\
> +		struct bpf_prog_array_item *_item;			\
> +		struct bpf_prog *_prog;					\
> +		struct bpf_prog_array *_array;				\
> +		u32 ret;						\
> +		u32 _success = 1;					\
> +		u32 _bypass = 0;					\
> +		preempt_disable();					\
> +		rcu_read_lock();					\
> +		_array = rcu_dereference(array);			\
> +		_item = &_array->items[0];				\
> +		while ((_prog = READ_ONCE(_item->prog))) {		\
> +			bpf_cgroup_storage_set(_item->cgroup_storage);	\
> +			ret = func(_prog, ctx);				\
> +			_success &= (ret > 0);				\
> +			_bypass |= (ret == 2);				\
> +			_item++;					\
> +		}							\
> +		rcu_read_unlock();					\
> +		preempt_enable();					\
> +		ret = _success ? _bypass : -EPERM;			\
> +		ret;							\
> +	})
> +
>  #define BPF_PROG_RUN_ARRAY(array, ctx, func)		\
>  	__BPF_PROG_RUN_ARRAY(array, ctx, func, false)
>  
> @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
>  extern const struct bpf_func_proto bpf_get_local_storage_proto;
>  extern const struct bpf_func_proto bpf_strtol_proto;
>  extern const struct bpf_func_proto bpf_strtoul_proto;
> +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> +extern const struct bpf_func_proto bpf_tcp_sock_proto;
>  
>  /* Shared helpers among cBPF and eBPF. */
>  void bpf_user_rnd_init_once(void);
> diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> index 5a9975678d6f..eec5aeeeaf92 100644
> --- a/include/linux/bpf_types.h
> +++ b/include/linux/bpf_types.h
> @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
>  #ifdef CONFIG_CGROUP_BPF
>  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
>  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
>  #endif
>  #ifdef CONFIG_BPF_LIRC_MODE2
>  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> diff --git a/include/linux/filter.h b/include/linux/filter.h
> index 43b45d6db36d..6e64d01e4e36 100644
> --- a/include/linux/filter.h
> +++ b/include/linux/filter.h
> @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
>  	u64 tmp_reg;
>  };
>  
> +struct bpf_sockopt_kern {
> +	struct sock	*sk;
> +	u8		*optval;
> +	u8		*optval_end;
> +	s32		level;
> +	s32		optname;
> +	u32		optlen;
> +
> +	/* Small on-stack optval buffer to avoid small allocations.
> +	 */
> +	u8 buf[64] __aligned(8);
> +};
> +
>  #endif /* __LINUX_FILTER_H__ */
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 7c6aef253173..310b6bbfded8 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -170,6 +170,7 @@ enum bpf_prog_type {
>  	BPF_PROG_TYPE_FLOW_DISSECTOR,
>  	BPF_PROG_TYPE_CGROUP_SYSCTL,
>  	BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> +	BPF_PROG_TYPE_CGROUP_SOCKOPT,
>  };
>  
>  enum bpf_attach_type {
> @@ -192,6 +193,8 @@ enum bpf_attach_type {
>  	BPF_LIRC_MODE2,
>  	BPF_FLOW_DISSECTOR,
>  	BPF_CGROUP_SYSCTL,
> +	BPF_CGROUP_GETSOCKOPT,
> +	BPF_CGROUP_SETSOCKOPT,
>  	__MAX_BPF_ATTACH_TYPE
>  };
>  
> @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
>  				 */
>  };
>  
> +struct bpf_sockopt {
> +	__bpf_md_ptr(struct bpf_sock *, sk);
> +
> +	__s32	level;
> +	__s32	optname;
> +
> +	__u32	optlen;
> +	__u32	optval;
> +	__u32	optval_end;
After looking at patch 6, I think optval and optval_end should be changed
to __bpf_md_ptr(void *, optval) and __bpf_md_ptr(void *, optval_end).
That should avoid the (__u8 *)(long) casting in the bpf_prog.

They need to be moved to the top of this struct.
The is_valid_access() also needs to be adjusted on the size check.

"struct sk_msg_md" and "struct sk_reuseport_md" could be
used as the examples.

> +};
> +
>  #endif /* _UAPI__LINUX_BPF_H__ */
> diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> index 1b65ab0df457..04bc1a09464e 100644
> --- a/kernel/bpf/cgroup.c
> +++ b/kernel/bpf/cgroup.c
> @@ -18,6 +18,7 @@
>  #include <linux/bpf.h>
>  #include <linux/bpf-cgroup.h>
>  #include <net/sock.h>
> +#include <net/bpf_sk_storage.h>
>  
>  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
>  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
>  }
>  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
>  
> +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> +					     enum bpf_attach_type attach_type)
> +{
> +	struct bpf_prog_array *prog_array;
> +	bool empty;
> +
> +	rcu_read_lock();
> +	prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> +	empty = bpf_prog_array_is_empty(prog_array);
> +	rcu_read_unlock();
> +
> +	return empty;
> +}
> +
> +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> +{
> +	if (unlikely(max_optlen > PAGE_SIZE))
> +		return -EINVAL;
> +
> +	if (likely(max_optlen <= sizeof(ctx->buf))) {
> +		ctx->optval = ctx->buf;
> +	} else {
> +		ctx->optval = kzalloc(max_optlen, GFP_USER);
> +		if (!ctx->optval)
> +			return -ENOMEM;
> +	}
> +
> +	ctx->optval_end = ctx->optval + max_optlen;
> +	ctx->optlen = max_optlen;
> +
> +	return 0;
> +}
> +
> +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> +{
> +	if (unlikely(ctx->optval != ctx->buf))
> +		kfree(ctx->optval);
> +}
> +
> +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> +				       int optname, char __user *optval,
> +				       unsigned int optlen)
> +{
> +	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> +	struct bpf_sockopt_kern ctx = {
> +		.sk = sk,
> +		.level = level,
> +		.optname = optname,
> +	};
> +	int ret;
> +
> +	/* Opportunistic check to see whether we have any BPF program
> +	 * attached to the hook so we don't waste time allocating
> +	 * memory and locking the socket.
> +	 */
> +	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> +		return 0;
> +
> +	ret = sockopt_alloc_buf(&ctx, optlen);
> +	if (ret)
> +		return ret;
> +
> +	if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> +		sockopt_free_buf(&ctx);
> +		return -EFAULT;
> +	}
> +
> +	lock_sock(sk);
> +	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> +		cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> +		&ctx, BPF_PROG_RUN);
> +	release_sock(sk);
> +
> +	sockopt_free_buf(&ctx);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> +
> +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> +				       int optname, char __user *optval,
> +				       int __user *optlen)
> +{
> +	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> +	struct bpf_sockopt_kern ctx = {
> +		.sk = sk,
> +		.level = level,
> +		.optname = optname,
> +	};
> +	int max_optlen;
> +	int ret;
> +
> +	/* Opportunistic check to see whether we have any BPF program
> +	 * attached to the hook so we don't waste time allocating
> +	 * memory and locking the socket.
> +	 */
> +	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> +		return 0;
> +
> +	if (get_user(max_optlen, optlen))
> +		return -EFAULT;
> +
> +	ret = sockopt_alloc_buf(&ctx, max_optlen);
> +	if (ret)
> +		return ret;
> +
> +	lock_sock(sk);
> +	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> +		cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> +		&ctx, BPF_PROG_RUN);
> +	release_sock(sk);
> +
> +	if (ret < 0) {
> +		sockopt_free_buf(&ctx);
> +		return ret;
> +	}
> +
> +	if (ctx.optlen > max_optlen) {
> +		sockopt_free_buf(&ctx);
> +		return -EFAULT;
> +	}
> +
> +	if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> +		sockopt_free_buf(&ctx);
> +		return -EFAULT;
> +	}
> +
> +	sockopt_free_buf(&ctx);
> +
> +	if (put_user(ctx.optlen, optlen))
> +		return -EFAULT;
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> +
>  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
>  			      size_t *lenp)
>  {
> @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
>  
>  const struct bpf_prog_ops cg_sysctl_prog_ops = {
>  };
> +
> +static const struct bpf_func_proto *
> +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> +{
> +	switch (func_id) {
> +	case BPF_FUNC_sk_fullsock:
> +		return &bpf_sk_fullsock_proto;
> +	case BPF_FUNC_sk_storage_get:
> +		return &bpf_sk_storage_get_proto;
> +	case BPF_FUNC_sk_storage_delete:
> +		return &bpf_sk_storage_delete_proto;
> +#ifdef CONFIG_INET
> +	case BPF_FUNC_tcp_sock:
> +		return &bpf_tcp_sock_proto;
> +#endif
> +	default:
> +		return cgroup_base_func_proto(func_id, prog);
> +	}
> +}
> +
> +static bool cg_sockopt_is_valid_access(int off, int size,
> +				       enum bpf_access_type type,
> +				       const struct bpf_prog *prog,
> +				       struct bpf_insn_access_aux *info)
> +{
> +	const int size_default = sizeof(__u32);
> +
> +	if (off < 0 || off >= sizeof(struct bpf_sockopt))
> +		return false;
> +
> +	if (off % size != 0)
> +		return false;
> +
> +	if (type == BPF_WRITE) {
> +		switch (off) {
> +		case offsetof(struct bpf_sockopt, optlen):
> +			if (size != size_default)
> +				return false;
> +			return prog->expected_attach_type ==
> +				BPF_CGROUP_GETSOCKOPT;
> +		default:
> +			return false;
> +		}
> +	}
> +
> +	switch (off) {
> +	case offsetof(struct bpf_sockopt, sk):
> +		if (size != sizeof(__u64))
> +			return false;
> +		info->reg_type = PTR_TO_SOCKET;
> +		break;
> +	case bpf_ctx_range(struct bpf_sockopt, optval):
> +		if (size != size_default)
> +			return false;
> +		info->reg_type = PTR_TO_PACKET;
> +		break;
> +	case bpf_ctx_range(struct bpf_sockopt, optval_end):
> +		if (size != size_default)
> +			return false;
> +		info->reg_type = PTR_TO_PACKET_END;
> +		break;
> +	default:
> +		if (size != size_default)
> +			return false;
> +		break;
> +	}
> +	return true;
> +}
> +
> +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> +					 const struct bpf_insn *si,
> +					 struct bpf_insn *insn_buf,
> +					 struct bpf_prog *prog,
> +					 u32 *target_size)
> +{
> +	struct bpf_insn *insn = insn_buf;
> +
> +	switch (si->off) {
> +	case offsetof(struct bpf_sockopt, sk):
> +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> +						       bpf_sockopt_kern, sk),
> +				      si->dst_reg, si->src_reg,
> +				      offsetof(struct bpf_sockopt_kern, sk));
> +		break;
> +	case offsetof(struct bpf_sockopt, level):
> +		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +				      offsetof(struct bpf_sockopt_kern, level));
> +		break;
> +	case offsetof(struct bpf_sockopt, optname):
> +		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
Nit.
May be also use BPF_FIELD_SIZEOF() for consistency.
Same for a few BPF_W below.

> +				      offsetof(struct bpf_sockopt_kern,
> +					       optname));
> +		break;
> +	case offsetof(struct bpf_sockopt, optlen):
> +		if (type == BPF_WRITE)
> +			*insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +					      offsetof(struct bpf_sockopt_kern,
> +						       optlen));
> +		else
> +			*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> +					      offsetof(struct bpf_sockopt_kern,
> +						       optlen));
> +		break;
> +	case offsetof(struct bpf_sockopt, optval):
> +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> +						       optval),
> +				      si->dst_reg, si->src_reg,
> +				      offsetof(struct bpf_sockopt_kern,
> +					       optval));
> +		break;
> +	case offsetof(struct bpf_sockopt, optval_end):
> +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> +						       optval_end),
> +				      si->dst_reg, si->src_reg,
> +				      offsetof(struct bpf_sockopt_kern,
> +					       optval_end));
> +		break;
> +	}
> +
> +	return insn - insn_buf;
> +}
> +
> +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> +				   bool direct_write,
> +				   const struct bpf_prog *prog)
> +{
> +	/* Nothing to do for sockopt argument. The data is kzalloc'ated.
> +	 */
> +	return 0;
> +}
> +
> +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> +	.get_func_proto		= cg_sockopt_func_proto,
> +	.is_valid_access	= cg_sockopt_is_valid_access,
> +	.convert_ctx_access	= cg_sockopt_convert_ctx_access,
> +	.gen_prologue		= cg_sockopt_get_prologue,
> +};
> +
> +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> +};

[ ... ]

> diff --git a/net/core/filter.c b/net/core/filter.c
> index 55bfc941d17a..4652c0a005ca 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
>  	return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
>  }
>  
> -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> +const struct bpf_func_proto bpf_sk_fullsock_proto = {
Exposing this is no longer needed.  PTR_TO_SOCKET is already a fullsock.

>  	.func		= bpf_sk_fullsock,
>  	.gpl_only	= false,
>  	.ret_type	= RET_PTR_TO_SOCKET_OR_NULL,
> @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
>  	return (unsigned long)NULL;
>  }
>

Re: [PATCH bpf-next v2 1/8] bpf: implement getsockopt and setsockopt hooks

[Stanislav Fomichev]

On 06/07, Martin Lau wrote:
> On Thu, Jun 06, 2019 at 10:51:39AM -0700, Stanislav Fomichev wrote:
> > Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and
> > BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
> > 
> > BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments.
> > BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
> > Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
> > 
> > The buffer memory is pre-allocated (because I don't think there is
> > a precedent for working with __user memory from bpf). This might be
> > slow to do for each {s,g}etsockopt call, that's why I've added
> > __cgroup_bpf_prog_array_is_empty that exits early if there is nothing
> > attached to a cgroup. Note, however, that there is a race between
> > __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup
> > program layout might have changed; this should not be a problem
> > because in general there is a race between multiple calls to
> > {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
> > 
> > The return code of the BPF program is handled as follows:
> > * 0: EPERM
> > * 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
> > * 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF
> >      prog exits
> > 
> > v2:
> > * moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
> > * aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
> > * bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
> > * added [0,2] return code check to verifier (Martin Lau)
> > * dropped unused buf[64] from the stack (Martin Lau)
> > * use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
> > * dropped bpf_target_off from ctx rewrites (Martin Lau)
> > * use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
> > 
> > Signed-off-by: Stanislav Fomichev <sdf@google.com>
> > ---
> >  include/linux/bpf-cgroup.h |  29 ++++
> >  include/linux/bpf.h        |  46 ++++++
> >  include/linux/bpf_types.h  |   1 +
> >  include/linux/filter.h     |  13 ++
> >  include/uapi/linux/bpf.h   |  14 ++
> >  kernel/bpf/cgroup.c        | 277 +++++++++++++++++++++++++++++++++++++
> >  kernel/bpf/core.c          |   9 ++
> >  kernel/bpf/syscall.c       |  19 +++
> >  kernel/bpf/verifier.c      |  15 ++
> >  net/core/filter.c          |   4 +-
> >  net/socket.c               |  18 +++
> >  11 files changed, 443 insertions(+), 2 deletions(-)
> > 
> > diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
> > index b631ee75762d..406f1ba82531 100644
> > --- a/include/linux/bpf-cgroup.h
> > +++ b/include/linux/bpf-cgroup.h
> > @@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> >  				   loff_t *ppos, void **new_buf,
> >  				   enum bpf_attach_type type);
> >  
> > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sock, int level,
> > +				       int optname, char __user *optval,
> > +				       unsigned int optlen);
> > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sock, int level,
> > +				       int optname, char __user *optval,
> > +				       int __user *optlen);
> > +
> >  static inline enum bpf_cgroup_storage_type cgroup_storage_type(
> >  	struct bpf_map *map)
> >  {
> > @@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
> >  	__ret;								       \
> >  })
> >  
> > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen)   \
> > +({									       \
> > +	int __ret = 0;							       \
> > +	if (cgroup_bpf_enabled)						       \
> > +		__ret = __cgroup_bpf_run_filter_setsockopt(sock, level,	       \
> > +							   optname, optval,    \
> > +							   optlen);	       \
> > +	__ret;								       \
> > +})
> > +
> > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen)   \
> > +({									       \
> > +	int __ret = 0;							       \
> > +	if (cgroup_bpf_enabled)						       \
> > +		__ret = __cgroup_bpf_run_filter_getsockopt(sock, level,	       \
> > +							   optname, optval,    \
> > +							   optlen);	       \
> > +	__ret;								       \
> > +})
> > +
> >  int cgroup_bpf_prog_attach(const union bpf_attr *attr,
> >  			   enum bpf_prog_type ptype, struct bpf_prog *prog);
> >  int cgroup_bpf_prog_detach(const union bpf_attr *attr,
> > @@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,
> >  #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; })
> >  #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(type,major,minor,access) ({ 0; })
> >  #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos,nbuf) ({ 0; })
> > +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> > +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen) ({ 0; })
> >  
> >  #define for_each_cgroup_storage_type(stype) for (; false; )
> >  
> > diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> > index e5a309e6a400..883a190bc0b8 100644
> > --- a/include/linux/bpf.h
> > +++ b/include/linux/bpf.h
> > @@ -520,6 +520,7 @@ struct bpf_prog_array {
> >  struct bpf_prog_array *bpf_prog_array_alloc(u32 prog_cnt, gfp_t flags);
> >  void bpf_prog_array_free(struct bpf_prog_array *progs);
> >  int bpf_prog_array_length(struct bpf_prog_array *progs);
> > +bool bpf_prog_array_is_empty(struct bpf_prog_array *array);
> >  int bpf_prog_array_copy_to_user(struct bpf_prog_array *progs,
> >  				__u32 __user *prog_ids, u32 cnt);
> >  
> > @@ -606,6 +607,49 @@ _out:							\
> >  		_ret;					\
> >  	})
> >  
> > +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type.
> > + *
> > + * Expected BPF program return values are:
> > + *   0: return -EPERM to the userspace
> > + *   1: sockopt was not handled by BPF, kernel should do it
> > + *   2: sockopt was handled by BPF, kernel not should do it and return
> > + *      to the userspace instead
> > + *
> > + * Note, that return '0' takes precedence over everything else. In other
> > + * words, if any single program in the prog array has returned 0,
> > + * the userspace will get -EPERM (regardless of what other programs
> > + * return).
> > + *
> > + * The macro itself returns:
> > + *        0: sockopt was not handled by BPF, kernel should do it
> > + *        1: sockopt was handled by BPF, kernel snot hould do it
> > + *   -EPERM: return error back to userspace
> > + */
> > +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(array, ctx, func)		\
> > +	({								\
> > +		struct bpf_prog_array_item *_item;			\
> > +		struct bpf_prog *_prog;					\
> > +		struct bpf_prog_array *_array;				\
> > +		u32 ret;						\
> > +		u32 _success = 1;					\
> > +		u32 _bypass = 0;					\
> > +		preempt_disable();					\
> > +		rcu_read_lock();					\
> > +		_array = rcu_dereference(array);			\
> > +		_item = &_array->items[0];				\
> > +		while ((_prog = READ_ONCE(_item->prog))) {		\
> > +			bpf_cgroup_storage_set(_item->cgroup_storage);	\
> > +			ret = func(_prog, ctx);				\
> > +			_success &= (ret > 0);				\
> > +			_bypass |= (ret == 2);				\
> > +			_item++;					\
> > +		}							\
> > +		rcu_read_unlock();					\
> > +		preempt_enable();					\
> > +		ret = _success ? _bypass : -EPERM;			\
> > +		ret;							\
> > +	})
> > +
> >  #define BPF_PROG_RUN_ARRAY(array, ctx, func)		\
> >  	__BPF_PROG_RUN_ARRAY(array, ctx, func, false)
> >  
> > @@ -1054,6 +1098,8 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;
> >  extern const struct bpf_func_proto bpf_get_local_storage_proto;
> >  extern const struct bpf_func_proto bpf_strtol_proto;
> >  extern const struct bpf_func_proto bpf_strtoul_proto;
> > +extern const struct bpf_func_proto bpf_sk_fullsock_proto;
> > +extern const struct bpf_func_proto bpf_tcp_sock_proto;
> >  
> >  /* Shared helpers among cBPF and eBPF. */
> >  void bpf_user_rnd_init_once(void);
> > diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
> > index 5a9975678d6f..eec5aeeeaf92 100644
> > --- a/include/linux/bpf_types.h
> > +++ b/include/linux/bpf_types.h
> > @@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)
> >  #ifdef CONFIG_CGROUP_BPF
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev)
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl)
> > +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)
> >  #endif
> >  #ifdef CONFIG_BPF_LIRC_MODE2
> >  BPF_PROG_TYPE(BPF_PROG_TYPE_LIRC_MODE2, lirc_mode2)
> > diff --git a/include/linux/filter.h b/include/linux/filter.h
> > index 43b45d6db36d..6e64d01e4e36 100644
> > --- a/include/linux/filter.h
> > +++ b/include/linux/filter.h
> > @@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern {
> >  	u64 tmp_reg;
> >  };
> >  
> > +struct bpf_sockopt_kern {
> > +	struct sock	*sk;
> > +	u8		*optval;
> > +	u8		*optval_end;
> > +	s32		level;
> > +	s32		optname;
> > +	u32		optlen;
> > +
> > +	/* Small on-stack optval buffer to avoid small allocations.
> > +	 */
> > +	u8 buf[64] __aligned(8);
> > +};
> > +
> >  #endif /* __LINUX_FILTER_H__ */
> > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > index 7c6aef253173..310b6bbfded8 100644
> > --- a/include/uapi/linux/bpf.h
> > +++ b/include/uapi/linux/bpf.h
> > @@ -170,6 +170,7 @@ enum bpf_prog_type {
> >  	BPF_PROG_TYPE_FLOW_DISSECTOR,
> >  	BPF_PROG_TYPE_CGROUP_SYSCTL,
> >  	BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
> > +	BPF_PROG_TYPE_CGROUP_SOCKOPT,
> >  };
> >  
> >  enum bpf_attach_type {
> > @@ -192,6 +193,8 @@ enum bpf_attach_type {
> >  	BPF_LIRC_MODE2,
> >  	BPF_FLOW_DISSECTOR,
> >  	BPF_CGROUP_SYSCTL,
> > +	BPF_CGROUP_GETSOCKOPT,
> > +	BPF_CGROUP_SETSOCKOPT,
> >  	__MAX_BPF_ATTACH_TYPE
> >  };
> >  
> > @@ -3533,4 +3536,15 @@ struct bpf_sysctl {
> >  				 */
> >  };
> >  
> > +struct bpf_sockopt {
> > +	__bpf_md_ptr(struct bpf_sock *, sk);
> > +
> > +	__s32	level;
> > +	__s32	optname;
> > +
> > +	__u32	optlen;
> > +	__u32	optval;
> > +	__u32	optval_end;
> After looking at patch 6, I think optval and optval_end should be changed
> to __bpf_md_ptr(void *, optval) and __bpf_md_ptr(void *, optval_end).
> That should avoid the (__u8 *)(long) casting in the bpf_prog.
Agreed, makes sense, will switch to __bpf_md_ptr. I had them as pointers
initially, but then switched to u32 for some reason.

> They need to be moved to the top of this struct.
> The is_valid_access() also needs to be adjusted on the size check.
> 
> "struct sk_msg_md" and "struct sk_reuseport_md" could be
> used as the examples.
> 
> > +};
> > +
> >  #endif /* _UAPI__LINUX_BPF_H__ */
> > diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> > index 1b65ab0df457..04bc1a09464e 100644
> > --- a/kernel/bpf/cgroup.c
> > +++ b/kernel/bpf/cgroup.c
> > @@ -18,6 +18,7 @@
> >  #include <linux/bpf.h>
> >  #include <linux/bpf-cgroup.h>
> >  #include <net/sock.h>
> > +#include <net/bpf_sk_storage.h>
> >  
> >  DEFINE_STATIC_KEY_FALSE(cgroup_bpf_enabled_key);
> >  EXPORT_SYMBOL(cgroup_bpf_enabled_key);
> > @@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
> >  }
> >  EXPORT_SYMBOL(__cgroup_bpf_run_filter_sysctl);
> >  
> > +static bool __cgroup_bpf_prog_array_is_empty(struct cgroup *cgrp,
> > +					     enum bpf_attach_type attach_type)
> > +{
> > +	struct bpf_prog_array *prog_array;
> > +	bool empty;
> > +
> > +	rcu_read_lock();
> > +	prog_array = rcu_dereference(cgrp->bpf.effective[attach_type]);
> > +	empty = bpf_prog_array_is_empty(prog_array);
> > +	rcu_read_unlock();
> > +
> > +	return empty;
> > +}
> > +
> > +static int sockopt_alloc_buf(struct bpf_sockopt_kern *ctx, int max_optlen)
> > +{
> > +	if (unlikely(max_optlen > PAGE_SIZE))
> > +		return -EINVAL;
> > +
> > +	if (likely(max_optlen <= sizeof(ctx->buf))) {
> > +		ctx->optval = ctx->buf;
> > +	} else {
> > +		ctx->optval = kzalloc(max_optlen, GFP_USER);
> > +		if (!ctx->optval)
> > +			return -ENOMEM;
> > +	}
> > +
> > +	ctx->optval_end = ctx->optval + max_optlen;
> > +	ctx->optlen = max_optlen;
> > +
> > +	return 0;
> > +}
> > +
> > +static void sockopt_free_buf(struct bpf_sockopt_kern *ctx)
> > +{
> > +	if (unlikely(ctx->optval != ctx->buf))
> > +		kfree(ctx->optval);
> > +}
> > +
> > +int __cgroup_bpf_run_filter_setsockopt(struct sock *sk, int level,
> > +				       int optname, char __user *optval,
> > +				       unsigned int optlen)
> > +{
> > +	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > +	struct bpf_sockopt_kern ctx = {
> > +		.sk = sk,
> > +		.level = level,
> > +		.optname = optname,
> > +	};
> > +	int ret;
> > +
> > +	/* Opportunistic check to see whether we have any BPF program
> > +	 * attached to the hook so we don't waste time allocating
> > +	 * memory and locking the socket.
> > +	 */
> > +	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_SETSOCKOPT))
> > +		return 0;
> > +
> > +	ret = sockopt_alloc_buf(&ctx, optlen);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (copy_from_user(ctx.optval, optval, optlen) != 0) {
> > +		sockopt_free_buf(&ctx);
> > +		return -EFAULT;
> > +	}
> > +
> > +	lock_sock(sk);
> > +	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > +		cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT],
> > +		&ctx, BPF_PROG_RUN);
> > +	release_sock(sk);
> > +
> > +	sockopt_free_buf(&ctx);
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_setsockopt);
> > +
> > +int __cgroup_bpf_run_filter_getsockopt(struct sock *sk, int level,
> > +				       int optname, char __user *optval,
> > +				       int __user *optlen)
> > +{
> > +	struct cgroup *cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data);
> > +	struct bpf_sockopt_kern ctx = {
> > +		.sk = sk,
> > +		.level = level,
> > +		.optname = optname,
> > +	};
> > +	int max_optlen;
> > +	int ret;
> > +
> > +	/* Opportunistic check to see whether we have any BPF program
> > +	 * attached to the hook so we don't waste time allocating
> > +	 * memory and locking the socket.
> > +	 */
> > +	if (__cgroup_bpf_prog_array_is_empty(cgrp, BPF_CGROUP_GETSOCKOPT))
> > +		return 0;
> > +
> > +	if (get_user(max_optlen, optlen))
> > +		return -EFAULT;
> > +
> > +	ret = sockopt_alloc_buf(&ctx, max_optlen);
> > +	if (ret)
> > +		return ret;
> > +
> > +	lock_sock(sk);
> > +	ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
> > +		cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT],
> > +		&ctx, BPF_PROG_RUN);
> > +	release_sock(sk);
> > +
> > +	if (ret < 0) {
> > +		sockopt_free_buf(&ctx);
> > +		return ret;
> > +	}
> > +
> > +	if (ctx.optlen > max_optlen) {
> > +		sockopt_free_buf(&ctx);
> > +		return -EFAULT;
> > +	}
> > +
> > +	if (copy_to_user(optval, ctx.optval, ctx.optlen) != 0) {
> > +		sockopt_free_buf(&ctx);
> > +		return -EFAULT;
> > +	}
> > +
> > +	sockopt_free_buf(&ctx);
> > +
> > +	if (put_user(ctx.optlen, optlen))
> > +		return -EFAULT;
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL(__cgroup_bpf_run_filter_getsockopt);
> > +
> >  static ssize_t sysctl_cpy_dir(const struct ctl_dir *dir, char **bufp,
> >  			      size_t *lenp)
> >  {
> > @@ -1184,3 +1321,143 @@ const struct bpf_verifier_ops cg_sysctl_verifier_ops = {
> >  
> >  const struct bpf_prog_ops cg_sysctl_prog_ops = {
> >  };
> > +
> > +static const struct bpf_func_proto *
> > +cg_sockopt_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
> > +{
> > +	switch (func_id) {
> > +	case BPF_FUNC_sk_fullsock:
> > +		return &bpf_sk_fullsock_proto;
> > +	case BPF_FUNC_sk_storage_get:
> > +		return &bpf_sk_storage_get_proto;
> > +	case BPF_FUNC_sk_storage_delete:
> > +		return &bpf_sk_storage_delete_proto;
> > +#ifdef CONFIG_INET
> > +	case BPF_FUNC_tcp_sock:
> > +		return &bpf_tcp_sock_proto;
> > +#endif
> > +	default:
> > +		return cgroup_base_func_proto(func_id, prog);
> > +	}
> > +}
> > +
> > +static bool cg_sockopt_is_valid_access(int off, int size,
> > +				       enum bpf_access_type type,
> > +				       const struct bpf_prog *prog,
> > +				       struct bpf_insn_access_aux *info)
> > +{
> > +	const int size_default = sizeof(__u32);
> > +
> > +	if (off < 0 || off >= sizeof(struct bpf_sockopt))
> > +		return false;
> > +
> > +	if (off % size != 0)
> > +		return false;
> > +
> > +	if (type == BPF_WRITE) {
> > +		switch (off) {
> > +		case offsetof(struct bpf_sockopt, optlen):
> > +			if (size != size_default)
> > +				return false;
> > +			return prog->expected_attach_type ==
> > +				BPF_CGROUP_GETSOCKOPT;
> > +		default:
> > +			return false;
> > +		}
> > +	}
> > +
> > +	switch (off) {
> > +	case offsetof(struct bpf_sockopt, sk):
> > +		if (size != sizeof(__u64))
> > +			return false;
> > +		info->reg_type = PTR_TO_SOCKET;
> > +		break;
> > +	case bpf_ctx_range(struct bpf_sockopt, optval):
> > +		if (size != size_default)
> > +			return false;
> > +		info->reg_type = PTR_TO_PACKET;
> > +		break;
> > +	case bpf_ctx_range(struct bpf_sockopt, optval_end):
> > +		if (size != size_default)
> > +			return false;
> > +		info->reg_type = PTR_TO_PACKET_END;
> > +		break;
> > +	default:
> > +		if (size != size_default)
> > +			return false;
> > +		break;
> > +	}
> > +	return true;
> > +}
> > +
> > +static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
> > +					 const struct bpf_insn *si,
> > +					 struct bpf_insn *insn_buf,
> > +					 struct bpf_prog *prog,
> > +					 u32 *target_size)
> > +{
> > +	struct bpf_insn *insn = insn_buf;
> > +
> > +	switch (si->off) {
> > +	case offsetof(struct bpf_sockopt, sk):
> > +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct
> > +						       bpf_sockopt_kern, sk),
> > +				      si->dst_reg, si->src_reg,
> > +				      offsetof(struct bpf_sockopt_kern, sk));
> > +		break;
> > +	case offsetof(struct bpf_sockopt, level):
> > +		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +				      offsetof(struct bpf_sockopt_kern, level));
> > +		break;
> > +	case offsetof(struct bpf_sockopt, optname):
> > +		*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> Nit.
> May be also use BPF_FIELD_SIZEOF() for consistency.
> Same for a few BPF_W below.
My understanding was that we use BPF_FIELD_SIZEOF for pointers (where
the size might change on 32 vs 64 bits).
But sure, I can use it everywhere for consistency.

> > +				      offsetof(struct bpf_sockopt_kern,
> > +					       optname));
> > +		break;
> > +	case offsetof(struct bpf_sockopt, optlen):
> > +		if (type == BPF_WRITE)
> > +			*insn++ = BPF_STX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +					      offsetof(struct bpf_sockopt_kern,
> > +						       optlen));
> > +		else
> > +			*insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->src_reg,
> > +					      offsetof(struct bpf_sockopt_kern,
> > +						       optlen));
> > +		break;
> > +	case offsetof(struct bpf_sockopt, optval):
> > +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > +						       optval),
> > +				      si->dst_reg, si->src_reg,
> > +				      offsetof(struct bpf_sockopt_kern,
> > +					       optval));
> > +		break;
> > +	case offsetof(struct bpf_sockopt, optval_end):
> > +		*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern,
> > +						       optval_end),
> > +				      si->dst_reg, si->src_reg,
> > +				      offsetof(struct bpf_sockopt_kern,
> > +					       optval_end));
> > +		break;
> > +	}
> > +
> > +	return insn - insn_buf;
> > +}
> > +
> > +static int cg_sockopt_get_prologue(struct bpf_insn *insn_buf,
> > +				   bool direct_write,
> > +				   const struct bpf_prog *prog)
> > +{
> > +	/* Nothing to do for sockopt argument. The data is kzalloc'ated.
> > +	 */
> > +	return 0;
> > +}
> > +
> > +const struct bpf_verifier_ops cg_sockopt_verifier_ops = {
> > +	.get_func_proto		= cg_sockopt_func_proto,
> > +	.is_valid_access	= cg_sockopt_is_valid_access,
> > +	.convert_ctx_access	= cg_sockopt_convert_ctx_access,
> > +	.gen_prologue		= cg_sockopt_get_prologue,
> > +};
> > +
> > +const struct bpf_prog_ops cg_sockopt_prog_ops = {
> > +};
> 
> [ ... ]
> 
> > diff --git a/net/core/filter.c b/net/core/filter.c
> > index 55bfc941d17a..4652c0a005ca 100644
> > --- a/net/core/filter.c
> > +++ b/net/core/filter.c
> > @@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)
> >  	return sk_fullsock(sk) ? (unsigned long)sk : (unsigned long)NULL;
> >  }
> >  
> > -static const struct bpf_func_proto bpf_sk_fullsock_proto = {
> > +const struct bpf_func_proto bpf_sk_fullsock_proto = {
> Exposing this is no longer needed.  PTR_TO_SOCKET is already a fullsock.
> 
> >  	.func		= bpf_sk_fullsock,
> >  	.gpl_only	= false,
> >  	.ret_type	= RET_PTR_TO_SOCKET_OR_NULL,
> > @@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)
> >  	return (unsigned long)NULL;
> >  }
> >

[PATCH bpf-next v2 2/8] bpf: sync bpf.h to tools/

[Stanislav Fomichev]

Export new prog type and hook points to the libbpf.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/include/uapi/linux/bpf.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 7c6aef253173..310b6bbfded8 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -170,6 +170,7 @@ enum bpf_prog_type {
 	BPF_PROG_TYPE_FLOW_DISSECTOR,
 	BPF_PROG_TYPE_CGROUP_SYSCTL,
 	BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
+	BPF_PROG_TYPE_CGROUP_SOCKOPT,
 };
 
 enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {
 	BPF_LIRC_MODE2,
 	BPF_FLOW_DISSECTOR,
 	BPF_CGROUP_SYSCTL,
+	BPF_CGROUP_GETSOCKOPT,
+	BPF_CGROUP_SETSOCKOPT,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -3533,4 +3536,15 @@ struct bpf_sysctl {
 				 */
 };
 
+struct bpf_sockopt {
+	__bpf_md_ptr(struct bpf_sock *, sk);
+
+	__s32	level;
+	__s32	optname;
+
+	__u32	optlen;
+	__u32	optval;
+	__u32	optval_end;
+};
+
 #endif /* _UAPI__LINUX_BPF_H__ */
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 3/8] libbpf: support sockopt hooks

[Stanislav Fomichev]

Make libbpf aware of new sockopt hooks so it can derive prog type
and hook point from the section names.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/lib/bpf/libbpf.c        | 5 +++++
 tools/lib/bpf/libbpf_probes.c | 1 +
 2 files changed, 6 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index ba89d9727137..cd3c692a8b5d 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -2243,6 +2243,7 @@ static bool bpf_prog_type__needs_kver(enum bpf_prog_type type)
 	case BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE:
 	case BPF_PROG_TYPE_PERF_EVENT:
 	case BPF_PROG_TYPE_CGROUP_SYSCTL:
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
 		return false;
 	case BPF_PROG_TYPE_KPROBE:
 	default:
@@ -3196,6 +3197,10 @@ static const struct {
 						BPF_CGROUP_UDP6_SENDMSG),
 	BPF_EAPROG_SEC("cgroup/sysctl",		BPF_PROG_TYPE_CGROUP_SYSCTL,
 						BPF_CGROUP_SYSCTL),
+	BPF_EAPROG_SEC("cgroup/getsockopt",	BPF_PROG_TYPE_CGROUP_SOCKOPT,
+						BPF_CGROUP_GETSOCKOPT),
+	BPF_EAPROG_SEC("cgroup/setsockopt",	BPF_PROG_TYPE_CGROUP_SOCKOPT,
+						BPF_CGROUP_SETSOCKOPT),
 };
 
 #undef BPF_PROG_SEC_IMPL
diff --git a/tools/lib/bpf/libbpf_probes.c b/tools/lib/bpf/libbpf_probes.c
index 5e2aa83f637a..7e21db11dde8 100644
--- a/tools/lib/bpf/libbpf_probes.c
+++ b/tools/lib/bpf/libbpf_probes.c
@@ -101,6 +101,7 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,
 	case BPF_PROG_TYPE_SK_REUSEPORT:
 	case BPF_PROG_TYPE_FLOW_DISSECTOR:
 	case BPF_PROG_TYPE_CGROUP_SYSCTL:
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
 	default:
 		break;
 	}
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 4/8] selftests/bpf: test sockopt section name

[Stanislav Fomichev]

Add tests that make sure libbpf section detection works.

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/testing/selftests/bpf/test_section_names.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/bpf/test_section_names.c b/tools/testing/selftests/bpf/test_section_names.c
index bebd4fbca1f4..5f84b3b8c90b 100644
--- a/tools/testing/selftests/bpf/test_section_names.c
+++ b/tools/testing/selftests/bpf/test_section_names.c
@@ -124,6 +124,16 @@ static struct sec_name_test tests[] = {
 		{0, BPF_PROG_TYPE_CGROUP_SYSCTL, BPF_CGROUP_SYSCTL},
 		{0, BPF_CGROUP_SYSCTL},
 	},
+	{
+		"cgroup/getsockopt",
+		{0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_GETSOCKOPT},
+		{0, BPF_CGROUP_GETSOCKOPT},
+	},
+	{
+		"cgroup/setsockopt",
+		{0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_SETSOCKOPT},
+		{0, BPF_CGROUP_SETSOCKOPT},
+	},
 };
 
 static int test_prog_type_by_name(const struct sec_name_test *test)
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 5/8] selftests/bpf: add sockopt test

[Stanislav Fomichev]

Add sockopt selftests:
* require proper expected_attach_type
* enforce context field read/write access
* test bpf_sockopt_handled handler
* test EPERM
* test limiting optlen from getsockopt
* test out-of-bounds access

v2:
* use return code 2 for kernel bypass

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/testing/selftests/bpf/.gitignore     |   1 +
 tools/testing/selftests/bpf/Makefile       |   3 +-
 tools/testing/selftests/bpf/test_sockopt.c | 773 +++++++++++++++++++++
 3 files changed, 776 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/test_sockopt.c

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 7470327edcfe..3fe92601223d 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -39,3 +39,4 @@ libbpf.so.*
 test_hashmap
 test_btf_dump
 xdping
+test_sockopt
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 2b426ae1cdc9..b982393b9181 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
 	test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \
 	test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \
 	test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \
-	test_btf_dump test_cgroup_attach xdping
+	test_btf_dump test_cgroup_attach xdping test_sockopt
 
 BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c)))
 TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -101,6 +101,7 @@ $(OUTPUT)/test_netcnt: cgroup_helpers.c
 $(OUTPUT)/test_sock_fields: cgroup_helpers.c
 $(OUTPUT)/test_sysctl: cgroup_helpers.c
 $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c
+$(OUTPUT)/test_sockopt: cgroup_helpers.c
 
 .PHONY: force
 
diff --git a/tools/testing/selftests/bpf/test_sockopt.c b/tools/testing/selftests/bpf/test_sockopt.c
new file mode 100644
index 000000000000..d4932cbf2935
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_sockopt.c
@@ -0,0 +1,773 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <errno.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+
+#include <linux/filter.h>
+#include <bpf/bpf.h>
+#include <bpf/libbpf.h>
+
+#include "bpf_rlimit.h"
+#include "bpf_util.h"
+#include "cgroup_helpers.h"
+
+#define CG_PATH				"/sockopt"
+
+static char bpf_log_buf[4096];
+static bool verbose;
+
+enum sockopt_test_error {
+	OK = 0,
+	DENY_LOAD,
+	DENY_ATTACH,
+	EPERM_GETSOCKOPT,
+	EFAULT_GETSOCKOPT,
+	EPERM_SETSOCKOPT,
+};
+
+static struct sockopt_test {
+	const char			*descr;
+	const struct bpf_insn		insns[64];
+	enum bpf_attach_type		attach_type;
+	enum bpf_attach_type		expected_attach_type;
+
+	int				level;
+	int				optname;
+
+	const char			set_optval[64];
+	socklen_t			set_optlen;
+
+	const char			get_optval[64];
+	socklen_t			get_optlen;
+	socklen_t			get_optlen_ret;
+
+	enum sockopt_test_error		error;
+} tests[] = {
+
+	/* ==================== getsockopt ====================  */
+
+	{
+		.descr = "getsockopt: no expected_attach_type",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = 0,
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "getsockopt: wrong expected_attach_type",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+		.error = DENY_ATTACH,
+	},
+	{
+		.descr = "getsockopt: bypass bpf hook",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.set_optval = { 1 << 3 },
+		.set_optlen = 1,
+
+		.get_optval = { 1 << 3 },
+		.get_optlen = 1,
+	},
+	{
+		.descr = "getsockopt: return EPERM from bpf hook",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.get_optlen = 1,
+		.error = EPERM_GETSOCKOPT,
+	},
+	{
+		.descr = "getsockopt: no optval bounds check, deny loading",
+		.insns = {
+			/* r6 = ctx->optval */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval)),
+
+			/* ctx->optval[0] = 0x80 */
+			BPF_MOV64_IMM(BPF_REG_0, 0x80),
+			BPF_STX_MEM(BPF_W, BPF_REG_6, BPF_REG_0, 0),
+
+			/* return 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "getsockopt: read ctx->level",
+		.insns = {
+			/* r6 = ctx->level */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, level)),
+
+			/* if (ctx->level == 123) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 123, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.level = 123,
+
+		.get_optlen = 1,
+	},
+	{
+		.descr = "getsockopt: deny writing to ctx->level",
+		.insns = {
+			/* ctx->level = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, level)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "getsockopt: read ctx->optname",
+		.insns = {
+			/* r6 = ctx->optname */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optname)),
+
+			/* if (ctx->optname == 123) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 123, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.optname = 123,
+
+		.get_optlen = 1,
+	},
+	{
+		.descr = "getsockopt: deny writing to ctx->optname",
+		.insns = {
+			/* ctx->optname = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optname)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "getsockopt: read ctx->optlen",
+		.insns = {
+			/* r6 = ctx->optlen */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optlen)),
+
+			/* if (ctx->optlen == 64) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 64, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.get_optlen = 64,
+	},
+	{
+		.descr = "getsockopt: deny bigger ctx->optlen",
+		.insns = {
+			/* ctx->optlen = 65 */
+			BPF_MOV64_IMM(BPF_REG_0, 65),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optlen)),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.get_optlen = 64,
+
+		.error = EFAULT_GETSOCKOPT,
+	},
+	{
+		.descr = "getsockopt: support smaller ctx->optlen",
+		.insns = {
+			/* ctx->optlen = 32 */
+			BPF_MOV64_IMM(BPF_REG_0, 32),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optlen)),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.get_optlen = 64,
+		.get_optlen_ret = 32,
+	},
+	{
+		.descr = "getsockopt: deny writing to ctx->optval",
+		.insns = {
+			/* ctx->optval = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optval)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "getsockopt: deny writing to ctx->optval_end",
+		.insns = {
+			/* ctx->optval_end = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optval_end)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+
+	{
+		.descr = "getsockopt: rewrite value",
+		.insns = {
+			/* r6 = ctx->optval */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval)),
+			/* r2 = ctx->optval */
+			BPF_MOV64_REG(BPF_REG_2, BPF_REG_6),
+			/* r6 = ctx->optval + 1 */
+			BPF_ALU64_IMM(BPF_ADD, BPF_REG_6, 1),
+
+			/* r7 = ctx->optval_end */
+			BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval_end)),
+
+			/* if (ctx->optval + 1 <= ctx->optval_end) { */
+			BPF_JMP_REG(BPF_JGT, BPF_REG_6, BPF_REG_7, 1),
+			/* ctx->optval[0] = 0xF0 */
+			BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0xF0),
+			/* } */
+
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_GETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.get_optval = { 0xF0 },
+		.get_optlen = 1,
+	},
+
+	/* ==================== setsockopt ====================  */
+
+	{
+		.descr = "setsockopt: no expected_attach_type",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = 0,
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: wrong expected_attach_type",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_GETSOCKOPT,
+		.error = DENY_ATTACH,
+	},
+	{
+		.descr = "setsockopt: bypass bpf hook",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.set_optval = { 1 << 3 },
+		.set_optlen = 1,
+
+		.get_optval = { 1 << 3 },
+		.get_optlen = 1,
+	},
+	{
+		.descr = "setsockopt: return EPERM from bpf hook",
+		.insns = {
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.set_optlen = 1,
+		.error = EPERM_SETSOCKOPT,
+	},
+	{
+		.descr = "setsockopt: no optval bounds check, deny loading",
+		.insns = {
+			/* r6 = ctx->optval */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval)),
+
+			/* r0 = ctx->optval[0] */
+			BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_6, 0),
+
+			/* return 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: read ctx->level",
+		.insns = {
+			/* r6 = ctx->level */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, level)),
+
+			/* if (ctx->level == 123) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 123, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.level = 123,
+
+		.set_optlen = 1,
+	},
+	{
+		.descr = "setsockopt: deny writing to ctx->level",
+		.insns = {
+			/* ctx->level = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, level)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: read ctx->optname",
+		.insns = {
+			/* r6 = ctx->optname */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optname)),
+
+			/* if (ctx->optname == 123) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 123, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.optname = 123,
+
+		.set_optlen = 1,
+	},
+	{
+		.descr = "setsockopt: deny writing to ctx->optname",
+		.insns = {
+			/* ctx->optname = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optname)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: read ctx->optlen",
+		.insns = {
+			/* r6 = ctx->optlen */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optlen)),
+
+			/* if (ctx->optlen == 64) { */
+			BPF_JMP_IMM(BPF_JNE, BPF_REG_6, 64, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_JMP_A(1),
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.set_optlen = 64,
+	},
+	{
+		.descr = "setsockopt: deny writing to ctx->optlen",
+		.insns = {
+			/* ctx->optlen = 32 */
+			BPF_MOV64_IMM(BPF_REG_0, 32),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optlen)),
+			BPF_MOV64_IMM(BPF_REG_0, 2),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.set_optlen = 64,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: deny writing to ctx->optval",
+		.insns = {
+			/* ctx->optval = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optval)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: deny writing to ctx->optval_end",
+		.insns = {
+			/* ctx->optval_end = 1 */
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				    offsetof(struct bpf_sockopt, optval_end)),
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.error = DENY_LOAD,
+	},
+	{
+		.descr = "setsockopt: allow IP_TOS <= 128",
+		.insns = {
+			/* r6 = ctx->optval */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval)),
+			/* r7 = ctx->optval + 1 */
+			BPF_MOV64_REG(BPF_REG_7, BPF_REG_6),
+			BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, 1),
+
+			/* r8 = ctx->optval_end */
+			BPF_LDX_MEM(BPF_W, BPF_REG_8, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval_end)),
+
+			/* if (ctx->optval + 1 <= ctx->optval_end) { */
+			BPF_JMP_REG(BPF_JGT, BPF_REG_7, BPF_REG_8, 4),
+
+			/* r9 = ctx->optval[0] */
+			BPF_LDX_MEM(BPF_B, BPF_REG_9, BPF_REG_6, 0),
+
+			/* if (ctx->optval[0] < 128) */
+			BPF_JMP_IMM(BPF_JGT, BPF_REG_9, 128, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_JMP_A(1),
+			/* } */
+
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.set_optval = { 0x80 },
+		.set_optlen = 1,
+		.get_optval = { 0x80 },
+		.get_optlen = 1,
+	},
+	{
+		.descr = "setsockopt: deny IP_TOS > 128",
+		.insns = {
+			/* r6 = ctx->optval */
+			BPF_LDX_MEM(BPF_W, BPF_REG_6, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval)),
+			/* r7 = ctx->optval + 1 */
+			BPF_MOV64_REG(BPF_REG_7, BPF_REG_6),
+			BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, 1),
+
+			/* r8 = ctx->optval_end */
+			BPF_LDX_MEM(BPF_W, BPF_REG_8, BPF_REG_1,
+				    offsetof(struct bpf_sockopt, optval_end)),
+
+			/* if (ctx->optval + 1 <= ctx->optval_end) { */
+			BPF_JMP_REG(BPF_JGT, BPF_REG_7, BPF_REG_8, 4),
+
+			/* r9 = ctx->optval[0] */
+			BPF_LDX_MEM(BPF_B, BPF_REG_9, BPF_REG_6, 0),
+
+			/* if (ctx->optval[0] < 128) */
+			BPF_JMP_IMM(BPF_JGT, BPF_REG_9, 128, 2),
+			BPF_MOV64_IMM(BPF_REG_0, 1),
+			BPF_JMP_A(1),
+			/* } */
+
+			/* } else { */
+			BPF_MOV64_IMM(BPF_REG_0, 0),
+			/* } */
+
+			BPF_EXIT_INSN(),
+		},
+		.attach_type = BPF_CGROUP_SETSOCKOPT,
+		.expected_attach_type = BPF_CGROUP_SETSOCKOPT,
+
+		.level = SOL_IP,
+		.optname = IP_TOS,
+
+		.set_optval = { 0x81 },
+		.set_optlen = 1,
+		.get_optval = { 0x00 },
+		.get_optlen = 1,
+
+		.error = EPERM_SETSOCKOPT,
+	},
+};
+
+static int load_prog(const struct bpf_insn *insns,
+		     enum bpf_attach_type expected_attach_type)
+{
+	struct bpf_load_program_attr attr = {
+		.prog_type = BPF_PROG_TYPE_CGROUP_SOCKOPT,
+		.expected_attach_type = expected_attach_type,
+		.insns = insns,
+		.license = "GPL",
+		.log_level = 2,
+	};
+	int fd;
+
+	for (;
+	     insns[attr.insns_cnt].code != (BPF_JMP | BPF_EXIT);
+	     attr.insns_cnt++) {
+	}
+	attr.insns_cnt++;
+
+	fd = bpf_load_program_xattr(&attr, bpf_log_buf, sizeof(bpf_log_buf));
+	if (verbose && fd < 0)
+		fprintf(stderr, "%s\n", bpf_log_buf);
+
+	return fd;
+}
+
+static int run_test(int cgroup_fd, struct sockopt_test *test)
+{
+	int sock_fd, err, prog_fd;
+	void *optval = NULL;
+	int ret = 0;
+
+	prog_fd = load_prog(test->insns, test->expected_attach_type);
+	if (prog_fd < 0) {
+		if (test->error == DENY_LOAD)
+			return 0;
+
+		perror("bpf_program__load");
+		return -1;
+	}
+
+	err = bpf_prog_attach(prog_fd, cgroup_fd, test->attach_type, 0);
+	if (err < 0) {
+		if (test->error == DENY_ATTACH)
+			goto close_prog_fd;
+
+		perror("bpf_prog_attach");
+		ret = -1;
+		goto close_prog_fd;
+	}
+
+	sock_fd = socket(AF_INET, SOCK_STREAM, 0);
+	if (sock_fd < 0) {
+		perror("socket");
+		ret = -1;
+		goto detach_prog;
+	}
+
+	if (test->set_optlen) {
+		err = setsockopt(sock_fd, test->level, test->optname,
+				 test->set_optval, test->set_optlen);
+		if (err) {
+			if (errno == EPERM && test->error == EPERM_SETSOCKOPT)
+				goto close_sock_fd;
+
+			perror("setsockopt");
+			ret = -1;
+			goto close_sock_fd;
+		}
+	}
+
+	if (test->get_optlen) {
+		optval = malloc(test->get_optlen);
+		socklen_t optlen = test->get_optlen;
+		socklen_t expected_get_optlen = test->get_optlen_ret ?:
+			test->get_optlen;
+
+		err = getsockopt(sock_fd, test->level, test->optname,
+				 optval, &optlen);
+		if (err) {
+			if (errno == EPERM && test->error == EPERM_GETSOCKOPT)
+				goto free_optval;
+			if (errno == EFAULT && test->error == EFAULT_GETSOCKOPT)
+				goto free_optval;
+
+			perror("getsockopt");
+			ret = -1;
+			goto free_optval;
+		}
+
+		if (optlen != expected_get_optlen) {
+			perror("getsockopt optlen");
+			ret = -1;
+			goto free_optval;
+		}
+
+		if (memcmp(optval, test->get_optval, optlen) != 0) {
+			perror("getsockopt optval");
+			ret = -1;
+			goto free_optval;
+		}
+	}
+
+	ret = test->error != OK;
+
+free_optval:
+	free(optval);
+close_sock_fd:
+	close(sock_fd);
+detach_prog:
+	bpf_prog_detach2(prog_fd, cgroup_fd, test->attach_type);
+close_prog_fd:
+	close(prog_fd);
+	return ret;
+}
+
+int main(int args, char **argv)
+{
+	int err = EXIT_FAILURE, error_cnt = 0;
+	int cgroup_fd, i;
+
+	if (setup_cgroup_environment())
+		goto cleanup_obj;
+
+	cgroup_fd = create_and_get_cgroup(CG_PATH);
+	if (cgroup_fd < 0)
+		goto cleanup_cgroup_env;
+
+	if (join_cgroup(CG_PATH))
+		goto cleanup_cgroup;
+
+	for (i = 0; i < ARRAY_SIZE(tests); i++) {
+		int err = run_test(cgroup_fd, &tests[i]);
+
+		if (err)
+			error_cnt++;
+
+		printf("#%d %s: %s\n", i, err ? "FAIL" : "PASS",
+		       tests[i].descr);
+	}
+
+	printf("Summary: %ld PASSED, %d FAILED\n",
+	       ARRAY_SIZE(tests) - error_cnt, error_cnt);
+	err = error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
+
+cleanup_cgroup:
+	close(cgroup_fd);
+cleanup_cgroup_env:
+	cleanup_cgroup_environment();
+cleanup_obj:
+	return err;
+}
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 6/8] selftests/bpf: add sockopt test that exercises sk helpers

[Stanislav Fomichev]

socktop test that introduces new SOL_CUSTOM sockopt level and
stores whatever users sets in sk storage. Whenever getsockopt
is called, the original value is retrieved.

v2:
* new test

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/testing/selftests/bpf/.gitignore        |   1 +
 tools/testing/selftests/bpf/Makefile          |   3 +-
 .../testing/selftests/bpf/progs/sockopt_sk.c  |  77 +++++++++
 tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++++++++++++++++
 4 files changed, 236 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c
 create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 3fe92601223d..8ac076c311d4 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -40,3 +40,4 @@ test_hashmap
 test_btf_dump
 xdping
 test_sockopt
+test_sockopt_sk
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index b982393b9181..9c167bd5ac18 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
 	test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \
 	test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \
 	test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \
-	test_btf_dump test_cgroup_attach xdping test_sockopt
+	test_btf_dump test_cgroup_attach xdping test_sockopt test_sockopt_sk
 
 BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c)))
 TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -102,6 +102,7 @@ $(OUTPUT)/test_sock_fields: cgroup_helpers.c
 $(OUTPUT)/test_sysctl: cgroup_helpers.c
 $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c
 $(OUTPUT)/test_sockopt: cgroup_helpers.c
+$(OUTPUT)/test_sockopt_sk: cgroup_helpers.c
 
 .PHONY: force
 
diff --git a/tools/testing/selftests/bpf/progs/sockopt_sk.c b/tools/testing/selftests/bpf/progs/sockopt_sk.c
new file mode 100644
index 000000000000..fcd48647dcd1
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/sockopt_sk.c
@@ -0,0 +1,77 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+char _license[] SEC("license") = "GPL";
+__u32 _version SEC("version") = 1;
+
+#define SOL_CUSTOM			0xdeadbeef
+
+struct socket_storage {
+	__u8 val;
+};
+
+struct bpf_map_def SEC("maps") socket_storage_map = {
+	.type = BPF_MAP_TYPE_SK_STORAGE,
+	.key_size = sizeof(int),
+	.value_size = sizeof(struct socket_storage),
+	.map_flags = BPF_F_NO_PREALLOC,
+};
+BPF_ANNOTATE_KV_PAIR(socket_storage_map, int, struct socket_storage);
+
+SEC("cgroup/getsockopt")
+int _getsockopt(struct bpf_sockopt *ctx)
+{
+	__u8 *optval_end = (__u8 *)(long)ctx->optval_end;
+	__u8 *optval = (__u8 *)(long)ctx->optval;
+	struct socket_storage *storage;
+	struct bpf_sock *sk;
+
+	sk = bpf_sk_fullsock(ctx->sk);
+	if (!sk)
+		return 0; /* EPERM, deny for non-full sockets */
+
+	if (ctx->level != SOL_CUSTOM)
+		return 0; /* EPERM, deny everything except custom level */
+
+	if (optval + 1 > optval_end)
+		return 0; /* EPERM, bounds check */
+
+	storage = bpf_sk_storage_get(&socket_storage_map, sk, 0,
+				     BPF_SK_STORAGE_GET_F_CREATE);
+	if (!storage)
+		return 0; /* EPERM, couldn't get sk storage */
+
+	optval[0] = storage->val;
+	ctx->optlen = 1;
+
+	return 2; /* bypass kernel */
+}
+
+SEC("cgroup/setsockopt")
+int _setsockopt(struct bpf_sockopt *ctx)
+{
+	__u8 *optval_end = (__u8 *)(long)ctx->optval_end;
+	__u8 *optval = (__u8 *)(long)ctx->optval;
+	struct socket_storage *storage;
+	struct bpf_sock *sk;
+
+	sk = bpf_sk_fullsock(ctx->sk);
+	if (!sk)
+		return 0; /* EPERM, deny for non-full sockets */
+
+	if (ctx->level != SOL_CUSTOM)
+		return 0; /* EPERM, deny everything except custom level */
+
+	if (optval + 1 > optval_end)
+		return 0; /* EPERM, bounds check */
+
+	storage = bpf_sk_storage_get(&socket_storage_map, sk, 0,
+				     BPF_SK_STORAGE_GET_F_CREATE);
+	if (!storage)
+		return 0; /* EPERM, couldn't get sk storage */
+
+	storage->val = optval[0];
+
+	return 2; /* bypass kernel */
+}
diff --git a/tools/testing/selftests/bpf/test_sockopt_sk.c b/tools/testing/selftests/bpf/test_sockopt_sk.c
new file mode 100644
index 000000000000..1acc055f94ac
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_sockopt_sk.c
@@ -0,0 +1,156 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <errno.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+
+#include <linux/filter.h>
+#include <bpf/bpf.h>
+#include <bpf/libbpf.h>
+
+#include "bpf_rlimit.h"
+#include "bpf_util.h"
+#include "cgroup_helpers.h"
+
+#define CG_PATH				"/sockopt"
+
+#define SOL_CUSTOM			0xdeadbeef
+
+static int getsetsockopt(void)
+{
+	int fd, err;
+	char buf[4] = { 0x01, 0x00, 0x00, 0x00 };
+	socklen_t optlen;
+
+	fd = socket(AF_INET, SOCK_STREAM, 0);
+	if (fd < 0) {
+		log_err("Failed to create socket");
+		return -1;
+	}
+
+	err = setsockopt(fd, SOL_IP, IP_TOS, buf, 1);
+	if (!err || errno != EPERM) {
+		log_err("Unexpected success from setsockopt");
+		goto err;
+	}
+
+	err = setsockopt(fd, SOL_CUSTOM, 0, buf, 1);
+	if (err) {
+		log_err("Failed to call setsockopt");
+		goto err;
+	}
+
+	buf[0] = 0;
+	optlen = 4;
+	err = getsockopt(fd, SOL_CUSTOM, 0, buf, &optlen);
+	if (err) {
+		log_err("Failed to call getsockopt");
+		goto err;
+	}
+
+	if (optlen != 1) {
+		log_err("Unexpected optlen %d != 1", optlen);
+		goto err;
+	}
+	if (buf[0] != 0x01) {
+		log_err("Unexpected buf[0] 0x%02x != 0x01", buf[0]);
+		goto err;
+	}
+
+	close(fd);
+	return 0;
+err:
+	close(fd);
+	return -1;
+}
+
+static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title)
+{
+	enum bpf_attach_type attach_type;
+	enum bpf_prog_type prog_type;
+	struct bpf_program *prog;
+	int err;
+
+	err = libbpf_prog_type_by_name(title, &prog_type, &attach_type);
+	if (err) {
+		log_err("Failed to deduct types for %s BPF program", title);
+		return -1;
+	}
+
+	prog = bpf_object__find_program_by_title(obj, title);
+	if (!prog) {
+		log_err("Failed to find %s BPF program", title);
+		return -1;
+	}
+
+	err = bpf_prog_attach(bpf_program__fd(prog), cgroup_fd,
+			      attach_type, 0);
+	if (err) {
+		log_err("Failed to attach %s BPF program", title);
+		return -1;
+	}
+
+	return 0;
+}
+
+static int run_test(int cgroup_fd)
+{
+	struct bpf_prog_load_attr attr = {
+		.file = "./sockopt_sk.o",
+	};
+	struct bpf_object *obj;
+	int ignored;
+	int err;
+
+	err = bpf_prog_load_xattr(&attr, &obj, &ignored);
+	if (err) {
+		log_err("Failed to load BPF object");
+		return -1;
+	}
+
+	err = prog_attach(obj, cgroup_fd, "cgroup/getsockopt");
+	if (err)
+		goto close_bpf_object;
+
+	err = prog_attach(obj, cgroup_fd, "cgroup/setsockopt");
+	if (err)
+		goto close_bpf_object;
+
+	err = getsetsockopt();
+
+close_bpf_object:
+	bpf_object__close(obj);
+	return err;
+}
+
+int main(int args, char **argv)
+{
+	int cgroup_fd;
+	int err = EXIT_SUCCESS;
+
+	if (setup_cgroup_environment())
+		goto cleanup_obj;
+
+	cgroup_fd = create_and_get_cgroup(CG_PATH);
+	if (cgroup_fd < 0)
+		goto cleanup_cgroup_env;
+
+	if (join_cgroup(CG_PATH))
+		goto cleanup_cgroup;
+
+	if (run_test(cgroup_fd))
+		err = EXIT_FAILURE;
+
+	printf("test_sockopt_sk: %s\n",
+	       err == EXIT_SUCCESS ? "PASSED" : "FAILED");
+
+cleanup_cgroup:
+	close(cgroup_fd);
+cleanup_cgroup_env:
+	cleanup_cgroup_environment();
+cleanup_obj:
+	return err;
+}
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 7/8] bpf: add sockopt documentation

[Stanislav Fomichev]

Provide user documentation about sockopt prog type and cgroup hooks.

v2:
* use return code 2 for kernel bypass

Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 Documentation/bpf/index.rst               |  1 +
 Documentation/bpf/prog_cgroup_sockopt.rst | 39 +++++++++++++++++++++++
 2 files changed, 40 insertions(+)
 create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst

diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst
index d3fe4cac0c90..801a6ed3f2e5 100644
--- a/Documentation/bpf/index.rst
+++ b/Documentation/bpf/index.rst
@@ -42,6 +42,7 @@ Program types
 .. toctree::
    :maxdepth: 1
 
+   prog_cgroup_sockopt
    prog_cgroup_sysctl
    prog_flow_dissector
 
diff --git a/Documentation/bpf/prog_cgroup_sockopt.rst b/Documentation/bpf/prog_cgroup_sockopt.rst
new file mode 100644
index 000000000000..b117451ab571
--- /dev/null
+++ b/Documentation/bpf/prog_cgroup_sockopt.rst
@@ -0,0 +1,39 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================
+BPF_PROG_TYPE_SOCKOPT
+=====================
+
+``BPF_PROG_TYPE_SOCKOPT`` program type can be attached to two cgroup hooks:
+
+* ``BPF_CGROUP_GETSOCKOPT`` - called every time process executes ``getsockopt``
+  system call.
+* ``BPF_CGROUP_SETSOCKOPT`` - called every time process executes ``setsockopt``
+  system call.
+
+The context (``struct bpf_sockopt``) has associated socket (``sk``) and
+all input arguments: ``level``, ``optname``, ``optval`` and ``optlen``.
+
+BPF_CGROUP_SETSOCKOPT
+=====================
+
+``BPF_CGROUP_SETSOCKOPT`` has a read-only context and this hook has
+access to cgroup and socket local storage.
+
+BPF_CGROUP_GETSOCKOPT
+=====================
+
+``BPF_CGROUP_GETSOCKOPT`` has to fill in ``optval`` and adjust
+``optlen`` accordingly. Input ``optlen`` contains the maximum length
+of data that can be returned to the userspace. In other words, BPF
+program can't increase ``optlen``, it can only decrease it.
+
+Return Type
+===========
+
+* ``0`` - reject the syscall, ``EPERM`` will be returned to the userspace.
+* ``1`` - success: after returning from the BPF hook, kernel will also
+  handle this socket option.
+* ``2`` - success: after returning from the BPF hook, kernel will _not_
+  handle this socket option; control will be returned to the userspace
+  instead.
-- 
2.22.0.rc1.311.g5d7573a151-goog

[PATCH bpf-next v2 8/8] bpftool: support cgroup sockopt

[Stanislav Fomichev]

Support sockopt prog type and cgroup hooks in the bpftool.

Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
 tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 7 +++++--
 tools/bpf/bpftool/Documentation/bpftool-prog.rst   | 2 +-
 tools/bpf/bpftool/bash-completion/bpftool          | 8 +++++---
 tools/bpf/bpftool/cgroup.c                         | 5 ++++-
 tools/bpf/bpftool/main.h                           | 1 +
 tools/bpf/bpftool/prog.c                           | 3 ++-
 6 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst b/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
index 36807735e2a5..cac088a320a6 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
@@ -29,7 +29,8 @@ CGROUP COMMANDS
 |	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
 |	*ATTACH_TYPE* := { **ingress** | **egress** | **sock_create** | **sock_ops** | **device** |
 |		**bind4** | **bind6** | **post_bind4** | **post_bind6** | **connect4** | **connect6** |
-|		**sendmsg4** | **sendmsg6** | **sysctl** }
+|		**sendmsg4** | **sendmsg6** | **sysctl** | **getsockopt** |
+|		**setsockopt** }
 |	*ATTACH_FLAGS* := { **multi** | **override** }
 
 DESCRIPTION
@@ -86,7 +87,9 @@ DESCRIPTION
 		  unconnected udp4 socket (since 4.18);
 		  **sendmsg6** call to sendto(2), sendmsg(2), sendmmsg(2) for an
 		  unconnected udp6 socket (since 4.18);
-		  **sysctl** sysctl access (since 5.2).
+		  **sysctl** sysctl access (since 5.2);
+		  **getsockopt** call to getsockopt (since 5.3);
+		  **setsockopt** call to setsockopt (since 5.3).
 
 	**bpftool cgroup detach** *CGROUP* *ATTACH_TYPE* *PROG*
 		  Detach *PROG* from the cgroup *CGROUP* and attach type
diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
index 228a5c863cc7..c6bade35032c 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
@@ -40,7 +40,7 @@ PROG COMMANDS
 |		**lwt_seg6local** | **sockops** | **sk_skb** | **sk_msg** | **lirc_mode2** |
 |		**cgroup/bind4** | **cgroup/bind6** | **cgroup/post_bind4** | **cgroup/post_bind6** |
 |		**cgroup/connect4** | **cgroup/connect6** | **cgroup/sendmsg4** | **cgroup/sendmsg6** |
-|		**cgroup/sysctl**
+|		**cgroup/sysctl** | **cgroup/getsockopt** | **cgroup/setsockopt**
 |	}
 |       *ATTACH_TYPE* := {
 |		**msg_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector**
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index 2725e27dfa42..7afb8b6fbaaa 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -378,7 +378,8 @@ _bpftool()
                                 cgroup/connect4 cgroup/connect6 \
                                 cgroup/sendmsg4 cgroup/sendmsg6 \
                                 cgroup/post_bind4 cgroup/post_bind6 \
-                                cgroup/sysctl" -- \
+                                cgroup/sysctl cgroup/getsockopt \
+                                cgroup/setsockopt" -- \
                                                    "$cur" ) )
                             return 0
                             ;;
@@ -688,7 +689,8 @@ _bpftool()
                 attach|detach)
                     local ATTACH_TYPES='ingress egress sock_create sock_ops \
                         device bind4 bind6 post_bind4 post_bind6 connect4 \
-                        connect6 sendmsg4 sendmsg6 sysctl'
+                        connect6 sendmsg4 sendmsg6 sysctl getsockopt \
+                        setsockopt'
                     local ATTACH_FLAGS='multi override'
                     local PROG_TYPE='id pinned tag'
                     case $prev in
@@ -698,7 +700,7 @@ _bpftool()
                             ;;
                         ingress|egress|sock_create|sock_ops|device|bind4|bind6|\
                         post_bind4|post_bind6|connect4|connect6|sendmsg4|\
-                        sendmsg6|sysctl)
+                        sendmsg6|sysctl|getsockopt|setsockopt)
                             COMPREPLY=( $( compgen -W "$PROG_TYPE" -- \
                                 "$cur" ) )
                             return 0
diff --git a/tools/bpf/bpftool/cgroup.c b/tools/bpf/bpftool/cgroup.c
index 7e22f115c8c1..3083f2e4886e 100644
--- a/tools/bpf/bpftool/cgroup.c
+++ b/tools/bpf/bpftool/cgroup.c
@@ -25,7 +25,8 @@
 	"       ATTACH_TYPE := { ingress | egress | sock_create |\n"	       \
 	"                        sock_ops | device | bind4 | bind6 |\n"	       \
 	"                        post_bind4 | post_bind6 | connect4 |\n"       \
-	"                        connect6 | sendmsg4 | sendmsg6 | sysctl }"
+	"                        connect6 | sendmsg4 | sendmsg6 | sysctl |\n"  \
+	"                        getsockopt | setsockopt }"
 
 static const char * const attach_type_strings[] = {
 	[BPF_CGROUP_INET_INGRESS] = "ingress",
@@ -42,6 +43,8 @@ static const char * const attach_type_strings[] = {
 	[BPF_CGROUP_UDP4_SENDMSG] = "sendmsg4",
 	[BPF_CGROUP_UDP6_SENDMSG] = "sendmsg6",
 	[BPF_CGROUP_SYSCTL] = "sysctl",
+	[BPF_CGROUP_GETSOCKOPT] = "getsockopt",
+	[BPF_CGROUP_SETSOCKOPT] = "setsockopt",
 	[__MAX_BPF_ATTACH_TYPE] = NULL,
 };
 
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 28a2a5857e14..9c5d9c80f71e 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -74,6 +74,7 @@ static const char * const prog_type_name[] = {
 	[BPF_PROG_TYPE_SK_REUSEPORT]		= "sk_reuseport",
 	[BPF_PROG_TYPE_FLOW_DISSECTOR]		= "flow_dissector",
 	[BPF_PROG_TYPE_CGROUP_SYSCTL]		= "cgroup_sysctl",
+	[BPF_PROG_TYPE_CGROUP_SOCKOPT]		= "cgroup_sockopt",
 };
 
 extern const char * const map_type_name[];
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 1f209c80d906..a201e1c83346 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -1070,7 +1070,8 @@ static int do_help(int argc, char **argv)
 		"                 sk_reuseport | flow_dissector | cgroup/sysctl |\n"
 		"                 cgroup/bind4 | cgroup/bind6 | cgroup/post_bind4 |\n"
 		"                 cgroup/post_bind6 | cgroup/connect4 | cgroup/connect6 |\n"
-		"                 cgroup/sendmsg4 | cgroup/sendmsg6 }\n"
+		"                 cgroup/sendmsg4 | cgroup/sendmsg6 | cgroup/getsockopt |\n"
+		"                 cgroup/setsockopt }\n"
 		"       ATTACH_TYPE := { msg_verdict | stream_verdict | stream_parser |\n"
 		"                        flow_dissector }\n"
 		"       " HELP_SPEC_OPTIONS "\n"
-- 
2.22.0.rc1.311.g5d7573a151-goog