[PATCHv3 bpf-next 00/26] bpf: Add multi uprobe link

[PATCHv3 bpf-next 00/26] bpf: Add multi uprobe link

[Jiri Olsa]

hi,
this patchset is adding support to attach multiple uprobes and usdt probes
through new uprobe_multi link.

The current uprobe is attached through the perf event and attaching many
uprobes takes a lot of time because of that.

The main reason is that we need to install perf event for each probed function
and profile shows perf event installation (perf_install_in_context) as culprit.

The new uprobe_multi link just creates raw uprobes and attaches the bpf
program to them without perf event being involved.

In addition to being faster we also save file descriptors. For the current
uprobe attach we use extra perf event fd for each probed function. The new
link just need one fd that covers all the functions we are attaching to.

v3 changes:
  - consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
  - remove bpf_prog_active check [Alexei]
  - change rcu locking [Andrii]
  - allocate ref_ctr_offsets conditionally [Andrii]
  - remove ctx check from bpf_uprobe_multi_cookie [Andrii]
  - move some elf_* functions in elf.c object [Andrii]
  - fix uprobe link detection code [Andrii]
  - add usdt.s program section [Andrii]
  - rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
  - remove extra case from attach_uprobe_multi [Andrii]
  - rework usdt_manager_attach_usdt [Andrii]
  - rework/rename new elf_find_* functions [Andrii]
  - elf iterator fixes [Andrii]
    - renames
    - elf_sym_iter_next loop restruct
    - return directly GElf_Sym in elf_sym
    - add elf_sym_offset helper
    - add st_type arg elf_sym_iter_new
    - get rid of '_' prefixed functions
    - simplify offsets handling
    - other smaller changes
  - added acks
  - todo:
    - seems like more elf_* helpers could go in elf.c object,
      I'll send that as follow up
    - will send fill_info support when [3] is merged [Andrii]


There's support for bpftrace [2] and tetragon [1].

Also available at:
  https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  uprobe_multi

thanks,
jirka


[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/20230628115329.248450-1-laoar.shao@gmail.com/
---
Jiri Olsa (26):
      bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
      bpf: Add multi uprobe link
      bpf: Add cookies support for uprobe_multi link
      bpf: Add pid filter support for uprobe_multi link
      bpf: Add bpf_get_func_ip helper support for uprobe link
      libbpf: Add uprobe_multi attach type and link names
      libbpf: Move elf_find_func_offset* functions to elf object
      libbpf: Add elf_open/elf_close functions
      libbpf: Add elf symbol iterator
      libbpf: Add elf_resolve_syms_offsets function
      libbpf: Add elf_resolve_pattern_offsets function
      libbpf: Add bpf_link_create support for multi uprobes
      libbpf: Add bpf_program__attach_uprobe_multi function
      libbpf: Add support for u[ret]probe.multi[.s] program sections
      libbpf: Add uprobe multi link detection
      libbpf: Add uprobe multi link support to bpf_program__attach_usdt
      selftests/bpf: Add uprobe_multi skel test
      selftests/bpf: Add uprobe_multi api test
      selftests/bpf: Add uprobe_multi link test
      selftests/bpf: Add uprobe_multi test program
      selftests/bpf: Add uprobe_multi bench test
      selftests/bpf: Add usdt_multi test program
      selftests/bpf: Add usdt_multi bench test
      selftests/bpf: Add uprobe_multi cookie test
      selftests/bpf: Add uprobe_multi pid filter tests
      selftests/bpf: Add extra link to uprobe_multi tests

 include/linux/trace_events.h                               |   6 ++
 include/uapi/linux/bpf.h                                   |  16 ++++
 kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
 kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 tools/include/uapi/linux/bpf.h                             |  16 ++++
 tools/lib/bpf/Build                                        |   2 +-
 tools/lib/bpf/bpf.c                                        |  11 +++
 tools/lib/bpf/bpf.h                                        |  11 ++-
 tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
 tools/lib/bpf/libbpf.h                                     |  27 ++++++
 tools/lib/bpf/libbpf.map                                   |   1 +
 tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
 tools/lib/bpf/libbpf_internal.h                            |   3 +
 tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
 tools/testing/selftests/bpf/Makefile                       |  10 +++
 tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
 tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
 tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
 22 files changed, 1969 insertions(+), 296 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
 create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
 create mode 100644 tools/testing/selftests/bpf/usdt_multi.c

[PATCHv3 bpf-next 01/26] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type

[Jiri Olsa]

Add extra attach_type checks from link_create under
bpf_prog_attach_check_attach_type.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 61 deletions(-)

diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index a2aef900519c..9046ad0f9b4e 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)
 	return fd;
 }
 
-static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
-					     enum bpf_attach_type attach_type)
-{
-	switch (prog->type) {
-	case BPF_PROG_TYPE_CGROUP_SOCK:
-	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
-	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
-	case BPF_PROG_TYPE_SK_LOOKUP:
-		return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
-	case BPF_PROG_TYPE_CGROUP_SKB:
-		if (!capable(CAP_NET_ADMIN))
-			/* cg-skb progs can be loaded by unpriv user.
-			 * check permissions at attach time.
-			 */
-			return -EPERM;
-		return prog->enforce_expected_attach_type &&
-			prog->expected_attach_type != attach_type ?
-			-EINVAL : 0;
-	case BPF_PROG_TYPE_KPROBE:
-		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
-		    attach_type != BPF_TRACE_KPROBE_MULTI)
-			return -EINVAL;
-		return 0;
-	default:
-		return 0;
-	}
-}
-
 static enum bpf_prog_type
 attach_type_to_prog_type(enum bpf_attach_type attach_type)
 {
@@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type)
 	}
 }
 
+static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
+					     enum bpf_attach_type attach_type)
+{
+	enum bpf_prog_type ptype;
+
+	switch (prog->type) {
+	case BPF_PROG_TYPE_CGROUP_SOCK:
+	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+	case BPF_PROG_TYPE_SK_LOOKUP:
+		return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
+	case BPF_PROG_TYPE_CGROUP_SKB:
+		if (!capable(CAP_NET_ADMIN))
+			/* cg-skb progs can be loaded by unpriv user.
+			 * check permissions at attach time.
+			 */
+			return -EPERM;
+		return prog->enforce_expected_attach_type &&
+			prog->expected_attach_type != attach_type ?
+			-EINVAL : 0;
+	case BPF_PROG_TYPE_KPROBE:
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI)
+			return -EINVAL;
+		if (attach_type != BPF_PERF_EVENT &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI)
+			return -EINVAL;
+		return 0;
+	case BPF_PROG_TYPE_EXT:
+		return 0;
+	case BPF_PROG_TYPE_NETFILTER:
+		if (attach_type != BPF_NETFILTER)
+			return -EINVAL;
+		return 0;
+	case BPF_PROG_TYPE_PERF_EVENT:
+	case BPF_PROG_TYPE_TRACEPOINT:
+		if (attach_type != BPF_PERF_EVENT)
+			return -EINVAL;
+		return 0;
+	default:
+		ptype = attach_type_to_prog_type(attach_type);
+		if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type)
+			return -EINVAL;
+		return 0;
+	}
+}
+
 #define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd
 
 #define BPF_F_ATTACH_MASK \
@@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 #define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
-	enum bpf_prog_type ptype;
 	struct bpf_prog *prog;
 	int ret;
 
@@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 	if (ret)
 		goto out;
 
-	switch (prog->type) {
-	case BPF_PROG_TYPE_EXT:
-		break;
-	case BPF_PROG_TYPE_NETFILTER:
-		if (attr->link_create.attach_type != BPF_NETFILTER) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	case BPF_PROG_TYPE_PERF_EVENT:
-	case BPF_PROG_TYPE_TRACEPOINT:
-		if (attr->link_create.attach_type != BPF_PERF_EVENT) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	case BPF_PROG_TYPE_KPROBE:
-		if (attr->link_create.attach_type != BPF_PERF_EVENT &&
-		    attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	default:
-		ptype = attach_type_to_prog_type(attr->link_create.attach_type);
-		if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	}
-
 	switch (prog->type) {
 	case BPF_PROG_TYPE_CGROUP_SKB:
 	case BPF_PROG_TYPE_CGROUP_SOCK:
-- 
2.41.0

[PATCHv3 bpf-next 02/26] bpf: Add multi uprobe link

[Jiri Olsa]

Adding new multi uprobe link that allows to attach bpf program
to multiple uprobes.

Uprobes to attach are specified via new link_create uprobe_multi
union:

  struct {
          __u32           flags;
          __u32           cnt;
          __aligned_u64   path;
          __aligned_u64   offsets;
          __aligned_u64   ref_ctr_offsets;
  } uprobe_multi;

Uprobes are defined for single binary specified in path and multiple
calling sites specified in offsets array with optional reference
counters specified in ref_ctr_offsets array. All specified arrays
have length of 'cnt'.

The 'flags' supports single bit for now that marks the uprobe as
return probe.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/linux/trace_events.h   |   6 +
 include/uapi/linux/bpf.h       |  14 ++
 kernel/bpf/syscall.c           |  14 +-
 kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  14 ++
 5 files changed, 282 insertions(+), 3 deletions(-)

diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 7c4a0b72334e..c71845e9d40a 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id,
 			    u32 *fd_type, const char **buf,
 			    u64 *probe_offset, u64 *probe_addr);
 int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
+int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
 #else
 static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx)
 {
@@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 {
 	return -EOPNOTSUPP;
 }
+static inline int
+bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 enum {
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 60a9d59beeab..a236139f08ce 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
 	BPF_LSM_CGROUP,
 	BPF_STRUCT_OPS,
 	BPF_NETFILTER,
+	BPF_TRACE_UPROBE_MULTI,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_KPROBE_MULTI = 8,
 	BPF_LINK_TYPE_STRUCT_OPS = 9,
 	BPF_LINK_TYPE_NETFILTER = 10,
+	BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
 				__s32		priority;
 				__u32		flags;
 			} netfilter;
+			struct {
+				__u32		flags;
+				__u32		cnt;
+				__aligned_u64	path;
+				__aligned_u64	offsets;
+				__aligned_u64	ref_ctr_offsets;
+			} uprobe_multi;
 		};
 	} link_create;
 
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 9046ad0f9b4e..3b0582a64ce4 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
 
 /* Clean up bpf_link and corresponding anon_inode file and FD. After
  * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
- * anon_inode's release() call. This helper marksbpf_link as
+ * anon_inode's release() call. This helper marks bpf_link as
  * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
  * is not decremented, it's the responsibility of a calling code that failed
  * to complete bpf_link initialization.
+ * This helper eventually calls link's dealloc callback, but does not call
+ * link's release callback.
  */
 void bpf_link_cleanup(struct bpf_link_primer *primer)
 {
@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
 		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
 		    attach_type != BPF_TRACE_KPROBE_MULTI)
 			return -EINVAL;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
+		    attach_type != BPF_TRACE_UPROBE_MULTI)
+			return -EINVAL;
 		if (attach_type != BPF_PERF_EVENT &&
-		    attach_type != BPF_TRACE_KPROBE_MULTI)
+		    attach_type != BPF_TRACE_KPROBE_MULTI &&
+		    attach_type != BPF_TRACE_UPROBE_MULTI)
 			return -EINVAL;
 		return 0;
 	case BPF_PROG_TYPE_EXT:
@@ -4748,8 +4754,10 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 	case BPF_PROG_TYPE_KPROBE:
 		if (attr->link_create.attach_type == BPF_PERF_EVENT)
 			ret = bpf_perf_link_attach(attr, prog);
-		else
+		else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI)
 			ret = bpf_kprobe_multi_link_attach(attr, prog);
+		else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI)
+			ret = bpf_uprobe_multi_link_attach(attr, prog);
 		break;
 	default:
 		ret = -EINVAL;
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 03b7f6b8e4f0..a0b9d034300f 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -23,6 +23,7 @@
 #include <linux/sort.h>
 #include <linux/key.h>
 #include <linux/verification.h>
+#include <linux/namei.h>
 
 #include <net/bpf_sk_storage.h>
 
@@ -2922,3 +2923,239 @@ static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
 	return 0;
 }
 #endif
+
+#ifdef CONFIG_UPROBES
+struct bpf_uprobe_multi_link;
+
+struct bpf_uprobe {
+	struct bpf_uprobe_multi_link *link;
+	loff_t offset;
+	struct uprobe_consumer consumer;
+};
+
+struct bpf_uprobe_multi_link {
+	struct path path;
+	struct bpf_link link;
+	u32 cnt;
+	struct bpf_uprobe *uprobes;
+};
+
+struct bpf_uprobe_multi_run_ctx {
+	struct bpf_run_ctx run_ctx;
+	unsigned long entry_ip;
+};
+
+static void bpf_uprobe_unregister(struct path *path, struct bpf_uprobe *uprobes,
+				  u32 cnt)
+{
+	u32 i;
+
+	for (i = 0; i < cnt; i++) {
+		uprobe_unregister(d_real_inode(path->dentry), uprobes[i].offset,
+				  &uprobes[i].consumer);
+	}
+}
+
+static void bpf_uprobe_multi_link_release(struct bpf_link *link)
+{
+	struct bpf_uprobe_multi_link *umulti_link;
+
+	umulti_link = container_of(link, struct bpf_uprobe_multi_link, link);
+	bpf_uprobe_unregister(&umulti_link->path, umulti_link->uprobes, umulti_link->cnt);
+	path_put(&umulti_link->path);
+}
+
+static void bpf_uprobe_multi_link_dealloc(struct bpf_link *link)
+{
+	struct bpf_uprobe_multi_link *umulti_link;
+
+	umulti_link = container_of(link, struct bpf_uprobe_multi_link, link);
+	kvfree(umulti_link->uprobes);
+	kfree(umulti_link);
+}
+
+static const struct bpf_link_ops bpf_uprobe_multi_link_lops = {
+	.release = bpf_uprobe_multi_link_release,
+	.dealloc = bpf_uprobe_multi_link_dealloc,
+};
+
+static int uprobe_prog_run(struct bpf_uprobe *uprobe,
+			   unsigned long entry_ip,
+			   struct pt_regs *regs)
+{
+	struct bpf_uprobe_multi_link *link = uprobe->link;
+	struct bpf_uprobe_multi_run_ctx run_ctx = {
+		.entry_ip = entry_ip,
+	};
+	struct bpf_prog *prog = link->link.prog;
+	bool sleepable = prog->aux->sleepable;
+	struct bpf_run_ctx *old_run_ctx;
+	int err = 0;
+
+	might_fault();
+
+	migrate_disable();
+
+	if (sleepable)
+		rcu_read_lock_trace();
+	else
+		rcu_read_lock();
+
+	old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);
+	err = bpf_prog_run(link->link.prog, regs);
+	bpf_reset_run_ctx(old_run_ctx);
+
+	if (sleepable)
+		rcu_read_unlock_trace();
+	else
+		rcu_read_unlock();
+
+	migrate_enable();
+	return err;
+}
+
+static int
+uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs)
+{
+	struct bpf_uprobe *uprobe;
+
+	uprobe = container_of(con, struct bpf_uprobe, consumer);
+	return uprobe_prog_run(uprobe, instruction_pointer(regs), regs);
+}
+
+static int
+uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs)
+{
+	struct bpf_uprobe *uprobe;
+
+	uprobe = container_of(con, struct bpf_uprobe, consumer);
+	return uprobe_prog_run(uprobe, func, regs);
+}
+
+int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
+{
+	struct bpf_uprobe_multi_link *link = NULL;
+	unsigned long __user *uref_ctr_offsets;
+	unsigned long *ref_ctr_offsets = NULL;
+	struct bpf_link_primer link_primer;
+	struct bpf_uprobe *uprobes = NULL;
+	unsigned long __user *uoffsets;
+	void __user *upath;
+	u32 flags, cnt, i;
+	struct path path;
+	char *name;
+	int err;
+
+	/* no support for 32bit archs yet */
+	if (sizeof(u64) != sizeof(void *))
+		return -EOPNOTSUPP;
+
+	if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI)
+		return -EINVAL;
+
+	flags = attr->link_create.uprobe_multi.flags;
+	if (flags & ~BPF_F_UPROBE_MULTI_RETURN)
+		return -EINVAL;
+
+	/*
+	 * path, offsets and cnt are mandatory,
+	 * ref_ctr_offsets is optional
+	 */
+	upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
+	uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
+	cnt = attr->link_create.uprobe_multi.cnt;
+
+	if (!upath || !uoffsets || !cnt)
+		return -EINVAL;
+
+	uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
+
+	name = strndup_user(upath, PATH_MAX);
+	if (IS_ERR(name)) {
+		err = PTR_ERR(name);
+		return err;
+	}
+
+	err = kern_path(name, LOOKUP_FOLLOW, &path);
+	kfree(name);
+	if (err)
+		return err;
+
+	if (!d_is_reg(path.dentry)) {
+		err = -EINVAL;
+		goto error_path_put;
+	}
+
+	err = -ENOMEM;
+
+	link = kzalloc(sizeof(*link), GFP_KERNEL);
+	uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL);
+
+	if (!uprobes || !link)
+		goto error_free;
+
+	if (uref_ctr_offsets) {
+		ref_ctr_offsets = kvcalloc(cnt, sizeof(*ref_ctr_offsets), GFP_KERNEL);
+		if (!ref_ctr_offsets)
+			goto error_free;
+	}
+
+	for (i = 0; i < cnt; i++) {
+		if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
+			err = -EFAULT;
+			goto error_free;
+		}
+		if (__get_user(uprobes[i].offset, uoffsets + i)) {
+			err = -EFAULT;
+			goto error_free;
+		}
+
+		uprobes[i].link = link;
+
+		if (flags & BPF_F_UPROBE_MULTI_RETURN)
+			uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
+		else
+			uprobes[i].consumer.handler = uprobe_multi_link_handler;
+	}
+
+	link->cnt = cnt;
+	link->uprobes = uprobes;
+	link->path = path;
+
+	bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI,
+		      &bpf_uprobe_multi_link_lops, prog);
+
+	err = bpf_link_prime(&link->link, &link_primer);
+	if (err)
+		goto error_free;
+
+	for (i = 0; i < cnt; i++) {
+		err = uprobe_register_refctr(d_real_inode(link->path.dentry),
+					     uprobes[i].offset,
+					     ref_ctr_offsets ? ref_ctr_offsets[i] : 0,
+					     &uprobes[i].consumer);
+		if (err) {
+			bpf_uprobe_unregister(&path, uprobes, i);
+			bpf_link_cleanup(&link_primer);
+			kvfree(ref_ctr_offsets);
+			return err;
+		}
+	}
+
+	kvfree(ref_ctr_offsets);
+	return bpf_link_settle(&link_primer);
+
+error_free:
+	kvfree(ref_ctr_offsets);
+	kvfree(uprobes);
+	kfree(link);
+error_path_put:
+	path_put(&path);
+	return err;
+}
+#else /* !CONFIG_UPROBES */
+int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
+{
+	return -EOPNOTSUPP;
+}
+#endif /* CONFIG_UPROBES */
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 60a9d59beeab..a236139f08ce 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
 	BPF_LSM_CGROUP,
 	BPF_STRUCT_OPS,
 	BPF_NETFILTER,
+	BPF_TRACE_UPROBE_MULTI,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_KPROBE_MULTI = 8,
 	BPF_LINK_TYPE_STRUCT_OPS = 9,
 	BPF_LINK_TYPE_NETFILTER = 10,
+	BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
 				__s32		priority;
 				__u32		flags;
 			} netfilter;
+			struct {
+				__u32		flags;
+				__u32		cnt;
+				__aligned_u64	path;
+				__aligned_u64	offsets;
+				__aligned_u64	ref_ctr_offsets;
+			} uprobe_multi;
 		};
 	} link_create;
 
-- 
2.41.0

[PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link

[Jiri Olsa]

Adding support to specify cookies array for uprobe_multi link.

The cookies array share indexes and length with other uprobe_multi
arrays (offsets/ref_ctr_offsets).

The cookies[i] value defines cookie for i-the uprobe and will be
returned by bpf_get_attach_cookie helper when called from ebpf
program hooked to that specific uprobe.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 44 insertions(+), 5 deletions(-)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index a236139f08ce..4103f395593b 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1592,6 +1592,7 @@ union bpf_attr {
 				__aligned_u64	path;
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
+				__aligned_u64	cookies;
 			} uprobe_multi;
 		};
 	} link_create;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 3b0582a64ce4..affad356c603 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 	return err;
 }
 
-#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
+#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
 	struct bpf_prog *prog;
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index a0b9d034300f..4657df8f884e 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -87,6 +87,8 @@ static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size,
 static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx);
+
 /**
  * trace_call_bpf - invoke BPF program
  * @call: tracepoint event
@@ -1099,6 +1101,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
 	.arg1_type	= ARG_PTR_TO_CTX,
 };
 
+BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs)
+{
+	return bpf_uprobe_multi_cookie(current->bpf_ctx);
+}
+
+static const struct bpf_func_proto bpf_get_attach_cookie_proto_umulti = {
+	.func		= bpf_get_attach_cookie_uprobe_multi,
+	.gpl_only	= false,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_PTR_TO_CTX,
+};
+
 BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx)
 {
 	struct bpf_trace_run_ctx *run_ctx;
@@ -1545,9 +1559,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 			&bpf_get_func_ip_proto_kprobe_multi :
 			&bpf_get_func_ip_proto_kprobe;
 	case BPF_FUNC_get_attach_cookie:
-		return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ?
-			&bpf_get_attach_cookie_proto_kmulti :
-			&bpf_get_attach_cookie_proto_trace;
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+			return &bpf_get_attach_cookie_proto_kmulti;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+			return &bpf_get_attach_cookie_proto_umulti;
+		return &bpf_get_attach_cookie_proto_trace;
 	default:
 		return bpf_tracing_func_proto(func_id, prog);
 	}
@@ -2930,6 +2946,7 @@ struct bpf_uprobe_multi_link;
 struct bpf_uprobe {
 	struct bpf_uprobe_multi_link *link;
 	loff_t offset;
+	u64 cookie;
 	struct uprobe_consumer consumer;
 };
 
@@ -2943,6 +2960,7 @@ struct bpf_uprobe_multi_link {
 struct bpf_uprobe_multi_run_ctx {
 	struct bpf_run_ctx run_ctx;
 	unsigned long entry_ip;
+	struct bpf_uprobe *uprobe;
 };
 
 static void bpf_uprobe_unregister(struct path *path, struct bpf_uprobe *uprobes,
@@ -2986,6 +3004,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 	struct bpf_uprobe_multi_link *link = uprobe->link;
 	struct bpf_uprobe_multi_run_ctx run_ctx = {
 		.entry_ip = entry_ip,
+		.uprobe = uprobe,
 	};
 	struct bpf_prog *prog = link->link.prog;
 	bool sleepable = prog->aux->sleepable;
@@ -3032,6 +3051,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
 	return uprobe_prog_run(uprobe, func, regs);
 }
 
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
+{
+	struct bpf_uprobe_multi_run_ctx *run_ctx;
+
+	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+	return run_ctx->uprobe->cookie;
+}
+
 int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 {
 	struct bpf_uprobe_multi_link *link = NULL;
@@ -3040,6 +3067,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	struct bpf_link_primer link_primer;
 	struct bpf_uprobe *uprobes = NULL;
 	unsigned long __user *uoffsets;
+	u64 __user *ucookies;
 	void __user *upath;
 	u32 flags, cnt, i;
 	struct path path;
@@ -3059,7 +3087,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 
 	/*
 	 * path, offsets and cnt are mandatory,
-	 * ref_ctr_offsets is optional
+	 * ref_ctr_offsets and cookies are optional
 	 */
 	upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
 	uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
@@ -3069,6 +3097,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 		return -EINVAL;
 
 	uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
+	ucookies = u64_to_user_ptr(attr->link_create.uprobe_multi.cookies);
 
 	name = strndup_user(upath, PATH_MAX);
 	if (IS_ERR(name)) {
@@ -3101,6 +3130,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	}
 
 	for (i = 0; i < cnt; i++) {
+		if (ucookies && __get_user(uprobes[i].cookie, ucookies + i)) {
+			err = -EFAULT;
+			goto error_free;
+		}
 		if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
 			err = -EFAULT;
 			goto error_free;
@@ -3158,4 +3191,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 {
 	return -EOPNOTSUPP;
 }
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
+{
+	return 0;
+}
 #endif /* CONFIG_UPROBES */
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index a236139f08ce..4103f395593b 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1592,6 +1592,7 @@ union bpf_attr {
 				__aligned_u64	path;
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
+				__aligned_u64	cookies;
 			} uprobe_multi;
 		};
 	} link_create;
-- 
2.41.0

[PATCHv3 bpf-next 04/26] bpf: Add pid filter support for uprobe_multi link

[Jiri Olsa]

Adding support to specify pid for uprobe_multi link and the uprobes
are created only for task with given pid value.

Using the consumer.filter filter callback for that, so the task gets
filtered during the uprobe installation.

We still need to check the task during runtime in the uprobe handler,
because the handler could get executed if there's another system
wide consumer on the same uprobe (thanks Oleg for the insight).

Cc: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 33 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 4103f395593b..347c5ed6286b 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1593,6 +1593,7 @@ union bpf_attr {
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
 				__aligned_u64	cookies;
+				__u32		pid;
 			} uprobe_multi;
 		};
 	} link_create;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index affad356c603..c16b9f80099d 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 	return err;
 }
 
-#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
+#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.pid
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
 	struct bpf_prog *prog;
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 4657df8f884e..4ef51fd0497f 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -2955,6 +2955,7 @@ struct bpf_uprobe_multi_link {
 	struct bpf_link link;
 	u32 cnt;
 	struct bpf_uprobe *uprobes;
+	struct task_struct *task;
 };
 
 struct bpf_uprobe_multi_run_ctx {
@@ -2981,6 +2982,8 @@ static void bpf_uprobe_multi_link_release(struct bpf_link *link)
 	umulti_link = container_of(link, struct bpf_uprobe_multi_link, link);
 	bpf_uprobe_unregister(&umulti_link->path, umulti_link->uprobes, umulti_link->cnt);
 	path_put(&umulti_link->path);
+	if (umulti_link->task)
+		put_task_struct(umulti_link->task);
 }
 
 static void bpf_uprobe_multi_link_dealloc(struct bpf_link *link)
@@ -3011,6 +3014,9 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 	struct bpf_run_ctx *old_run_ctx;
 	int err = 0;
 
+	if (link->task && current != link->task)
+		return 0;
+
 	might_fault();
 
 	migrate_disable();
@@ -3033,6 +3039,16 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 	return err;
 }
 
+static bool
+uprobe_multi_link_filter(struct uprobe_consumer *con, enum uprobe_filter_ctx ctx,
+			 struct mm_struct *mm)
+{
+	struct bpf_uprobe *uprobe;
+
+	uprobe = container_of(con, struct bpf_uprobe, consumer);
+	return uprobe->link->task->mm == mm;
+}
+
 static int
 uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs)
 {
@@ -3066,12 +3082,14 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	unsigned long *ref_ctr_offsets = NULL;
 	struct bpf_link_primer link_primer;
 	struct bpf_uprobe *uprobes = NULL;
+	struct task_struct *task = NULL;
 	unsigned long __user *uoffsets;
 	u64 __user *ucookies;
 	void __user *upath;
 	u32 flags, cnt, i;
 	struct path path;
 	char *name;
+	pid_t pid;
 	int err;
 
 	/* no support for 32bit archs yet */
@@ -3115,6 +3133,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 		goto error_path_put;
 	}
 
+	pid = attr->link_create.uprobe_multi.pid;
+	if (pid) {
+		rcu_read_lock();
+		task = get_pid_task(find_vpid(pid), PIDTYPE_PID);
+		rcu_read_unlock();
+		if (!task)
+			goto error_path_put;
+	}
+
 	err = -ENOMEM;
 
 	link = kzalloc(sizeof(*link), GFP_KERNEL);
@@ -3149,11 +3176,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 			uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
 		else
 			uprobes[i].consumer.handler = uprobe_multi_link_handler;
+
+		if (pid)
+			uprobes[i].consumer.filter = uprobe_multi_link_filter;
 	}
 
 	link->cnt = cnt;
 	link->uprobes = uprobes;
 	link->path = path;
+	link->task = task;
 
 	bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI,
 		      &bpf_uprobe_multi_link_lops, prog);
@@ -3182,6 +3213,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	kvfree(ref_ctr_offsets);
 	kvfree(uprobes);
 	kfree(link);
+	if (task)
+		put_task_struct(task);
 error_path_put:
 	path_put(&path);
 	return err;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 4103f395593b..347c5ed6286b 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1593,6 +1593,7 @@ union bpf_attr {
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
 				__aligned_u64	cookies;
+				__u32		pid;
 			} uprobe_multi;
 		};
 	} link_create;
-- 
2.41.0

[PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Jiri Olsa]

Adding support for bpf_get_func_ip helper being called from
ebpf program attached by uprobe_multi link.

It returns the ip of the uprobe.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 4ef51fd0497f..f5a41c1604b8 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
 static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx);
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
 /**
  * trace_call_bpf - invoke BPF program
@@ -1101,6 +1102,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
 	.arg1_type	= ARG_PTR_TO_CTX,
 };
 
+BPF_CALL_1(bpf_get_func_ip_uprobe_multi, struct pt_regs *, regs)
+{
+	return bpf_uprobe_multi_entry_ip(current->bpf_ctx);
+}
+
+static const struct bpf_func_proto bpf_get_func_ip_proto_uprobe_multi = {
+	.func		= bpf_get_func_ip_uprobe_multi,
+	.gpl_only	= false,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_PTR_TO_CTX,
+};
+
 BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs)
 {
 	return bpf_uprobe_multi_cookie(current->bpf_ctx);
@@ -1555,9 +1568,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_override_return_proto;
 #endif
 	case BPF_FUNC_get_func_ip:
-		return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ?
-			&bpf_get_func_ip_proto_kprobe_multi :
-			&bpf_get_func_ip_proto_kprobe;
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+			return &bpf_get_func_ip_proto_kprobe_multi;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+			return &bpf_get_func_ip_proto_uprobe_multi;
+		return &bpf_get_func_ip_proto_kprobe;
 	case BPF_FUNC_get_attach_cookie:
 		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
 			return &bpf_get_attach_cookie_proto_kmulti;
@@ -3067,6 +3082,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
 	return uprobe_prog_run(uprobe, func, regs);
 }
 
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
+{
+	struct bpf_uprobe_multi_run_ctx *run_ctx;
+
+	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+	return run_ctx->entry_ip;
+}
+
 static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
 {
 	struct bpf_uprobe_multi_run_ctx *run_ctx;
@@ -3228,4 +3251,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
 {
 	return 0;
 }
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
+{
+	return 0;
+}
 #endif /* CONFIG_UPROBES */
-- 
2.41.0

[PATCHv3 bpf-next 06/26] libbpf: Add uprobe_multi attach type and link names

[Jiri Olsa]

Adding new uprobe_multi attach type and link names,
so the functions can resolve the new values.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/libbpf.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 214f828ece6b..53dcf25c4878 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -118,6 +118,7 @@ static const char * const attach_type_name[] = {
 	[BPF_TRACE_KPROBE_MULTI]	= "trace_kprobe_multi",
 	[BPF_STRUCT_OPS]		= "struct_ops",
 	[BPF_NETFILTER]			= "netfilter",
+	[BPF_TRACE_UPROBE_MULTI]	= "trace_uprobe_multi",
 };
 
 static const char * const link_type_name[] = {
@@ -132,6 +133,7 @@ static const char * const link_type_name[] = {
 	[BPF_LINK_TYPE_KPROBE_MULTI]		= "kprobe_multi",
 	[BPF_LINK_TYPE_STRUCT_OPS]		= "struct_ops",
 	[BPF_LINK_TYPE_NETFILTER]		= "netfilter",
+	[BPF_LINK_TYPE_UPROBE_MULTI]		= "uprobe_multi",
 };
 
 static const char * const map_type_name[] = {
-- 
2.41.0

[PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Jiri Olsa]

Adding new elf object that will contain elf related functions.
There's no functional change.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h

diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build
index b8b0a6369363..2d0c282c8588 100644
--- a/tools/lib/bpf/Build
+++ b/tools/lib/bpf/Build
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
 	    netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
 	    btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-	    usdt.o zip.o
+	    usdt.o zip.o elf.o
diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
new file mode 100644
index 000000000000..2b62b4af28ce
--- /dev/null
+++ b/tools/lib/bpf/elf.c
@@ -0,0 +1,198 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+#include <libelf.h>
+#include <gelf.h>
+#include <fcntl.h>
+#include <linux/kernel.h>
+
+#include "libbpf_elf.h"
+#include "libbpf_internal.h"
+#include "str_error.h"
+
+#define STRERR_BUFSIZE  128
+
+/* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
+static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
+{
+	while ((scn = elf_nextscn(elf, scn)) != NULL) {
+		GElf_Shdr sh;
+
+		if (!gelf_getshdr(scn, &sh))
+			continue;
+		if (sh.sh_type == sh_type)
+			return scn;
+	}
+	return NULL;
+}
+
+/* Find offset of function name in the provided ELF object. "binary_path" is
+ * the path to the ELF binary represented by "elf", and only used for error
+ * reporting matters. "name" matches symbol name or name@@LIB for library
+ * functions.
+ */
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
+{
+	int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	bool is_shared_lib, is_name_qualified;
+	long ret = -ENOENT;
+	size_t name_len;
+	GElf_Ehdr ehdr;
+
+	if (!gelf_getehdr(elf, &ehdr)) {
+		pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
+		ret = -LIBBPF_ERRNO__FORMAT;
+		goto out;
+	}
+	/* for shared lib case, we do not need to calculate relative offset */
+	is_shared_lib = ehdr.e_type == ET_DYN;
+
+	name_len = strlen(name);
+	/* Does name specify "@@LIB"? */
+	is_name_qualified = strstr(name, "@@") != NULL;
+
+	/* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if
+	 * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically
+	 * linked binary may not have SHT_DYMSYM, so absence of a section should not be
+	 * reported as a warning/error.
+	 */
+	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+		size_t nr_syms, strtabidx, idx;
+		Elf_Data *symbols = NULL;
+		Elf_Scn *scn = NULL;
+		int last_bind = -1;
+		const char *sname;
+		GElf_Shdr sh;
+
+		scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
+		if (!scn) {
+			pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
+				 binary_path);
+			continue;
+		}
+		if (!gelf_getshdr(scn, &sh))
+			continue;
+		strtabidx = sh.sh_link;
+		symbols = elf_getdata(scn, 0);
+		if (!symbols) {
+			pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
+				binary_path, elf_errmsg(-1));
+			ret = -LIBBPF_ERRNO__FORMAT;
+			goto out;
+		}
+		nr_syms = symbols->d_size / sh.sh_entsize;
+
+		for (idx = 0; idx < nr_syms; idx++) {
+			int curr_bind;
+			GElf_Sym sym;
+			Elf_Scn *sym_scn;
+			GElf_Shdr sym_sh;
+
+			if (!gelf_getsym(symbols, idx, &sym))
+				continue;
+
+			if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
+				continue;
+
+			sname = elf_strptr(elf, strtabidx, sym.st_name);
+			if (!sname)
+				continue;
+
+			curr_bind = GELF_ST_BIND(sym.st_info);
+
+			/* User can specify func, func@@LIB or func@@LIB_VERSION. */
+			if (strncmp(sname, name, name_len) != 0)
+				continue;
+			/* ...but we don't want a search for "foo" to match 'foo2" also, so any
+			 * additional characters in sname should be of the form "@@LIB".
+			 */
+			if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+				continue;
+
+			if (ret >= 0) {
+				/* handle multiple matches */
+				if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
+					/* Only accept one non-weak bind. */
+					pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
+						sname, name, binary_path);
+					ret = -LIBBPF_ERRNO__FORMAT;
+					goto out;
+				} else if (curr_bind == STB_WEAK) {
+					/* already have a non-weak bind, and
+					 * this is a weak bind, so ignore.
+					 */
+					continue;
+				}
+			}
+
+			/* Transform symbol's virtual address (absolute for
+			 * binaries and relative for shared libs) into file
+			 * offset, which is what kernel is expecting for
+			 * uprobe/uretprobe attachment.
+			 * See Documentation/trace/uprobetracer.rst for more
+			 * details.
+			 * This is done by looking up symbol's containing
+			 * section's header and using it's virtual address
+			 * (sh_addr) and corresponding file offset (sh_offset)
+			 * to transform sym.st_value (virtual address) into
+			 * desired final file offset.
+			 */
+			sym_scn = elf_getscn(elf, sym.st_shndx);
+			if (!sym_scn)
+				continue;
+			if (!gelf_getshdr(sym_scn, &sym_sh))
+				continue;
+
+			ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
+			last_bind = curr_bind;
+		}
+		if (ret > 0)
+			break;
+	}
+
+	if (ret > 0) {
+		pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", name, binary_path,
+			 ret);
+	} else {
+		if (ret == 0) {
+			pr_warn("elf: '%s' is 0 in symtab for '%s': %s\n", name, binary_path,
+				is_shared_lib ? "should not be 0 in a shared library" :
+						"try using shared library path instead");
+			ret = -ENOENT;
+		} else {
+			pr_warn("elf: failed to find symbol '%s' in '%s'\n", name, binary_path);
+		}
+	}
+out:
+	return ret;
+}
+
+/* Find offset of function name in ELF object specified by path. "name" matches
+ * symbol name or name@@LIB for library functions.
+ */
+long elf_find_func_offset_from_file(const char *binary_path, const char *name)
+{
+	char errmsg[STRERR_BUFSIZE];
+	long ret = -ENOENT;
+	Elf *elf;
+	int fd;
+
+	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+	if (fd < 0) {
+		ret = -errno;
+		pr_warn("failed to open %s: %s\n", binary_path,
+			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+		return ret;
+	}
+	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+	if (!elf) {
+		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+		close(fd);
+		return -LIBBPF_ERRNO__FORMAT;
+	}
+
+	ret = elf_find_func_offset(elf, binary_path, name);
+	elf_end(elf);
+	close(fd);
+	return ret;
+}
+
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 53dcf25c4878..093add8124d8 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -54,6 +54,7 @@
 #include "hashmap.h"
 #include "bpf_gen_internal.h"
 #include "zip.h"
+#include "libbpf_elf.h"
 
 #ifndef BPF_FS_MAGIC
 #define BPF_FS_MAGIC		0xcafe4a11
@@ -10811,191 +10812,6 @@ static int perf_event_uprobe_open_legacy(const char *probe_name, bool retprobe,
 	return err;
 }
 
-/* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
-static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
-{
-	while ((scn = elf_nextscn(elf, scn)) != NULL) {
-		GElf_Shdr sh;
-
-		if (!gelf_getshdr(scn, &sh))
-			continue;
-		if (sh.sh_type == sh_type)
-			return scn;
-	}
-	return NULL;
-}
-
-/* Find offset of function name in the provided ELF object. "binary_path" is
- * the path to the ELF binary represented by "elf", and only used for error
- * reporting matters. "name" matches symbol name or name@@LIB for library
- * functions.
- */
-static long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
-{
-	int i, sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
-	bool is_shared_lib, is_name_qualified;
-	long ret = -ENOENT;
-	size_t name_len;
-	GElf_Ehdr ehdr;
-
-	if (!gelf_getehdr(elf, &ehdr)) {
-		pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
-		ret = -LIBBPF_ERRNO__FORMAT;
-		goto out;
-	}
-	/* for shared lib case, we do not need to calculate relative offset */
-	is_shared_lib = ehdr.e_type == ET_DYN;
-
-	name_len = strlen(name);
-	/* Does name specify "@@LIB"? */
-	is_name_qualified = strstr(name, "@@") != NULL;
-
-	/* Search SHT_DYNSYM, SHT_SYMTAB for symbol. This search order is used because if
-	 * a binary is stripped, it may only have SHT_DYNSYM, and a fully-statically
-	 * linked binary may not have SHT_DYMSYM, so absence of a section should not be
-	 * reported as a warning/error.
-	 */
-	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-		size_t nr_syms, strtabidx, idx;
-		Elf_Data *symbols = NULL;
-		Elf_Scn *scn = NULL;
-		int last_bind = -1;
-		const char *sname;
-		GElf_Shdr sh;
-
-		scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-		if (!scn) {
-			pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-				 binary_path);
-			continue;
-		}
-		if (!gelf_getshdr(scn, &sh))
-			continue;
-		strtabidx = sh.sh_link;
-		symbols = elf_getdata(scn, 0);
-		if (!symbols) {
-			pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-				binary_path, elf_errmsg(-1));
-			ret = -LIBBPF_ERRNO__FORMAT;
-			goto out;
-		}
-		nr_syms = symbols->d_size / sh.sh_entsize;
-
-		for (idx = 0; idx < nr_syms; idx++) {
-			int curr_bind;
-			GElf_Sym sym;
-			Elf_Scn *sym_scn;
-			GElf_Shdr sym_sh;
-
-			if (!gelf_getsym(symbols, idx, &sym))
-				continue;
-
-			if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-				continue;
-
-			sname = elf_strptr(elf, strtabidx, sym.st_name);
-			if (!sname)
-				continue;
-
-			curr_bind = GELF_ST_BIND(sym.st_info);
-
-			/* User can specify func, func@@LIB or func@@LIB_VERSION. */
-			if (strncmp(sname, name, name_len) != 0)
-				continue;
-			/* ...but we don't want a search for "foo" to match 'foo2" also, so any
-			 * additional characters in sname should be of the form "@@LIB".
-			 */
-			if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
-				continue;
-
-			if (ret >= 0) {
-				/* handle multiple matches */
-				if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
-					/* Only accept one non-weak bind. */
-					pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
-						sname, name, binary_path);
-					ret = -LIBBPF_ERRNO__FORMAT;
-					goto out;
-				} else if (curr_bind == STB_WEAK) {
-					/* already have a non-weak bind, and
-					 * this is a weak bind, so ignore.
-					 */
-					continue;
-				}
-			}
-
-			/* Transform symbol's virtual address (absolute for
-			 * binaries and relative for shared libs) into file
-			 * offset, which is what kernel is expecting for
-			 * uprobe/uretprobe attachment.
-			 * See Documentation/trace/uprobetracer.rst for more
-			 * details.
-			 * This is done by looking up symbol's containing
-			 * section's header and using it's virtual address
-			 * (sh_addr) and corresponding file offset (sh_offset)
-			 * to transform sym.st_value (virtual address) into
-			 * desired final file offset.
-			 */
-			sym_scn = elf_getscn(elf, sym.st_shndx);
-			if (!sym_scn)
-				continue;
-			if (!gelf_getshdr(sym_scn, &sym_sh))
-				continue;
-
-			ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
-			last_bind = curr_bind;
-		}
-		if (ret > 0)
-			break;
-	}
-
-	if (ret > 0) {
-		pr_debug("elf: symbol address match for '%s' in '%s': 0x%lx\n", name, binary_path,
-			 ret);
-	} else {
-		if (ret == 0) {
-			pr_warn("elf: '%s' is 0 in symtab for '%s': %s\n", name, binary_path,
-				is_shared_lib ? "should not be 0 in a shared library" :
-						"try using shared library path instead");
-			ret = -ENOENT;
-		} else {
-			pr_warn("elf: failed to find symbol '%s' in '%s'\n", name, binary_path);
-		}
-	}
-out:
-	return ret;
-}
-
-/* Find offset of function name in ELF object specified by path. "name" matches
- * symbol name or name@@LIB for library functions.
- */
-static long elf_find_func_offset_from_file(const char *binary_path, const char *name)
-{
-	char errmsg[STRERR_BUFSIZE];
-	long ret = -ENOENT;
-	Elf *elf;
-	int fd;
-
-	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		ret = -errno;
-		pr_warn("failed to open %s: %s\n", binary_path,
-			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
-		return ret;
-	}
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-		close(fd);
-		return -LIBBPF_ERRNO__FORMAT;
-	}
-
-	ret = elf_find_func_offset(elf, binary_path, name);
-	elf_end(elf);
-	close(fd);
-	return ret;
-}
-
 /* Find offset of function name in archive specified by path. Currently
  * supported are .zip files that do not compress their contents, as used on
  * Android in the form of APKs, for example. "file_name" is the name of the ELF
diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
new file mode 100644
index 000000000000..1b652220fabf
--- /dev/null
+++ b/tools/lib/bpf/libbpf_elf.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_LIBBPF_ELF_H
+#define __LIBBPF_LIBBPF_ELF_H
+
+#include <libelf.h>
+
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
+long elf_find_func_offset_from_file(const char *binary_path, const char *name);
+
+#endif /* *__LIBBPF_LIBBPF_ELF_H */
-- 
2.41.0

[PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions

[Jiri Olsa]

Adding elf_open/elf_close functions and using it in
elf_find_func_offset_from_file function. It will be
used in following changes to save some common code.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
 tools/lib/bpf/libbpf_elf.h |  8 ++++++
 tools/lib/bpf/usdt.c       | 31 ++++++--------------
 3 files changed, 56 insertions(+), 42 deletions(-)

diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
index 2b62b4af28ce..74e35071d22e 100644
--- a/tools/lib/bpf/elf.c
+++ b/tools/lib/bpf/elf.c
@@ -11,6 +11,40 @@
 
 #define STRERR_BUFSIZE  128
 
+int elf_open(const char *binary_path, struct elf_fd *elf_fd)
+{
+	char errmsg[STRERR_BUFSIZE];
+	int fd, ret;
+	Elf *elf;
+
+	if (elf_version(EV_CURRENT) == EV_NONE) {
+		pr_warn("elf: failed to init libelf for %s\n", binary_path);
+		return -LIBBPF_ERRNO__LIBELF;
+	}
+	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+	if (fd < 0) {
+		ret = -errno;
+		pr_warn("elf: failed to open %s: %s\n", binary_path,
+			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+		return ret;
+	}
+	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+	if (!elf) {
+		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+		close(fd);
+		return -LIBBPF_ERRNO__FORMAT;
+	}
+	elf_fd->fd = fd;
+	elf_fd->elf = elf;
+	return 0;
+}
+
+void elf_close(struct elf_fd *elf_fd)
+{
+	elf_end(elf_fd->elf);
+	close(elf_fd->fd);
+}
+
 /* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
 static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 {
@@ -171,28 +205,13 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
  */
 long elf_find_func_offset_from_file(const char *binary_path, const char *name)
 {
-	char errmsg[STRERR_BUFSIZE];
+	struct elf_fd elf_fd;
 	long ret = -ENOENT;
-	Elf *elf;
-	int fd;
 
-	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		ret = -errno;
-		pr_warn("failed to open %s: %s\n", binary_path,
-			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+	ret = elf_open(binary_path, &elf_fd);
+	if (ret)
 		return ret;
-	}
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-		close(fd);
-		return -LIBBPF_ERRNO__FORMAT;
-	}
-
-	ret = elf_find_func_offset(elf, binary_path, name);
-	elf_end(elf);
-	close(fd);
+	ret = elf_find_func_offset(elf_fd.elf, binary_path, name);
+	elf_close(&elf_fd);
 	return ret;
 }
-
diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
index 1b652220fabf..c763ac35a85e 100644
--- a/tools/lib/bpf/libbpf_elf.h
+++ b/tools/lib/bpf/libbpf_elf.h
@@ -5,6 +5,14 @@
 
 #include <libelf.h>
 
+struct elf_fd {
+	Elf *elf;
+	int fd;
+};
+
+int elf_open(const char *binary_path, struct elf_fd *elf_fd);
+void elf_close(struct elf_fd *elf_fd);
+
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
 long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
index f1a141555f08..9fa883ebc0bd 100644
--- a/tools/lib/bpf/usdt.c
+++ b/tools/lib/bpf/usdt.c
@@ -19,6 +19,7 @@
 #include "libbpf.h"
 #include "libbpf_common.h"
 #include "libbpf_internal.h"
+#include "libbpf_elf.h"
 #include "hashmap.h"
 
 /* libbpf's USDT support consists of BPF-side state/code and user-space
@@ -943,32 +944,22 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 					  const char *usdt_provider, const char *usdt_name,
 					  __u64 usdt_cookie)
 {
-	int i, fd, err, spec_map_fd, ip_map_fd;
+	int i, err, spec_map_fd, ip_map_fd;
 	LIBBPF_OPTS(bpf_uprobe_opts, opts);
 	struct hashmap *specs_hash = NULL;
 	struct bpf_link_usdt *link = NULL;
 	struct usdt_target *targets = NULL;
+	struct elf_fd elf_fd;
 	size_t target_cnt;
-	Elf *elf;
 
 	spec_map_fd = bpf_map__fd(man->specs_map);
 	ip_map_fd = bpf_map__fd(man->ip_to_spec_id_map);
 
-	fd = open(path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		err = -errno;
-		pr_warn("usdt: failed to open ELF binary '%s': %d\n", path, err);
+	err = elf_open(path, &elf_fd);
+	if (err)
 		return libbpf_err_ptr(err);
-	}
 
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		err = -EBADF;
-		pr_warn("usdt: failed to parse ELF binary '%s': %s\n", path, elf_errmsg(-1));
-		goto err_out;
-	}
-
-	err = sanity_check_usdt_elf(elf, path);
+	err = sanity_check_usdt_elf(elf_fd.elf, path);
 	if (err)
 		goto err_out;
 
@@ -981,7 +972,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 	/* discover USDT in given binary, optionally limiting
 	 * activations to a given PID, if pid > 0
 	 */
-	err = collect_usdt_targets(man, elf, path, pid, usdt_provider, usdt_name,
+	err = collect_usdt_targets(man, elf_fd.elf, path, pid, usdt_provider, usdt_name,
 				   usdt_cookie, &targets, &target_cnt);
 	if (err <= 0) {
 		err = (err == 0) ? -ENOENT : err;
@@ -1066,9 +1057,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 
 	free(targets);
 	hashmap__free(specs_hash);
-	elf_end(elf);
-	close(fd);
-
+	elf_close(&elf_fd);
 	return &link->link;
 
 err_out:
@@ -1076,9 +1065,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 		bpf_link__destroy(&link->link);
 	free(targets);
 	hashmap__free(specs_hash);
-	if (elf)
-		elf_end(elf);
-	close(fd);
+	elf_close(&elf_fd);
 	return libbpf_err_ptr(err);
 }
 
-- 
2.41.0

[PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

[Jiri Olsa]

Adding elf symbol iterator object (and some functions) that follow
open-coded iterator pattern and some functions to ease up iterating
elf object symbols.

The idea is to iterate single symbol section with:

  struct elf_sym_iter iter;
  struct elf_sym *sym;

  if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))
        goto error;

  while ((sym = elf_sym_iter_next(&iter))) {
        ...
  }

I considered opening the elf inside the iterator and iterate all symbol
sections, but then it gets more complicated wrt user checks for when
the next section is processed.

Plus side is the we don't need 'exit' function, because caller/user is
in charge of that.

The returned iterated symbol object from elf_sym_iter_next function
is placed inside the struct elf_sym_iter, so no extra allocation or
argument is needed.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
 1 file changed, 117 insertions(+), 61 deletions(-)

diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
index 74e35071d22e..fcce4bd2478f 100644
--- a/tools/lib/bpf/elf.c
+++ b/tools/lib/bpf/elf.c
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 	return NULL;
 }
 
+struct elf_sym {
+	const char *name;
+	GElf_Sym sym;
+	GElf_Shdr sh;
+};
+
+struct elf_sym_iter {
+	Elf *elf;
+	Elf_Data *syms;
+	size_t nr_syms;
+	size_t strtabidx;
+	size_t next_sym_idx;
+	struct elf_sym sym;
+	int st_type;
+};
+
+static int elf_sym_iter_new(struct elf_sym_iter *iter,
+			    Elf *elf, const char *binary_path,
+			    int sh_type, int st_type)
+{
+	Elf_Scn *scn = NULL;
+	GElf_Ehdr ehdr;
+	GElf_Shdr sh;
+
+	memset(iter, 0, sizeof(*iter));
+
+	if (!gelf_getehdr(elf, &ehdr)) {
+		pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
+		return -EINVAL;
+	}
+
+	scn = elf_find_next_scn_by_type(elf, sh_type, NULL);
+	if (!scn) {
+		pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
+			 binary_path);
+		return -ENOENT;
+	}
+
+	if (!gelf_getshdr(scn, &sh))
+		return -EINVAL;
+
+	iter->strtabidx = sh.sh_link;
+	iter->syms = elf_getdata(scn, 0);
+	if (!iter->syms) {
+		pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
+			binary_path, elf_errmsg(-1));
+		return -EINVAL;
+	}
+	iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
+	iter->elf = elf;
+	iter->st_type = st_type;
+	return 0;
+}
+
+static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
+{
+	struct elf_sym *ret = &iter->sym;
+	GElf_Sym *sym = &ret->sym;
+	const char *name = NULL;
+	Elf_Scn *sym_scn;
+	size_t idx;
+
+	for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
+		if (!gelf_getsym(iter->syms, idx, sym))
+			continue;
+		if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
+			continue;
+		name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
+		if (!name)
+			continue;
+
+		/* Transform symbol's virtual address (absolute for
+		 * binaries and relative for shared libs) into file
+		 * offset, which is what kernel is expecting for
+		 * uprobe/uretprobe attachment.
+		 * See Documentation/trace/uprobetracer.rst for more
+		 * details.
+		 * This is done by looking up symbol's containing
+		 * section's header and using iter's virtual address
+		 * (sh_addr) and corresponding file offset (sh_offset)
+		 * to transform sym.st_value (virtual address) into
+		 * desired final file offset.
+		 */
+		sym_scn = elf_getscn(iter->elf, sym->st_shndx);
+		if (!sym_scn)
+			continue;
+		if (!gelf_getshdr(sym_scn, &ret->sh))
+			continue;
+
+		iter->next_sym_idx = idx + 1;
+		ret->name = name;
+		return ret;
+	}
+
+	return NULL;
+}
+
+static unsigned long elf_sym_offset(struct elf_sym *sym)
+{
+	return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
+}
+
 /* Find offset of function name in the provided ELF object. "binary_path" is
  * the path to the ELF binary represented by "elf", and only used for error
  * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 	 * reported as a warning/error.
 	 */
 	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-		size_t nr_syms, strtabidx, idx;
-		Elf_Data *symbols = NULL;
-		Elf_Scn *scn = NULL;
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
 		int last_bind = -1;
-		const char *sname;
-		GElf_Shdr sh;
+		int curr_bind;
 
-		scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-		if (!scn) {
-			pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-				 binary_path);
-			continue;
-		}
-		if (!gelf_getshdr(scn, &sh))
-			continue;
-		strtabidx = sh.sh_link;
-		symbols = elf_getdata(scn, 0);
-		if (!symbols) {
-			pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-				binary_path, elf_errmsg(-1));
-			ret = -LIBBPF_ERRNO__FORMAT;
+		ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
+		if (ret) {
+			if (ret == -ENOENT)
+				continue;
 			goto out;
 		}
-		nr_syms = symbols->d_size / sh.sh_entsize;
-
-		for (idx = 0; idx < nr_syms; idx++) {
-			int curr_bind;
-			GElf_Sym sym;
-			Elf_Scn *sym_scn;
-			GElf_Shdr sym_sh;
-
-			if (!gelf_getsym(symbols, idx, &sym))
-				continue;
-
-			if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-				continue;
-
-			sname = elf_strptr(elf, strtabidx, sym.st_name);
-			if (!sname)
-				continue;
-
-			curr_bind = GELF_ST_BIND(sym.st_info);
 
+		while ((sym = elf_sym_iter_next(&iter))) {
 			/* User can specify func, func@@LIB or func@@LIB_VERSION. */
-			if (strncmp(sname, name, name_len) != 0)
+			if (strncmp(sym->name, name, name_len) != 0)
 				continue;
 			/* ...but we don't want a search for "foo" to match 'foo2" also, so any
 			 * additional characters in sname should be of the form "@@LIB".
 			 */
-			if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+			if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
 				continue;
 
-			if (ret >= 0) {
+			curr_bind = GELF_ST_BIND(sym->sym.st_info);
+
+			if (ret > 0) {
 				/* handle multiple matches */
 				if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
 					/* Only accept one non-weak bind. */
 					pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
-						sname, name, binary_path);
+						sym->name, name, binary_path);
 					ret = -LIBBPF_ERRNO__FORMAT;
 					goto out;
 				} else if (curr_bind == STB_WEAK) {
@@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 				}
 			}
 
-			/* Transform symbol's virtual address (absolute for
-			 * binaries and relative for shared libs) into file
-			 * offset, which is what kernel is expecting for
-			 * uprobe/uretprobe attachment.
-			 * See Documentation/trace/uprobetracer.rst for more
-			 * details.
-			 * This is done by looking up symbol's containing
-			 * section's header and using it's virtual address
-			 * (sh_addr) and corresponding file offset (sh_offset)
-			 * to transform sym.st_value (virtual address) into
-			 * desired final file offset.
-			 */
-			sym_scn = elf_getscn(elf, sym.st_shndx);
-			if (!sym_scn)
-				continue;
-			if (!gelf_getshdr(sym_scn, &sym_sh))
-				continue;
-
-			ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
+			ret = elf_sym_offset(sym);
 			last_bind = curr_bind;
 		}
 		if (ret > 0)
-- 
2.41.0

[PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function

[Jiri Olsa]

Adding elf_resolve_syms_offsets function that looks up
offsets for symbols specified in syms array argument.

Offsets are returned in allocated array with the 'cnt' size,
that needs to be released by the caller.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_elf.h |   2 +
 2 files changed, 107 insertions(+)

diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
index fcce4bd2478f..7e2f3b2e1fb6 100644
--- a/tools/lib/bpf/elf.c
+++ b/tools/lib/bpf/elf.c
@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
 	elf_close(&elf_fd);
 	return ret;
 }
+
+struct symbol {
+	const char *name;
+	int bind;
+	int idx;
+};
+
+static int symbol_cmp(const void *_a, const void *_b)
+{
+	const struct symbol *a = _a;
+	const struct symbol *b = _b;
+
+	return strcmp(a->name, b->name);
+}
+
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+			     const char **syms, unsigned long **poffsets)
+{
+	int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	int err = 0, i, cnt_done = 0;
+	unsigned long *offsets;
+	struct symbol *symbols;
+	struct elf_fd elf_fd;
+
+	err = elf_open(binary_path, &elf_fd);
+	if (err)
+		return err;
+
+	offsets = calloc(cnt, sizeof(*offsets));
+	symbols = calloc(cnt, sizeof(*symbols));
+
+	if (!offsets || !symbols) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	for (i = 0; i < cnt; i++) {
+		symbols[i].name = syms[i];
+		symbols[i].idx = i;
+	}
+
+	qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
+
+	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
+
+		err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+		if (err) {
+			if (err == -ENOENT)
+				continue;
+			goto out;
+		}
+
+		while ((sym = elf_sym_iter_next(&iter))) {
+			int bind = GELF_ST_BIND(sym->sym.st_info);
+			struct symbol *found, tmp = {
+				.name = sym->name,
+			};
+			unsigned long *offset;
+
+			found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
+			if (!found)
+				continue;
+
+			offset = &offsets[found->idx];
+			if (*offset > 0) {
+				/* same offset, no problem */
+				if (*offset == elf_sym_offset(sym))
+					continue;
+				/* handle multiple matches */
+				if (found->bind != STB_WEAK && bind != STB_WEAK) {
+					/* Only accept one non-weak bind. */
+					pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",
+						sym->name, found->name, binary_path);
+					err = -LIBBPF_ERRNO__FORMAT;
+					goto out;
+				} else if (bind == STB_WEAK) {
+					/* already have a non-weak bind, and
+					 * this is a weak bind, so ignore.
+					 */
+					continue;
+				}
+			} else {
+				cnt_done++;
+			}
+			*offset = elf_sym_offset(sym);
+			found->bind = bind;
+		}
+	}
+
+	if (cnt != cnt_done) {
+		err = -ENOENT;
+		goto out;
+	}
+
+	*poffsets = offsets;
+
+out:
+	free(symbols);
+	if (err)
+		free(offsets);
+	elf_close(&elf_fd);
+	return err;
+}
diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
index c763ac35a85e..026c7b378727 100644
--- a/tools/lib/bpf/libbpf_elf.h
+++ b/tools/lib/bpf/libbpf_elf.h
@@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
 long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+			     const char **syms, unsigned long **poffsets);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
-- 
2.41.0

[PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function

[Jiri Olsa]

Adding elf_resolve_pattern_offsets function that looks up
offsets for symbols specified by pattern argument.

The 'pattern' argument allows wildcards (*?' supported).

Offsets are returned in allocated array together with its
size and needs to be released by the caller.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c          |  2 +-
 tools/lib/bpf/libbpf_elf.h      |  3 ++
 tools/lib/bpf/libbpf_internal.h |  1 +
 4 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644
--- a/tools/lib/bpf/elf.c
+++ b/tools/lib/bpf/elf.c
@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
 	elf_close(&elf_fd);
 	return err;
 }
+
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+				unsigned long **poffsets, size_t *pcnt)
+{
+	int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	unsigned long *offsets = NULL;
+	size_t cap = 0, cnt = 0;
+	struct elf_fd elf_fd;
+	int err = 0, i;
+
+	err = elf_open(binary_path, &elf_fd);
+	if (err)
+		return err;
+
+	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
+
+		err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+		if (err) {
+			if (err == -ENOENT)
+				continue;
+			goto out;
+		}
+
+		while ((sym = elf_sym_iter_next(&iter))) {
+			if (!glob_match(sym->name, pattern))
+				continue;
+
+			err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
+						cnt + 1);
+			if (err)
+				goto out;
+
+			offsets[cnt++] = elf_sym_offset(sym);
+		}
+
+		/* If we found anything in the first symbol section,
+		 * do not search others to avoid duplicates.
+		 */
+		if (cnt)
+			break;
+	}
+
+	if (cnt) {
+		*poffsets = offsets;
+		*pcnt = cnt;
+	} else {
+		err = -ENOENT;
+	}
+
+out:
+	if (err)
+		free(offsets);
+	elf_close(&elf_fd);
+	return err;
+}
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 093add8124d8..f33ef7cb1adc 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog,
 }
 
 /* Adapted from perf/util/string.c */
-static bool glob_match(const char *str, const char *pat)
+bool glob_match(const char *str, const char *pat)
 {
 	while (*str && *pat && *pat != '*') {
 		if (*pat == '?') {      /* Matches any single character */
diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
index 026c7b378727..0c75d3b2398b 100644
--- a/tools/lib/bpf/libbpf_elf.h
+++ b/tools/lib/bpf/libbpf_elf.h
@@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
 int elf_resolve_syms_offsets(const char *binary_path, int cnt,
 			     const char **syms, unsigned long **poffsets);
+
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+				 unsigned long **poffsets, size_t *pcnt);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index e4d05662a96c..7d75b92e531a 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)
 #define PROG_LOAD_ATTEMPTS 5
 int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);
 
+bool glob_match(const char *str, const char *pat);
 #endif /* __LIBBPF_LIBBPF_INTERNAL_H */
-- 
2.41.0

[PATCHv3 bpf-next 12/26] libbpf: Add bpf_link_create support for multi uprobes

[Jiri Olsa]

Adding new uprobe_multi struct to bpf_link_create_opts object
to pass multiple uprobe data to link_create attr uapi.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/bpf.c | 11 +++++++++++
 tools/lib/bpf/bpf.h | 11 ++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index ed86b37d8024..0fd35c91f50c 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -733,6 +733,17 @@ int bpf_link_create(int prog_fd, int target_fd,
 		if (!OPTS_ZEROED(opts, kprobe_multi))
 			return libbpf_err(-EINVAL);
 		break;
+	case BPF_TRACE_UPROBE_MULTI:
+		attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0);
+		attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0);
+		attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0));
+		attr.link_create.uprobe_multi.offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.offsets, 0));
+		attr.link_create.uprobe_multi.ref_ctr_offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.ref_ctr_offsets, 0));
+		attr.link_create.uprobe_multi.cookies = ptr_to_u64(OPTS_GET(opts, uprobe_multi.cookies, 0));
+		attr.link_create.uprobe_multi.pid = OPTS_GET(opts, uprobe_multi.pid, 0);
+		if (!OPTS_ZEROED(opts, uprobe_multi))
+			return libbpf_err(-EINVAL);
+		break;
 	case BPF_TRACE_FENTRY:
 	case BPF_TRACE_FEXIT:
 	case BPF_MODIFY_RETURN:
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 9aa0ee473754..82979b4f2769 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -346,13 +346,22 @@ struct bpf_link_create_opts {
 			const unsigned long *addrs;
 			const __u64 *cookies;
 		} kprobe_multi;
+		struct {
+			__u32 flags;
+			__u32 cnt;
+			const char *path;
+			const unsigned long *offsets;
+			const unsigned long *ref_ctr_offsets;
+			const __u64 *cookies;
+			__u32 pid;
+		} uprobe_multi;
 		struct {
 			__u64 cookie;
 		} tracing;
 	};
 	size_t :0;
 };
-#define bpf_link_create_opts__last_field kprobe_multi.cookies
+#define bpf_link_create_opts__last_field uprobe_multi.pid
 
 LIBBPF_API int bpf_link_create(int prog_fd, int target_fd,
 			       enum bpf_attach_type attach_type,
-- 
2.41.0

[PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

[Jiri Olsa]

Adding bpf_program__attach_uprobe_multi function that
allows to attach multiple uprobes with uprobe_multi link.

The user can specify uprobes with direct arguments:

  binary_path/func_pattern/pid

or with struct bpf_uprobe_multi_opts opts argument fields:

  const char **syms;
  const unsigned long *offsets;
  const unsigned long *ref_ctr_offsets;
  const __u64 *cookies;

User can specify 2 mutually exclusive set of inputs:

 1) use only path/func_pattern/pid arguments

 2) use path/pid with allowed combinations of:
    syms/offsets/ref_ctr_offsets/cookies/cnt

    - syms and offsets are mutually exclusive
    - ref_ctr_offsets and cookies are optional

Any other usage results in error.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/libbpf.c   | 122 +++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h   |  27 +++++++++
 tools/lib/bpf/libbpf.map |   1 +
 3 files changed, 150 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index f33ef7cb1adc..b942f248038e 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)
 	return -ENOENT;
 }
 
+struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+				 pid_t pid,
+				 const char *path,
+				 const char *func_pattern,
+				 const struct bpf_uprobe_multi_opts *opts)
+{
+	const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL;
+	LIBBPF_OPTS(bpf_link_create_opts, lopts);
+	unsigned long *resolved_offsets = NULL;
+	int err = 0, link_fd, prog_fd;
+	struct bpf_link *link = NULL;
+	char errmsg[STRERR_BUFSIZE];
+	char full_path[PATH_MAX];
+	const __u64 *cookies;
+	const char **syms;
+	bool has_pattern;
+	bool retprobe;
+	size_t cnt;
+
+	if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
+		return libbpf_err_ptr(-EINVAL);
+
+	syms = OPTS_GET(opts, syms, NULL);
+	offsets = OPTS_GET(opts, offsets, NULL);
+	ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
+	cookies = OPTS_GET(opts, cookies, NULL);
+	cnt = OPTS_GET(opts, cnt, 0);
+
+	/*
+	 * User can specify 2 mutually exclusive set of inputs:
+	 *
+	 * 1) use only path/func_pattern/pid arguments
+	 *
+	 * 2) use path/pid with allowed combinations of:
+	 *    syms/offsets/ref_ctr_offsets/cookies/cnt
+	 *
+	 *    - syms and offsets are mutually exclusive
+	 *    - ref_ctr_offsets and cookies are optional
+	 *
+	 * Any other usage results in error.
+	 */
+
+	if (!path && !func_pattern && !cnt)
+		return libbpf_err_ptr(-EINVAL);
+	if (func_pattern && !path)
+		return libbpf_err_ptr(-EINVAL);
+
+	has_pattern = path && func_pattern;
+
+	if (has_pattern) {
+		if (syms || offsets || ref_ctr_offsets || cookies || cnt)
+			return libbpf_err_ptr(-EINVAL);
+	} else {
+		if (!cnt)
+			return libbpf_err_ptr(-EINVAL);
+		if (!!syms == !!offsets)
+			return libbpf_err_ptr(-EINVAL);
+	}
+
+	if (has_pattern) {
+		if (!strchr(path, '/')) {
+			err = resolve_full_path(path, full_path, sizeof(full_path));
+			if (err) {
+				pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+					prog->name, path, err);
+				return libbpf_err_ptr(err);
+			}
+			path = full_path;
+		}
+
+		err = elf_resolve_pattern_offsets(path, func_pattern,
+						  &resolved_offsets, &cnt);
+		if (err < 0)
+			return libbpf_err_ptr(err);
+		offsets = resolved_offsets;
+	} else if (syms) {
+		err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
+		if (err < 0)
+			return libbpf_err_ptr(err);
+		offsets = resolved_offsets;
+	}
+
+	retprobe = OPTS_GET(opts, retprobe, false);
+
+	lopts.uprobe_multi.path = path;
+	lopts.uprobe_multi.offsets = offsets;
+	lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
+	lopts.uprobe_multi.cookies = cookies;
+	lopts.uprobe_multi.cnt = cnt;
+	lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;
+
+	if (pid == 0)
+		pid = getpid();
+	if (pid > 0)
+		lopts.uprobe_multi.pid = pid;
+
+	link = calloc(1, sizeof(*link));
+	if (!link) {
+		err = -ENOMEM;
+		goto error;
+	}
+	link->detach = &bpf_link__detach_fd;
+
+	prog_fd = bpf_program__fd(prog);
+	link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+	if (link_fd < 0) {
+		err = -errno;
+		pr_warn("prog '%s': failed to attach: %s\n",
+			prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+		goto error;
+	}
+	link->fd = link_fd;
+	free(resolved_offsets);
+	return link;
+
+error:
+	free(resolved_offsets);
+	free(link);
+	return libbpf_err_ptr(err);
+}
+
 LIBBPF_API struct bpf_link *
 bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 				const char *binary_path, size_t func_offset,
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 754da73c643b..7c218f610210 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
 				      const char *pattern,
 				      const struct bpf_kprobe_multi_opts *opts);
 
+struct bpf_uprobe_multi_opts {
+	/* size of this struct, for forward/backward compatibility */
+	size_t sz;
+	/* array of function symbols to attach */
+	const char **syms;
+	/* array of function addresses to attach */
+	const unsigned long *offsets;
+	/* array of refctr offsets to attach */
+	const unsigned long *ref_ctr_offsets;
+	/* array of user-provided values fetchable through bpf_get_attach_cookie */
+	const __u64 *cookies;
+	/* number of elements in syms/addrs/cookies arrays */
+	size_t cnt;
+	/* create return uprobes */
+	bool retprobe;
+	size_t :0;
+};
+
+#define bpf_uprobe_multi_opts__last_field retprobe
+
+LIBBPF_API struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+				 pid_t pid,
+				 const char *binary_path,
+				 const char *func_pattern,
+				 const struct bpf_uprobe_multi_opts *opts);
+
 struct bpf_ksyscall_opts {
 	/* size of this struct, for forward/backward compatibility */
 	size_t sz;
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 7521a2fb7626..d8d11ea6c35e 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -395,4 +395,5 @@ LIBBPF_1.2.0 {
 LIBBPF_1.3.0 {
 	global:
 		bpf_obj_pin_opts;
+		bpf_program__attach_uprobe_multi;
 } LIBBPF_1.2.0;
-- 
2.41.0

[PATCHv3 bpf-next 14/26] libbpf: Add support for u[ret]probe.multi[.s] program sections

[Jiri Olsa]

Adding support for several uprobe_multi program sections
to allow auto attach of multi_uprobe programs.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index b942f248038e..06092b9752f1 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 
@@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("uretprobe.s+",		KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
 	SEC_DEF("kprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
 	SEC_DEF("kretprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+	SEC_DEF("uprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+	SEC_DEF("uretprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+	SEC_DEF("uprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
+	SEC_DEF("uretprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("ksyscall+",		KPROBE,	0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("usdt+",		KPROBE,	0, SEC_NONE, attach_usdt),
@@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
 	return libbpf_get_error(*link);
 }
 
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
+{
+	char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	int n, ret = -EINVAL;
+
+	*link = NULL;
+
+	n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%ms",
+		   &probe_type, &binary_path, &func_name);
+	switch (n) {
+	case 1:
+		/* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */
+		ret = 0;
+		break;
+	case 3:
+		opts.retprobe = strcmp(probe_type, "uretprobe.multi") == 0;
+		*link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
+		ret = libbpf_get_error(*link);
+		break;
+	default:
+		pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name,
+			prog->sec_name);
+		break;
+	}
+	free(probe_type);
+	free(binary_path);
+	free(func_name);
+	return ret;
+}
+
 static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz,
 					 const char *binary_path, uint64_t offset)
 {
-- 
2.41.0

[PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection

[Jiri Olsa]

Adding uprobe-multi link detection. It will be used later in
bpf_program__attach_usdt function to check and use uprobe_multi
link over standard uprobe links.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_internal.h |  2 ++
 2 files changed, 37 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 06092b9752f1..4f61f9dc1748 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
 	return link_fd < 0 && err == -EBADF;
 }
 
+static int probe_uprobe_multi_link(void)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
+		.expected_attach_type = BPF_TRACE_UPROBE_MULTI,
+	);
+	LIBBPF_OPTS(bpf_link_create_opts, link_opts);
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	unsigned long offset = 0;
+	int prog_fd, link_fd;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
+				insns, ARRAY_SIZE(insns), &load_opts);
+	if (prog_fd < 0)
+		return -errno;
+
+	/* create single uprobe on offset 0 in current process */
+	link_opts.uprobe_multi.path = "/proc/self/exe";
+	link_opts.uprobe_multi.offsets = &offset;
+	link_opts.uprobe_multi.cnt = 1;
+
+	link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
+
+	if (link_fd >= 0)
+		close(link_fd);
+	close(prog_fd);
+
+	return link_fd >= 0;
+}
+
 static int probe_kern_bpf_cookie(void)
 {
 	struct bpf_insn insns[] = {
@@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
 	[FEAT_SYSCALL_WRAPPER] = {
 		"Kernel using syscall wrapper", probe_kern_syscall_wrapper,
 	},
+	[FEAT_UPROBE_MULTI_LINK] = {
+		"BPF uprobe multi link support", probe_uprobe_multi_link,
+	},
 };
 
 bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 7d75b92e531a..9c04b3fe1207 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -354,6 +354,8 @@ enum kern_feature_id {
 	FEAT_BTF_ENUM64,
 	/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
 	FEAT_SYSCALL_WRAPPER,
+	/* BPF uprobe_multi link support */
+	FEAT_UPROBE_MULTI_LINK,
 	__FEAT_CNT,
 };
 
-- 
2.41.0

[PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt

[Jiri Olsa]

Adding support for usdt_manager_attach_usdt to use uprobe_multi
link to attach to usdt probes.

The uprobe_multi support is detected before the usdt program is
loaded and its expected_attach_type is set accordingly.

If uprobe_multi support is detected the usdt_manager_attach_usdt
gathers uprobes info and calls bpf_program__attach_uprobe to
create all needed uprobes.

If uprobe_multi support is not detected the old behaviour stays.

Also adding usdt.s program section for sleepable usdt probes.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/libbpf.c | 13 +++++--
 tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
 2 files changed, 75 insertions(+), 16 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 4f61f9dc1748..e234c2e860f9 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -365,6 +365,8 @@ enum sec_def_flags {
 	SEC_SLEEPABLE = 8,
 	/* BPF program support non-linear XDP buffer */
 	SEC_XDP_FRAGS = 16,
+	/* Setup proper attach type for usdt probes. */
+	SEC_USDT = 32,
 };
 
 struct bpf_sec_def {
@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
 	if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
 		opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
 
+	/* special check for usdt to use uprobe_multi link */
+	if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))
+		prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
+
 	if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
 		int btf_obj_fd = 0, btf_type_id = 0, err;
 		const char *attach_name;
@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 	if (!insns || !insns_cnt)
 		return -EINVAL;
 
-	load_attr.expected_attach_type = prog->expected_attach_type;
 	if (kernel_supports(obj, FEAT_PROG_NAME))
 		prog_name = prog->name;
 	load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 		insns_cnt = prog->insns_cnt;
 	}
 
+	/* allow prog_prepare_load_fn to change expected_attach_type */
+	load_attr.expected_attach_type = prog->expected_attach_type;
+
 	if (obj->gen_loader) {
 		bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
 				   license, insns, insns_cnt, &load_attr,
@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("uretprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("ksyscall+",		KPROBE,	0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
-	SEC_DEF("usdt+",		KPROBE,	0, SEC_NONE, attach_usdt),
+	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
+	SEC_DEF("usdt.s+",		KPROBE,	0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),
 	SEC_DEF("tc",			SCHED_CLS, 0, SEC_NONE),
 	SEC_DEF("classifier",		SCHED_CLS, 0, SEC_NONE),
 	SEC_DEF("action",		SCHED_ACT, 0, SEC_NONE),
diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
index 9fa883ebc0bd..6ff66a8eaf85 100644
--- a/tools/lib/bpf/usdt.c
+++ b/tools/lib/bpf/usdt.c
@@ -809,6 +809,8 @@ struct bpf_link_usdt {
 		long abs_ip;
 		struct bpf_link *link;
 	} *uprobes;
+
+	struct bpf_link *multi_link;
 };
 
 static int bpf_link_usdt_detach(struct bpf_link *link)
@@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)
 	struct usdt_manager *man = usdt_link->usdt_man;
 	int i;
 
+	/* When having multi_link, uprobe_cnt is 0 */
+	bpf_link__destroy(usdt_link->multi_link);
+
 	for (i = 0; i < usdt_link->uprobe_cnt; i++) {
 		/* detach underlying uprobe link */
 		bpf_link__destroy(usdt_link->uprobes[i].link);
@@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 					  const char *usdt_provider, const char *usdt_name,
 					  __u64 usdt_cookie)
 {
+	unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;
 	int i, err, spec_map_fd, ip_map_fd;
 	LIBBPF_OPTS(bpf_uprobe_opts, opts);
 	struct hashmap *specs_hash = NULL;
 	struct bpf_link_usdt *link = NULL;
 	struct usdt_target *targets = NULL;
+	__u64 *cookies = NULL;
 	struct elf_fd elf_fd;
 	size_t target_cnt;
 
@@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 	link->link.detach = &bpf_link_usdt_detach;
 	link->link.dealloc = &bpf_link_usdt_dealloc;
 
-	link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
-	if (!link->uprobes) {
-		err = -ENOMEM;
-		goto err_out;
+	if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
+		offsets = calloc(target_cnt, sizeof(*offsets));
+		cookies = calloc(target_cnt, sizeof(*cookies));
+		ref_ctr_offsets = calloc(target_cnt, sizeof(*ref_ctr_offsets));
+
+		if (!offsets || !ref_ctr_offsets || !cookies) {
+			err = -ENOMEM;
+			goto err_out;
+		}
+	} else {
+		link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
+		if (!link->uprobes) {
+			err = -ENOMEM;
+			goto err_out;
+		}
 	}
 
 	for (i = 0; i < target_cnt; i++) {
@@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 			goto err_out;
 		}
 
-		opts.ref_ctr_offset = target->sema_off;
-		opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
-		uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
-							      target->rel_ip, &opts);
-		err = libbpf_get_error(uprobe_link);
+		if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
+			offsets[i] = target->rel_ip;
+			ref_ctr_offsets[i] = target->sema_off;
+			cookies[i] = spec_id;
+		} else {
+			opts.ref_ctr_offset = target->sema_off;
+			opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
+			uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
+								      target->rel_ip, &opts);
+			err = libbpf_get_error(uprobe_link);
+			if (err) {
+				pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
+					i, usdt_provider, usdt_name, path, err);
+				goto err_out;
+			}
+
+			link->uprobes[i].link = uprobe_link;
+			link->uprobes[i].abs_ip = target->abs_ip;
+			link->uprobe_cnt++;
+		}
+	}
+
+	if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
+		LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi,
+			.cnt = target_cnt,
+			.offsets = offsets,
+			.ref_ctr_offsets = ref_ctr_offsets,
+			.cookies = cookies,
+		);
+
+		link->multi_link = bpf_program__attach_uprobe_multi(prog, pid, path,
+								    NULL, &opts_multi);
+		err = libbpf_get_error(link->multi_link);
 		if (err) {
-			pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
-				i, usdt_provider, usdt_name, path, err);
+			pr_warn("usdt: failed to attach uprobe multi for '%s:%s' in '%s': %d\n",
+				usdt_provider, usdt_name, path, err);
 			goto err_out;
 		}
 
-		link->uprobes[i].link = uprobe_link;
-		link->uprobes[i].abs_ip = target->abs_ip;
-		link->uprobe_cnt++;
+		free(offsets);
+		free(ref_ctr_offsets);
+		free(cookies);
 	}
 
 	free(targets);
@@ -1061,6 +1107,10 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 	return &link->link;
 
 err_out:
+	free(offsets);
+	free(ref_ctr_offsets);
+	free(cookies);
+
 	if (link)
 		bpf_link__destroy(&link->link);
 	free(targets);
-- 
2.41.0

[PATCHv3 bpf-next 17/26] selftests/bpf: Add uprobe_multi skel test

[Jiri Olsa]

Adding uprobe_multi test for skeleton load/attach functions,
to test skeleton auto attach for uprobe_multi link.

Test that bpf_get_func_ip works properly for uprobe_multi
attachment.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 76 ++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        | 91 +++++++++++++++++++
 2 files changed, 167 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
new file mode 100644
index 000000000000..5cd1116bbb62
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <unistd.h>
+#include <test_progs.h>
+#include "uprobe_multi.skel.h"
+
+static char test_data[] = "test_data";
+
+noinline void uprobe_multi_func_1(void)
+{
+	asm volatile ("");
+}
+
+noinline void uprobe_multi_func_2(void)
+{
+	asm volatile ("");
+}
+
+noinline void uprobe_multi_func_3(void)
+{
+	asm volatile ("");
+}
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel)
+{
+	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
+	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
+	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
+
+	skel->bss->user_ptr = test_data;
+	skel->bss->pid = getpid();
+
+	/* trigger all probes */
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	/*
+	 * There are 2 entry and 2 exit probe called for each uprobe_multi_func_[123]
+	 * function and each slepable probe (6) increments uprobe_multi_sleep_result.
+	 */
+	ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 2, "uprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 2, "uprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 2, "uprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 2, "uretprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 2, "uretprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 2, "uretprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 6, "uprobe_multi_sleep_result");
+}
+
+static void test_skel_api(void)
+{
+	struct uprobe_multi *skel = NULL;
+	int err;
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	err = uprobe_multi__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi__attach"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	uprobe_multi__destroy(skel);
+}
+
+void test_uprobe_multi_test(void)
+{
+	if (test__start_subtest("skel_api"))
+		test_skel_api();
+}
diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
new file mode 100644
index 000000000000..1eeb9b7b9cad
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
@@ -0,0 +1,91 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <stdbool.h>
+
+char _license[] SEC("license") = "GPL";
+
+__u64 uprobe_multi_func_1_addr = 0;
+__u64 uprobe_multi_func_2_addr = 0;
+__u64 uprobe_multi_func_3_addr = 0;
+
+__u64 uprobe_multi_func_1_result = 0;
+__u64 uprobe_multi_func_2_result = 0;
+__u64 uprobe_multi_func_3_result = 0;
+
+__u64 uretprobe_multi_func_1_result = 0;
+__u64 uretprobe_multi_func_2_result = 0;
+__u64 uretprobe_multi_func_3_result = 0;
+
+__u64 uprobe_multi_sleep_result = 0;
+
+int pid = 0;
+bool test_cookie = false;
+void *user_ptr = 0;
+
+static __always_inline bool verify_sleepable_user_copy(void)
+{
+	char data[9];
+
+	bpf_copy_from_user(data, sizeof(data), user_ptr);
+	return bpf_strncmp(data, sizeof(data), "test_data") == 0;
+}
+
+static void uprobe_multi_check(void *ctx, bool is_return, bool is_sleep)
+{
+	if (bpf_get_current_pid_tgid() >> 32 != pid)
+		return;
+
+	__u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0;
+	__u64 addr = bpf_get_func_ip(ctx);
+
+#define SET(__var, __addr, __cookie) ({			\
+	if (addr == __addr &&				\
+	   (!test_cookie || (cookie == __cookie)))	\
+		__var += 1;				\
+})
+
+	if (is_return) {
+		SET(uretprobe_multi_func_1_result, uprobe_multi_func_1_addr, 2);
+		SET(uretprobe_multi_func_2_result, uprobe_multi_func_2_addr, 3);
+		SET(uretprobe_multi_func_3_result, uprobe_multi_func_3_addr, 1);
+	} else {
+		SET(uprobe_multi_func_1_result, uprobe_multi_func_1_addr, 3);
+		SET(uprobe_multi_func_2_result, uprobe_multi_func_2_addr, 1);
+		SET(uprobe_multi_func_3_result, uprobe_multi_func_3_addr, 2);
+	}
+
+#undef SET
+
+	if (is_sleep && verify_sleepable_user_copy())
+		uprobe_multi_sleep_result += 1;
+}
+
+SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*")
+int test_uprobe(struct pt_regs *ctx)
+{
+	uprobe_multi_check(ctx, false, false);
+	return 0;
+}
+
+SEC("uretprobe.multi//proc/self/exe:uprobe_multi_func_*")
+int test_uretprobe(struct pt_regs *ctx)
+{
+	uprobe_multi_check(ctx, true, false);
+	return 0;
+}
+
+SEC("uprobe.multi.s//proc/self/exe:uprobe_multi_func_*")
+int test_uprobe_sleep(struct pt_regs *ctx)
+{
+	uprobe_multi_check(ctx, false, true);
+	return 0;
+}
+
+SEC("uretprobe.multi.s//proc/self/exe:uprobe_multi_func_*")
+int test_uretprobe_sleep(struct pt_regs *ctx)
+{
+	uprobe_multi_check(ctx, true, true);
+	return 0;
+}
-- 
2.41.0

[PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test

[Jiri Olsa]

Adding uprobe_multi test for bpf_program__attach_uprobe_multi
attach function.

Testing attachment using glob patterns and via bpf_uprobe_multi_opts
paths/syms fields.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
 1 file changed, 71 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index 5cd1116bbb62..f97a68871e73 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -69,8 +69,79 @@ static void test_skel_api(void)
 	uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+	struct bpf_link *link1 = NULL, *link2 = NULL;
+	struct bpf_link *link3 = NULL, *link4 = NULL;
+	struct uprobe_multi *skel = NULL;
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	opts->retprobe = false;
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	opts->retprobe = true;
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	opts->retprobe = false;
+	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	opts->retprobe = true;
+	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	bpf_link__destroy(link4);
+	bpf_link__destroy(link3);
+	bpf_link__destroy(link2);
+	bpf_link__destroy(link1);
+	uprobe_multi__destroy(skel);
+}
+
+static void test_attach_api_pattern(void)
+{
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+
+	test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
+	test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
+}
+
+static void test_attach_api_syms(void)
+{
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+
+	opts.syms = syms;
+	opts.cnt = ARRAY_SIZE(syms);
+	test_attach_api("/proc/self/exe", NULL, &opts);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
 		test_skel_api();
+	if (test__start_subtest("attach_api_pattern"))
+		test_attach_api_pattern();
+	if (test__start_subtest("attach_api_syms"))
+		test_attach_api_syms();
 }
-- 
2.41.0

[PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test

[Jiri Olsa]

Adding uprobe_multi test for bpf_link_create attach function.

Testing attachment using the struct bpf_link_create_opts.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index f97a68871e73..fd858636b8b0 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -3,6 +3,7 @@
 #include <unistd.h>
 #include <test_progs.h>
 #include "uprobe_multi.skel.h"
+#include "bpf/libbpf_elf.h"
 
 static char test_data[] = "test_data";
 
@@ -136,6 +137,71 @@ static void test_attach_api_syms(void)
 	test_attach_api("/proc/self/exe", NULL, &opts);
 }
 
+static void test_link_api(void)
+{
+	int prog_fd, link1_fd = -1, link2_fd = -1, link3_fd = -1, link4_fd = -1;
+	LIBBPF_OPTS(bpf_link_create_opts, opts);
+	const char *path = "/proc/self/exe";
+	struct uprobe_multi *skel = NULL;
+	unsigned long *offsets = NULL;
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+	int err;
+
+	err = elf_resolve_syms_offsets(path, 3, syms, (unsigned long **) &offsets);
+	if (!ASSERT_OK(err, "elf_resolve_syms_offsets"))
+		return;
+
+	opts.uprobe_multi.path = path;
+	opts.uprobe_multi.offsets = offsets;
+	opts.uprobe_multi.cnt = ARRAY_SIZE(syms);
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe);
+	link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link1_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+	prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
+	link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
+	link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link3_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+	prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
+	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
+		goto cleanup;
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	if (link1_fd >= 0)
+		close(link1_fd);
+	if (link2_fd >= 0)
+		close(link2_fd);
+	if (link3_fd >= 0)
+		close(link3_fd);
+	if (link4_fd >= 0)
+		close(link4_fd);
+
+	uprobe_multi__destroy(skel);
+	free(offsets);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
 		test_attach_api_pattern();
 	if (test__start_subtest("attach_api_syms"))
 		test_attach_api_syms();
+	if (test__start_subtest("link_api"))
+		test_link_api();
 }
-- 
2.41.0

[PATCHv3 bpf-next 20/26] selftests/bpf: Add uprobe_multi test program

[Jiri Olsa]

Adding uprobe_multi test program that defines 50k uprobe_multi_func_*
functions and will serve as attach point for uprobe_multi bench test
in following patch.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/testing/selftests/bpf/Makefile       |  5 ++
 tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index ad6b585e0d7c..acf7c9a29082 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko	\
 		       $(OUTPUT)/liburandom_read.so			\
 		       $(OUTPUT)/xdp_synproxy				\
 		       $(OUTPUT)/sign-file				\
+		       $(OUTPUT)/uprobe_multi				\
 		       ima_setup.sh 					\
 		       verify_sig_setup.sh				\
 		       $(wildcard progs/btf_dump_test_case_*.c)		\
@@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o
 	$(call msg,BINARY,,$@)
 	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
 
+$(OUTPUT)/uprobe_multi: uprobe_multi.c
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
diff --git a/tools/testing/selftests/bpf/uprobe_multi.c b/tools/testing/selftests/bpf/uprobe_multi.c
new file mode 100644
index 000000000000..115a7f6cebfa
--- /dev/null
+++ b/tools/testing/selftests/bpf/uprobe_multi.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <stdio.h>
+
+#define __PASTE(a, b) a##b
+#define PASTE(a, b) __PASTE(a, b)
+
+#define NAME(name, idx) PASTE(name, idx)
+
+#define DEF(name, idx)  int NAME(name, idx)(void) { return 0; }
+#define CALL(name, idx) NAME(name, idx)();
+
+#define F(body, name, idx) body(name, idx)
+
+#define F10(body, name, idx) \
+	F(body, PASTE(name, idx), 0) F(body, PASTE(name, idx), 1) F(body, PASTE(name, idx), 2) \
+	F(body, PASTE(name, idx), 3) F(body, PASTE(name, idx), 4) F(body, PASTE(name, idx), 5) \
+	F(body, PASTE(name, idx), 6) F(body, PASTE(name, idx), 7) F(body, PASTE(name, idx), 8) \
+	F(body, PASTE(name, idx), 9)
+
+#define F100(body, name, idx) \
+	F10(body, PASTE(name, idx), 0) F10(body, PASTE(name, idx), 1) F10(body, PASTE(name, idx), 2) \
+	F10(body, PASTE(name, idx), 3) F10(body, PASTE(name, idx), 4) F10(body, PASTE(name, idx), 5) \
+	F10(body, PASTE(name, idx), 6) F10(body, PASTE(name, idx), 7) F10(body, PASTE(name, idx), 8) \
+	F10(body, PASTE(name, idx), 9)
+
+#define F1000(body, name, idx) \
+	F100(body, PASTE(name, idx), 0) F100(body, PASTE(name, idx), 1) F100(body, PASTE(name, idx), 2) \
+	F100(body, PASTE(name, idx), 3) F100(body, PASTE(name, idx), 4) F100(body, PASTE(name, idx), 5) \
+	F100(body, PASTE(name, idx), 6) F100(body, PASTE(name, idx), 7) F100(body, PASTE(name, idx), 8) \
+	F100(body, PASTE(name, idx), 9)
+
+#define F10000(body, name, idx) \
+	F1000(body, PASTE(name, idx), 0) F1000(body, PASTE(name, idx), 1) F1000(body, PASTE(name, idx), 2) \
+	F1000(body, PASTE(name, idx), 3) F1000(body, PASTE(name, idx), 4) F1000(body, PASTE(name, idx), 5) \
+	F1000(body, PASTE(name, idx), 6) F1000(body, PASTE(name, idx), 7) F1000(body, PASTE(name, idx), 8) \
+	F1000(body, PASTE(name, idx), 9)
+
+F10000(DEF, uprobe_multi_func_, 0)
+F10000(DEF, uprobe_multi_func_, 1)
+F10000(DEF, uprobe_multi_func_, 2)
+F10000(DEF, uprobe_multi_func_, 3)
+F10000(DEF, uprobe_multi_func_, 4)
+
+int main(void)
+{
+	F10000(CALL, uprobe_multi_func_, 0)
+	F10000(CALL, uprobe_multi_func_, 1)
+	F10000(CALL, uprobe_multi_func_, 2)
+	F10000(CALL, uprobe_multi_func_, 3)
+	F10000(CALL, uprobe_multi_func_, 4)
+	return 0;
+}
-- 
2.41.0

[PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test

[Jiri Olsa]

Adding test that attaches 50k uprobes in uprobe_multi binary.

After the attach is done we run the binary and make sure we
get proper amount of hits.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
 2 files changed, 65 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index fd858636b8b0..547d46965d70 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -202,6 +202,60 @@ static void test_link_api(void)
 	free(offsets);
 }
 
+static inline __u64 get_time_ns(void)
+{
+	struct timespec t;
+
+	clock_gettime(CLOCK_MONOTONIC, &t);
+	return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
+}
+
+static void test_bench_attach_uprobe(void)
+{
+	long attach_start_ns, attach_end_ns;
+	long detach_start_ns, detach_end_ns;
+	double attach_delta, detach_delta;
+	struct uprobe_multi *skel = NULL;
+	struct bpf_program *prog;
+	int err;
+
+	skel = uprobe_multi__open();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+		goto cleanup;
+
+	bpf_object__for_each_program(prog, skel->obj)
+		bpf_program__set_autoload(prog, false);
+
+	bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);
+
+	err = uprobe_multi__load(skel);
+	if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
+		goto cleanup;
+
+	attach_start_ns = get_time_ns();
+
+	err = uprobe_multi__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi__attach"))
+		goto cleanup;
+
+	attach_end_ns = get_time_ns();
+
+	system("./uprobe_multi");
+
+	ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
+
+cleanup:
+	detach_start_ns = get_time_ns();
+	uprobe_multi__destroy(skel);
+	detach_end_ns = get_time_ns();
+
+	attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+	detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+	printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
 		test_attach_api_syms();
 	if (test__start_subtest("link_api"))
 		test_link_api();
+	if (test__start_subtest("bench_uprobe"))
+		test_bench_attach_uprobe();
 }
diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
index 1eeb9b7b9cad..cd73139dc881 100644
--- a/tools/testing/selftests/bpf/progs/uprobe_multi.c
+++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
 	uprobe_multi_check(ctx, true, true);
 	return 0;
 }
+
+int count;
+
+SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
+int test_uprobe_bench(struct pt_regs *ctx)
+{
+	count++;
+	return 0;
+}
-- 
2.41.0

[PATCHv3 bpf-next 22/26] selftests/bpf: Add usdt_multi test program

[Jiri Olsa]

Adding usdt_multi test program that defines 50k usdts and will
serve as attach point for uprobe_multi usdt bench test in
following patch.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/testing/selftests/bpf/Makefile     |  5 +++++
 tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/usdt_multi.c

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index acf7c9a29082..9762467cf0ba 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko	\
 		       $(OUTPUT)/xdp_synproxy				\
 		       $(OUTPUT)/sign-file				\
 		       $(OUTPUT)/uprobe_multi				\
+		       $(OUTPUT)/usdt_multi				\
 		       ima_setup.sh 					\
 		       verify_sig_setup.sh				\
 		       $(wildcard progs/btf_dump_test_case_*.c)		\
@@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c
 	$(call msg,BINARY,,$@)
 	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
 
+$(OUTPUT)/usdt_multi: usdt_multi.c
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
diff --git a/tools/testing/selftests/bpf/usdt_multi.c b/tools/testing/selftests/bpf/usdt_multi.c
new file mode 100644
index 000000000000..fedf856bad2b
--- /dev/null
+++ b/tools/testing/selftests/bpf/usdt_multi.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <sdt.h>
+
+#define PROBE STAP_PROBE(test, usdt);
+
+#define PROBE10    PROBE PROBE PROBE PROBE PROBE \
+		   PROBE PROBE PROBE PROBE PROBE
+#define PROBE100   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \
+		   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10
+#define PROBE1000  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \
+		   PROBE100 PROBE100 PROBE100 PROBE100 PROBE100
+#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \
+		   PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000
+
+int main(void)
+{
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	return 0;
+}
-- 
2.41.0

[PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test

[Jiri Olsa]

Adding test that attaches 50k usdt probes in usdt_multi binary.

After the attach is done we run the binary and make sure we get
proper amount of hits.

With current uprobes:

  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

   Performance counter stats for './test_progs -n 254/6':

      1353.659680562 seconds time elapsed

With uprobe_multi link:

  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

   Performance counter stats for './test_progs -n 254/6':

         0.322046364 seconds time elapsed

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 50 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi_usdt.c   | 16 ++++++
 2 files changed, 66 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index 547d46965d70..b12dc1f992e5 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -4,6 +4,7 @@
 #include <test_progs.h>
 #include "uprobe_multi.skel.h"
 #include "bpf/libbpf_elf.h"
+#include "uprobe_multi_usdt.skel.h"
 
 static char test_data[] = "test_data";
 
@@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)
 	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
 }
 
+static void test_bench_attach_usdt(void)
+{
+	struct uprobe_multi_usdt *skel = NULL;
+	long attach_start_ns, attach_end_ns;
+	long detach_start_ns, detach_end_ns;
+	double attach_delta, detach_delta;
+	struct bpf_program *prog;
+	int err;
+
+	skel = uprobe_multi_usdt__open();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+		goto cleanup;
+
+	bpf_object__for_each_program(prog, skel->obj)
+		bpf_program__set_autoload(prog, false);
+
+	bpf_program__set_autoload(skel->progs.usdt0, true);
+
+	err = uprobe_multi_usdt__load(skel);
+	if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
+		goto cleanup;
+
+	attach_start_ns = get_time_ns();
+
+	skel->links.usdt0 = bpf_program__attach_usdt(skel->progs.usdt0, -1, "./usdt_multi",
+						     "test", "usdt", NULL);
+	if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
+		goto cleanup;
+
+	attach_end_ns = get_time_ns();
+
+	system("./usdt_multi");
+
+	ASSERT_EQ(skel->bss->count, 50000, "usdt_count");
+
+cleanup:
+	detach_start_ns = get_time_ns();
+	uprobe_multi_usdt__destroy(skel);
+	detach_end_ns = get_time_ns();
+
+	attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+	detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+	printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)
 		test_link_api();
 	if (test__start_subtest("bench_uprobe"))
 		test_bench_attach_uprobe();
+	if (test__start_subtest("bench_usdt"))
+		test_bench_attach_usdt();
 }
diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
new file mode 100644
index 000000000000..9e1c33d0bd2f
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/usdt.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+int count;
+
+SEC("usdt")
+int usdt0(struct pt_regs *ctx)
+{
+	count++;
+	return 0;
+}
-- 
2.41.0

[PATCHv3 bpf-next 24/26] selftests/bpf: Add uprobe_multi cookie test

[Jiri Olsa]

Adding test for cookies setup/retrieval in uprobe_link uprobes
and making sure bpf_get_attach_cookie works properly.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../selftests/bpf/prog_tests/bpf_cookie.c     | 78 +++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c
index 26b2d1bffdfd..4162d1724b79 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c
@@ -11,6 +11,7 @@
 #include <bpf/btf.h>
 #include "test_bpf_cookie.skel.h"
 #include "kprobe_multi.skel.h"
+#include "uprobe_multi.skel.h"
 
 /* uprobe attach point */
 static noinline void trigger_func(void)
@@ -239,6 +240,81 @@ static void kprobe_multi_attach_api_subtest(void)
 	bpf_link__destroy(link1);
 	kprobe_multi__destroy(skel);
 }
+
+/* defined in prog_tests/uprobe_multi_test.c */
+void uprobe_multi_func_1(void);
+void uprobe_multi_func_2(void);
+void uprobe_multi_func_3(void);
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel)
+{
+	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
+	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
+	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
+
+	skel->bss->pid = getpid();
+	skel->bss->test_cookie = true;
+
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 1, "uprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 1, "uprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 1, "uprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 1, "uretprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 1, "uretprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 1, "uretprobe_multi_func_3_result");
+}
+
+static void uprobe_multi_attach_api_subtest(void)
+{
+	struct bpf_link *link1 = NULL, *link2 = NULL;
+	struct uprobe_multi *skel = NULL;
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+	__u64 cookies[3];
+
+	cookies[0] = 3; /* uprobe_multi_func_1 */
+	cookies[1] = 1; /* uprobe_multi_func_2 */
+	cookies[2] = 2; /* uprobe_multi_func_3 */
+
+	opts.syms = syms;
+	opts.cnt = ARRAY_SIZE(syms);
+	opts.cookies = &cookies[0];
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi"))
+		goto cleanup;
+
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+						 "/proc/self/exe", NULL, &opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	cookies[0] = 2; /* uprobe_multi_func_1 */
+	cookies[1] = 3; /* uprobe_multi_func_2 */
+	cookies[2] = 1; /* uprobe_multi_func_3 */
+
+	opts.retprobe = true;
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+						      "/proc/self/exe", NULL, &opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	bpf_link__destroy(link2);
+	bpf_link__destroy(link1);
+	uprobe_multi__destroy(skel);
+}
+
 static void uprobe_subtest(struct test_bpf_cookie *skel)
 {
 	DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts);
@@ -515,6 +591,8 @@ void test_bpf_cookie(void)
 		kprobe_multi_attach_api_subtest();
 	if (test__start_subtest("uprobe"))
 		uprobe_subtest(skel);
+	if (test__start_subtest("multi_uprobe_attach_api"))
+		uprobe_multi_attach_api_subtest();
 	if (test__start_subtest("tracepoint"))
 		tp_subtest(skel);
 	if (test__start_subtest("perf_event"))
-- 
2.41.0

[PATCHv3 bpf-next 25/26] selftests/bpf: Add uprobe_multi pid filter tests

[Jiri Olsa]

Running api and link tests also with pid filter and checking
the probe gets executed only for specific pid.

Spawning extra process to trigger attached uprobes and checking
we get correct counts from executed programs.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 131 ++++++++++++++++--
 .../selftests/bpf/progs/uprobe_multi.c        |   6 +-
 2 files changed, 125 insertions(+), 12 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index b12dc1f992e5..8ca7a45e21e6 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -23,14 +23,86 @@ noinline void uprobe_multi_func_3(void)
 	asm volatile ("");
 }
 
-static void uprobe_multi_test_run(struct uprobe_multi *skel)
+struct child {
+	int go[2];
+	int pid;
+};
+
+static void release_child(struct child *child)
+{
+	int child_status;
+
+	if (!child)
+		return;
+	close(child->go[1]);
+	close(child->go[0]);
+	if (child->pid > 0)
+		waitpid(child->pid, &child_status, 0);
+}
+
+static void kick_child(struct child *child)
+{
+	char c = 1;
+
+	if (child) {
+		write(child->go[1], &c, 1);
+		release_child(child);
+	}
+	fflush(NULL);
+}
+
+static struct child *spawn_child(void)
+{
+	static struct child child;
+	int err;
+	int c;
+
+	/* pid filter */
+	if (!ASSERT_OK(pipe(child.go), "pipe"))
+		return NULL;
+
+	child.pid = fork();
+	if (child.pid < 0) {
+		release_child(&child);
+		return NULL;
+	}
+
+	/* child */
+	if (child.pid == 0) {
+		close(child.go[1]);
+		fflush(NULL);
+		/* wait for parent's kick */
+		err = read(child.go[0], &c, 1);
+		if (!ASSERT_EQ(err, 1, "child_read_pipe"))
+			exit(err);
+
+		uprobe_multi_func_1();
+		uprobe_multi_func_2();
+		uprobe_multi_func_3();
+
+		exit(errno);
+	}
+
+	return &child;
+}
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel, struct child *child)
 {
 	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
 	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
 	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
 
 	skel->bss->user_ptr = test_data;
-	skel->bss->pid = getpid();
+
+	/*
+	 * Disable pid check in bpf program if we are pid filter test,
+	 * because the probe should be executed only by child->pid
+	 * passed at the probe attach.
+	 */
+	skel->bss->pid = child ? 0 : getpid();
+
+	if (child)
+		kick_child(child);
 
 	/* trigger all probes */
 	uprobe_multi_func_1();
@@ -50,6 +122,9 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel)
 	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 2, "uretprobe_multi_func_3_result");
 
 	ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 6, "uprobe_multi_sleep_result");
+
+	if (child)
+		ASSERT_EQ(skel->bss->child_pid, child->pid, "uprobe_multi_child_pid");
 }
 
 static void test_skel_api(void)
@@ -65,17 +140,19 @@ static void test_skel_api(void)
 	if (!ASSERT_OK(err, "uprobe_multi__attach"))
 		goto cleanup;
 
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, NULL);
 
 cleanup:
 	uprobe_multi__destroy(skel);
 }
 
 static void
-test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+__test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts,
+		  struct child *child)
 {
 	struct bpf_link *link1 = NULL, *link2 = NULL;
 	struct bpf_link *link3 = NULL, *link4 = NULL;
+	pid_t pid = child ? child->pid : -1;
 	struct uprobe_multi *skel = NULL;
 
 	skel = uprobe_multi__open_and_load();
@@ -83,30 +160,30 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi
 		goto cleanup;
 
 	opts->retprobe = false;
-	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
 		goto cleanup;
 
 	opts->retprobe = true;
-	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
 	opts->retprobe = false;
-	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
 		goto cleanup;
 
 	opts->retprobe = true;
-	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, child);
 
 cleanup:
 	bpf_link__destroy(link4);
@@ -116,6 +193,22 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi
 	uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+	struct child *child;
+
+	/* no pid filter */
+	__test_attach_api(binary, pattern, opts, NULL);
+
+	/* pid filter */
+	child = spawn_child();
+	if (!child)
+		return;
+
+	__test_attach_api(binary, pattern, opts, child);
+}
+
 static void test_attach_api_pattern(void)
 {
 	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
@@ -138,7 +231,7 @@ static void test_attach_api_syms(void)
 	test_attach_api("/proc/self/exe", NULL, &opts);
 }
 
-static void test_link_api(void)
+static void __test_link_api(struct child *child)
 {
 	int prog_fd, link1_fd = -1, link2_fd = -1, link3_fd = -1, link4_fd = -1;
 	LIBBPF_OPTS(bpf_link_create_opts, opts);
@@ -159,6 +252,7 @@ static void test_link_api(void)
 	opts.uprobe_multi.path = path;
 	opts.uprobe_multi.offsets = offsets;
 	opts.uprobe_multi.cnt = ARRAY_SIZE(syms);
+	opts.uprobe_multi.pid = child ? child->pid : 0;
 
 	skel = uprobe_multi__open_and_load();
 	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
@@ -187,7 +281,7 @@ static void test_link_api(void)
 	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
 	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
 		goto cleanup;
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, child);
 
 cleanup:
 	if (link1_fd >= 0)
@@ -203,6 +297,21 @@ static void test_link_api(void)
 	free(offsets);
 }
 
+void test_link_api(void)
+{
+	struct child *child;
+
+	/* no pid filter */
+	__test_link_api(NULL);
+
+	/* pid filter */
+	child = spawn_child();
+	if (!child)
+		return;
+
+	__test_link_api(child);
+}
+
 static inline __u64 get_time_ns(void)
 {
 	struct timespec t;
diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
index cd73139dc881..1086312b5d1e 100644
--- a/tools/testing/selftests/bpf/progs/uprobe_multi.c
+++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
@@ -21,6 +21,8 @@ __u64 uretprobe_multi_func_3_result = 0;
 __u64 uprobe_multi_sleep_result = 0;
 
 int pid = 0;
+int child_pid = 0;
+
 bool test_cookie = false;
 void *user_ptr = 0;
 
@@ -34,7 +36,9 @@ static __always_inline bool verify_sleepable_user_copy(void)
 
 static void uprobe_multi_check(void *ctx, bool is_return, bool is_sleep)
 {
-	if (bpf_get_current_pid_tgid() >> 32 != pid)
+	child_pid = bpf_get_current_pid_tgid() >> 32;
+
+	if (pid && child_pid != pid)
 		return;
 
 	__u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0;
-- 
2.41.0

[PATCHv3 bpf-next 26/26] selftests/bpf: Add extra link to uprobe_multi tests

[Jiri Olsa]

Attaching extra program to same functions system wide for api
and link tests.

This way we can test the pid filter works properly when there's
extra system wide consumer on the same uprobe that will trigger
the original uprobe handler.

We expect to have the same counts as before.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 19 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  6 ++++++
 2 files changed, 25 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
index 8ca7a45e21e6..39bd16786cd0 100644
--- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
@@ -153,6 +153,7 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mul
 	struct bpf_link *link1 = NULL, *link2 = NULL;
 	struct bpf_link *link3 = NULL, *link4 = NULL;
 	pid_t pid = child ? child->pid : -1;
+	struct bpf_link *link_extra = NULL;
 	struct uprobe_multi *skel = NULL;
 
 	skel = uprobe_multi__open_and_load();
@@ -183,9 +184,16 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mul
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
+	opts->retprobe = false;
+	link_extra = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_extra, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link_extra, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
 	uprobe_multi_test_run(skel, child);
 
 cleanup:
+	bpf_link__destroy(link_extra);
 	bpf_link__destroy(link4);
 	bpf_link__destroy(link3);
 	bpf_link__destroy(link2);
@@ -243,6 +251,7 @@ static void __test_link_api(struct child *child)
 		"uprobe_multi_func_2",
 		"uprobe_multi_func_3",
 	};
+	int link_extra_fd = -1;
 	int err;
 
 	err = elf_resolve_syms_offsets(path, 3, syms, (unsigned long **) &offsets);
@@ -281,6 +290,14 @@ static void __test_link_api(struct child *child)
 	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
 	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
 		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	opts.uprobe_multi.pid = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe_extra);
+	link_extra_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link_extra_fd, 0, "link_extra_fd"))
+		goto cleanup;
+
 	uprobe_multi_test_run(skel, child);
 
 cleanup:
@@ -292,6 +309,8 @@ static void __test_link_api(struct child *child)
 		close(link3_fd);
 	if (link4_fd >= 0)
 		close(link4_fd);
+	if (link_extra_fd >= 0)
+		close(link_extra_fd);
 
 	uprobe_multi__destroy(skel);
 	free(offsets);
diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
index 1086312b5d1e..8ee957670303 100644
--- a/tools/testing/selftests/bpf/progs/uprobe_multi.c
+++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
@@ -102,3 +102,9 @@ int test_uprobe_bench(struct pt_regs *ctx)
 	count++;
 	return 0;
 }
+
+SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*")
+int test_uprobe_extra(struct pt_regs *ctx)
+{
+	return 0;
+}
-- 
2.41.0

Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link

[Yafang Shao]

On Fri, Jun 30, 2023 at 4:34 PM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding support to specify cookies array for uprobe_multi link.
>
> The cookies array share indexes and length with other uprobe_multi
> arrays (offsets/ref_ctr_offsets).
>
> The cookies[i] value defines cookie for i-the uprobe and will be
> returned by bpf_get_attach_cookie helper when called from ebpf
> program hooked to that specific uprobe.
>
> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  include/uapi/linux/bpf.h       |  1 +
>  kernel/bpf/syscall.c           |  2 +-
>  kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
>  tools/include/uapi/linux/bpf.h |  1 +
>  4 files changed, 44 insertions(+), 5 deletions(-)
>
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index a236139f08ce..4103f395593b 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -1592,6 +1592,7 @@ union bpf_attr {
>                                 __aligned_u64   path;
>                                 __aligned_u64   offsets;
>                                 __aligned_u64   ref_ctr_offsets;
> +                               __aligned_u64   cookies;

Could you pls. explain why the 'cookies' is defined as a pointer ? Are
there real use cases to define different cookies for different probes
in a single uprobe_multi ?  Why can't we just use one cookie for all
probes?  Per my understanding, the cookie is used to identify the user
or the application, rather than each probe.

We also defined it the same way in kprobe_multi...


>                         } uprobe_multi;
>                 };
>         } link_create;
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 3b0582a64ce4..affad356c603 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
>         return err;
>  }
>
> -#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
> +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
>  static int link_create(union bpf_attr *attr, bpfptr_t uattr)
>  {
>         struct bpf_prog *prog;
> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index a0b9d034300f..4657df8f884e 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -87,6 +87,8 @@ static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size,
>  static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
>  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
>
> +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx);
> +
>  /**
>   * trace_call_bpf - invoke BPF program
>   * @call: tracepoint event
> @@ -1099,6 +1101,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
>         .arg1_type      = ARG_PTR_TO_CTX,
>  };
>
> +BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs)
> +{
> +       return bpf_uprobe_multi_cookie(current->bpf_ctx);
> +}
> +
> +static const struct bpf_func_proto bpf_get_attach_cookie_proto_umulti = {
> +       .func           = bpf_get_attach_cookie_uprobe_multi,
> +       .gpl_only       = false,
> +       .ret_type       = RET_INTEGER,
> +       .arg1_type      = ARG_PTR_TO_CTX,
> +};
> +
>  BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx)
>  {
>         struct bpf_trace_run_ctx *run_ctx;
> @@ -1545,9 +1559,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
>                         &bpf_get_func_ip_proto_kprobe_multi :
>                         &bpf_get_func_ip_proto_kprobe;
>         case BPF_FUNC_get_attach_cookie:
> -               return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ?
> -                       &bpf_get_attach_cookie_proto_kmulti :
> -                       &bpf_get_attach_cookie_proto_trace;
> +               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
> +                       return &bpf_get_attach_cookie_proto_kmulti;
> +               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
> +                       return &bpf_get_attach_cookie_proto_umulti;
> +               return &bpf_get_attach_cookie_proto_trace;
>         default:
>                 return bpf_tracing_func_proto(func_id, prog);
>         }
> @@ -2930,6 +2946,7 @@ struct bpf_uprobe_multi_link;
>  struct bpf_uprobe {
>         struct bpf_uprobe_multi_link *link;
>         loff_t offset;
> +       u64 cookie;
>         struct uprobe_consumer consumer;
>  };
>
> @@ -2943,6 +2960,7 @@ struct bpf_uprobe_multi_link {
>  struct bpf_uprobe_multi_run_ctx {
>         struct bpf_run_ctx run_ctx;
>         unsigned long entry_ip;
> +       struct bpf_uprobe *uprobe;
>  };
>
>  static void bpf_uprobe_unregister(struct path *path, struct bpf_uprobe *uprobes,
> @@ -2986,6 +3004,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
>         struct bpf_uprobe_multi_link *link = uprobe->link;
>         struct bpf_uprobe_multi_run_ctx run_ctx = {
>                 .entry_ip = entry_ip,
> +               .uprobe = uprobe,
>         };
>         struct bpf_prog *prog = link->link.prog;
>         bool sleepable = prog->aux->sleepable;
> @@ -3032,6 +3051,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
>         return uprobe_prog_run(uprobe, func, regs);
>  }
>
> +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
> +{
> +       struct bpf_uprobe_multi_run_ctx *run_ctx;
> +
> +       run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
> +       return run_ctx->uprobe->cookie;
> +}
> +
>  int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
>  {
>         struct bpf_uprobe_multi_link *link = NULL;
> @@ -3040,6 +3067,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
>         struct bpf_link_primer link_primer;
>         struct bpf_uprobe *uprobes = NULL;
>         unsigned long __user *uoffsets;
> +       u64 __user *ucookies;
>         void __user *upath;
>         u32 flags, cnt, i;
>         struct path path;
> @@ -3059,7 +3087,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
>
>         /*
>          * path, offsets and cnt are mandatory,
> -        * ref_ctr_offsets is optional
> +        * ref_ctr_offsets and cookies are optional
>          */
>         upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
>         uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
> @@ -3069,6 +3097,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
>                 return -EINVAL;
>
>         uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
> +       ucookies = u64_to_user_ptr(attr->link_create.uprobe_multi.cookies);
>
>         name = strndup_user(upath, PATH_MAX);
>         if (IS_ERR(name)) {
> @@ -3101,6 +3130,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
>         }
>
>         for (i = 0; i < cnt; i++) {
> +               if (ucookies && __get_user(uprobes[i].cookie, ucookies + i)) {
> +                       err = -EFAULT;
> +                       goto error_free;
> +               }
>                 if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
>                         err = -EFAULT;
>                         goto error_free;
> @@ -3158,4 +3191,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
>  {
>         return -EOPNOTSUPP;
>  }
> +static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
> +{
> +       return 0;
> +}
>  #endif /* CONFIG_UPROBES */
> diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> index a236139f08ce..4103f395593b 100644
> --- a/tools/include/uapi/linux/bpf.h
> +++ b/tools/include/uapi/linux/bpf.h
> @@ -1592,6 +1592,7 @@ union bpf_attr {
>                                 __aligned_u64   path;
>                                 __aligned_u64   offsets;
>                                 __aligned_u64   ref_ctr_offsets;
> +                               __aligned_u64   cookies;
>                         } uprobe_multi;
>                 };
>         } link_create;
> --
> 2.41.0
>
>


-- 
Regards
Yafang

Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link

[Jiri Olsa]

On Sat, Jul 01, 2023 at 11:40:05AM +0800, Yafang Shao wrote:
> On Fri, Jun 30, 2023 at 4:34 PM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding support to specify cookies array for uprobe_multi link.
> >
> > The cookies array share indexes and length with other uprobe_multi
> > arrays (offsets/ref_ctr_offsets).
> >
> > The cookies[i] value defines cookie for i-the uprobe and will be
> > returned by bpf_get_attach_cookie helper when called from ebpf
> > program hooked to that specific uprobe.
> >
> > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  include/uapi/linux/bpf.h       |  1 +
> >  kernel/bpf/syscall.c           |  2 +-
> >  kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
> >  tools/include/uapi/linux/bpf.h |  1 +
> >  4 files changed, 44 insertions(+), 5 deletions(-)
> >
> > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > index a236139f08ce..4103f395593b 100644
> > --- a/include/uapi/linux/bpf.h
> > +++ b/include/uapi/linux/bpf.h
> > @@ -1592,6 +1592,7 @@ union bpf_attr {
> >                                 __aligned_u64   path;
> >                                 __aligned_u64   offsets;
> >                                 __aligned_u64   ref_ctr_offsets;
> > +                               __aligned_u64   cookies;
> 
> Could you pls. explain why the 'cookies' is defined as a pointer ? Are
> there real use cases to define different cookies for different probes
> in a single uprobe_multi ?  Why can't we just use one cookie for all
> probes?  Per my understanding, the cookie is used to identify the user
> or the application, rather than each probe.
> 
> We also defined it the same way in kprobe_multi...

yes, each probed function (or symbol/offset for uprobe) can get
its own cookie and we use that to get probe-specific config in
generic bpf program with the bpf_get_attach_cookie helper

jirka

Re: [PATCHv3 bpf-next 00/26] bpf: Add multi uprobe link

[Daniel Borkmann]

On 6/30/23 10:33 AM, Jiri Olsa wrote:
> hi,
> this patchset is adding support to attach multiple uprobes and usdt probes
> through new uprobe_multi link.
> 
> The current uprobe is attached through the perf event and attaching many
> uprobes takes a lot of time because of that.
> 
> The main reason is that we need to install perf event for each probed function
> and profile shows perf event installation (perf_install_in_context) as culprit.
> 
> The new uprobe_multi link just creates raw uprobes and attaches the bpf
> program to them without perf event being involved.
> 
> In addition to being faster we also save file descriptors. For the current
> uprobe attach we use extra perf event fd for each probed function. The new
> link just need one fd that covers all the functions we are attaching to.
> 
> v3 changes:
>    - consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
>    - remove bpf_prog_active check [Alexei]
>    - change rcu locking [Andrii]
>    - allocate ref_ctr_offsets conditionally [Andrii]
>    - remove ctx check from bpf_uprobe_multi_cookie [Andrii]
>    - move some elf_* functions in elf.c object [Andrii]
>    - fix uprobe link detection code [Andrii]
>    - add usdt.s program section [Andrii]
>    - rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
>    - remove extra case from attach_uprobe_multi [Andrii]
>    - rework usdt_manager_attach_usdt [Andrii]
>    - rework/rename new elf_find_* functions [Andrii]
>    - elf iterator fixes [Andrii]
>      - renames
>      - elf_sym_iter_next loop restruct
>      - return directly GElf_Sym in elf_sym
>      - add elf_sym_offset helper
>      - add st_type arg elf_sym_iter_new
>      - get rid of '_' prefixed functions
>      - simplify offsets handling
>      - other smaller changes
>    - added acks
>    - todo:
>      - seems like more elf_* helpers could go in elf.c object,
>        I'll send that as follow up
>      - will send fill_info support when [3] is merged [Andrii]
> 
> 
> There's support for bpftrace [2] and tetragon [1].
> 
> Also available at:
>    https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>    uprobe_multi
> 
> thanks,
> jirka
> 
> 
> [1] https://github.com/cilium/tetragon/pull/936
> [2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
> [3] https://lore.kernel.org/bpf/20230628115329.248450-1-laoar.shao@gmail.com/
> ---
> Jiri Olsa (26):
>        bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
>        bpf: Add multi uprobe link
>        bpf: Add cookies support for uprobe_multi link
>        bpf: Add pid filter support for uprobe_multi link
>        bpf: Add bpf_get_func_ip helper support for uprobe link
>        libbpf: Add uprobe_multi attach type and link names
>        libbpf: Move elf_find_func_offset* functions to elf object
>        libbpf: Add elf_open/elf_close functions
>        libbpf: Add elf symbol iterator
>        libbpf: Add elf_resolve_syms_offsets function
>        libbpf: Add elf_resolve_pattern_offsets function
>        libbpf: Add bpf_link_create support for multi uprobes
>        libbpf: Add bpf_program__attach_uprobe_multi function
>        libbpf: Add support for u[ret]probe.multi[.s] program sections
>        libbpf: Add uprobe multi link detection
>        libbpf: Add uprobe multi link support to bpf_program__attach_usdt
>        selftests/bpf: Add uprobe_multi skel test
>        selftests/bpf: Add uprobe_multi api test
>        selftests/bpf: Add uprobe_multi link test
>        selftests/bpf: Add uprobe_multi test program
>        selftests/bpf: Add uprobe_multi bench test
>        selftests/bpf: Add usdt_multi test program
>        selftests/bpf: Add usdt_multi bench test
>        selftests/bpf: Add uprobe_multi cookie test
>        selftests/bpf: Add uprobe_multi pid filter tests
>        selftests/bpf: Add extra link to uprobe_multi tests
> 
>   include/linux/trace_events.h                               |   6 ++
>   include/uapi/linux/bpf.h                                   |  16 ++++
>   kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
>   kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
>   tools/include/uapi/linux/bpf.h                             |  16 ++++
>   tools/lib/bpf/Build                                        |   2 +-
>   tools/lib/bpf/bpf.c                                        |  11 +++
>   tools/lib/bpf/bpf.h                                        |  11 ++-
>   tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
>   tools/lib/bpf/libbpf.h                                     |  27 ++++++
>   tools/lib/bpf/libbpf.map                                   |   1 +
>   tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
>   tools/lib/bpf/libbpf_internal.h                            |   3 +
>   tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
>   tools/testing/selftests/bpf/Makefile                       |  10 +++
>   tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
>   tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
>   tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
>   tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
>   tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
>   22 files changed, 1969 insertions(+), 296 deletions(-)
>   create mode 100644 tools/lib/bpf/elf.c
>   create mode 100644 tools/lib/bpf/libbpf_elf.h
>   create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
>   create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
>   create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
>   create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
>   create mode 100644 tools/testing/selftests/bpf/usdt_multi.c

Looks like BPF CI did not trigger because of build error with LLVM:

https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097

   [...]
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK(err, "uprobe_multi__attach"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK(err, "uprobe_multi__attach"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning
           long attach_start_ns, attach_end_ns;
                                              ^
                                               = 0
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                           ^~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                           ^~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning
           long attach_start_ns, attach_end_ns;
                               ^
                                = 0
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                           ^~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning
           long attach_start_ns, attach_end_ns;
                                              ^
                                               = 0
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                           ^~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                           ^~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning
           long attach_start_ns, attach_end_ns;
                               ^
                                = 0
   10 errors generated.
     TEST-OBJ [test_progs] lookup_key.test.o
   make: *** [Makefile:578: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
   make: *** Waiting for unfinished jobs....
   make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
   Error: Process completed with exit code 2.

Re: [PATCHv3 bpf-next 00/26] bpf: Add multi uprobe link

[Jiri Olsa]

On Wed, Jul 05, 2023 at 02:45:42PM +0200, Daniel Borkmann wrote:
> On 6/30/23 10:33 AM, Jiri Olsa wrote:
> > hi,
> > this patchset is adding support to attach multiple uprobes and usdt probes
> > through new uprobe_multi link.
> > 
> > The current uprobe is attached through the perf event and attaching many
> > uprobes takes a lot of time because of that.
> > 
> > The main reason is that we need to install perf event for each probed function
> > and profile shows perf event installation (perf_install_in_context) as culprit.
> > 
> > The new uprobe_multi link just creates raw uprobes and attaches the bpf
> > program to them without perf event being involved.
> > 
> > In addition to being faster we also save file descriptors. For the current
> > uprobe attach we use extra perf event fd for each probed function. The new
> > link just need one fd that covers all the functions we are attaching to.
> > 
> > v3 changes:
> >    - consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
> >    - remove bpf_prog_active check [Alexei]
> >    - change rcu locking [Andrii]
> >    - allocate ref_ctr_offsets conditionally [Andrii]
> >    - remove ctx check from bpf_uprobe_multi_cookie [Andrii]
> >    - move some elf_* functions in elf.c object [Andrii]
> >    - fix uprobe link detection code [Andrii]
> >    - add usdt.s program section [Andrii]
> >    - rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
> >    - remove extra case from attach_uprobe_multi [Andrii]
> >    - rework usdt_manager_attach_usdt [Andrii]
> >    - rework/rename new elf_find_* functions [Andrii]
> >    - elf iterator fixes [Andrii]
> >      - renames
> >      - elf_sym_iter_next loop restruct
> >      - return directly GElf_Sym in elf_sym
> >      - add elf_sym_offset helper
> >      - add st_type arg elf_sym_iter_new
> >      - get rid of '_' prefixed functions
> >      - simplify offsets handling
> >      - other smaller changes
> >    - added acks
> >    - todo:
> >      - seems like more elf_* helpers could go in elf.c object,
> >        I'll send that as follow up
> >      - will send fill_info support when [3] is merged [Andrii]
> > 
> > 
> > There's support for bpftrace [2] and tetragon [1].
> > 
> > Also available at:
> >    https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
> >    uprobe_multi
> > 
> > thanks,
> > jirka
> > 
> > 
> > [1] https://github.com/cilium/tetragon/pull/936
> > [2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
> > [3] https://lore.kernel.org/bpf/20230628115329.248450-1-laoar.shao@gmail.com/
> > ---
> > Jiri Olsa (26):
> >        bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
> >        bpf: Add multi uprobe link
> >        bpf: Add cookies support for uprobe_multi link
> >        bpf: Add pid filter support for uprobe_multi link
> >        bpf: Add bpf_get_func_ip helper support for uprobe link
> >        libbpf: Add uprobe_multi attach type and link names
> >        libbpf: Move elf_find_func_offset* functions to elf object
> >        libbpf: Add elf_open/elf_close functions
> >        libbpf: Add elf symbol iterator
> >        libbpf: Add elf_resolve_syms_offsets function
> >        libbpf: Add elf_resolve_pattern_offsets function
> >        libbpf: Add bpf_link_create support for multi uprobes
> >        libbpf: Add bpf_program__attach_uprobe_multi function
> >        libbpf: Add support for u[ret]probe.multi[.s] program sections
> >        libbpf: Add uprobe multi link detection
> >        libbpf: Add uprobe multi link support to bpf_program__attach_usdt
> >        selftests/bpf: Add uprobe_multi skel test
> >        selftests/bpf: Add uprobe_multi api test
> >        selftests/bpf: Add uprobe_multi link test
> >        selftests/bpf: Add uprobe_multi test program
> >        selftests/bpf: Add uprobe_multi bench test
> >        selftests/bpf: Add usdt_multi test program
> >        selftests/bpf: Add usdt_multi bench test
> >        selftests/bpf: Add uprobe_multi cookie test
> >        selftests/bpf: Add uprobe_multi pid filter tests
> >        selftests/bpf: Add extra link to uprobe_multi tests
> > 
> >   include/linux/trace_events.h                               |   6 ++
> >   include/uapi/linux/bpf.h                                   |  16 ++++
> >   kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
> >   kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
> >   tools/include/uapi/linux/bpf.h                             |  16 ++++
> >   tools/lib/bpf/Build                                        |   2 +-
> >   tools/lib/bpf/bpf.c                                        |  11 +++
> >   tools/lib/bpf/bpf.h                                        |  11 ++-
> >   tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >   tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
> >   tools/lib/bpf/libbpf.h                                     |  27 ++++++
> >   tools/lib/bpf/libbpf.map                                   |   1 +
> >   tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
> >   tools/lib/bpf/libbpf_internal.h                            |   3 +
> >   tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
> >   tools/testing/selftests/bpf/Makefile                       |  10 +++
> >   tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
> >   tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >   tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
> >   tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
> >   tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
> >   tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
> >   22 files changed, 1969 insertions(+), 296 deletions(-)
> >   create mode 100644 tools/lib/bpf/elf.c
> >   create mode 100644 tools/lib/bpf/libbpf_elf.h
> >   create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> >   create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
> >   create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
> >   create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
> >   create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
> 
> Looks like BPF CI did not trigger because of build error with LLVM:
> 
> https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097

will check, thanks

jirka

> 
>   [...]
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK(err, "uprobe_multi__attach"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK(err, "uprobe_multi__attach"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning
>           long attach_start_ns, attach_end_ns;
>                                              ^
>                                               = 0
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                                           ^~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                                           ^~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning
>           long attach_start_ns, attach_end_ns;
>                               ^
>                                = 0
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                           ^~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning
>           long attach_start_ns, attach_end_ns;
>                                              ^
>                                               = 0
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                                           ^~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
>           attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
>                                           ^~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
>           if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
>           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>   /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning
>           long attach_start_ns, attach_end_ns;
>                               ^
>                                = 0
>   10 errors generated.
>     TEST-OBJ [test_progs] lookup_key.test.o
>   make: *** [Makefile:578: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
>   make: *** Waiting for unfinished jobs....
>   make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
>   Error: Process completed with exit code 2.

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding support for bpf_get_func_ip helper being called from
> ebpf program attached by uprobe_multi link.
>
> It returns the ip of the uprobe.
>
> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
>  1 file changed, 30 insertions(+), 3 deletions(-)
>

A slight aside related to bpf_get_func_ip() support in
uprobe/uretprobe. We just had a conversation with Alastair and Jordan
(cc'ed) about bpftrace and using bpf_get_func_ip() there with
uretprobes, and it seems like it doesn't work.

Is that intentional or we just missed that bpf_get_func_ip() doesn't
work with uprobes/uretprobes? Do you think it would be hard to add
support for them for bpf_get_func_ip()? It's a very useful helper,
would be nice to have it working in all cases where it has meaningful
behavior (and I think it does for uprobe and uretprobe).

Thanks!

> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 4ef51fd0497f..f5a41c1604b8 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
> @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
>  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
>

[...]

Re: [PATCHv3 bpf-next 01/26] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Add extra attach_type checks from link_create under
> bpf_prog_attach_check_attach_type.
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------
>  1 file changed, 47 insertions(+), 61 deletions(-)
>
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index a2aef900519c..9046ad0f9b4e 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)
>         return fd;
>  }
>
> -static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> -                                            enum bpf_attach_type attach_type)
> -{
> -       switch (prog->type) {
> -       case BPF_PROG_TYPE_CGROUP_SOCK:
> -       case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> -       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> -       case BPF_PROG_TYPE_SK_LOOKUP:
> -               return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
> -       case BPF_PROG_TYPE_CGROUP_SKB:
> -               if (!capable(CAP_NET_ADMIN))
> -                       /* cg-skb progs can be loaded by unpriv user.
> -                        * check permissions at attach time.
> -                        */
> -                       return -EPERM;
> -               return prog->enforce_expected_attach_type &&
> -                       prog->expected_attach_type != attach_type ?
> -                       -EINVAL : 0;
> -       case BPF_PROG_TYPE_KPROBE:
> -               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
> -                   attach_type != BPF_TRACE_KPROBE_MULTI)
> -                       return -EINVAL;
> -               return 0;
> -       default:
> -               return 0;
> -       }
> -}
> -
>  static enum bpf_prog_type
>  attach_type_to_prog_type(enum bpf_attach_type attach_type)
>  {
> @@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type)
>         }
>  }
>
> +static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> +                                            enum bpf_attach_type attach_type)
> +{
> +       enum bpf_prog_type ptype;
> +
> +       switch (prog->type) {
> +       case BPF_PROG_TYPE_CGROUP_SOCK:
> +       case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
> +       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
> +       case BPF_PROG_TYPE_SK_LOOKUP:
> +               return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
> +       case BPF_PROG_TYPE_CGROUP_SKB:
> +               if (!capable(CAP_NET_ADMIN))
> +                       /* cg-skb progs can be loaded by unpriv user.
> +                        * check permissions at attach time.
> +                        */
> +                       return -EPERM;
> +               return prog->enforce_expected_attach_type &&
> +                       prog->expected_attach_type != attach_type ?
> +                       -EINVAL : 0;
> +       case BPF_PROG_TYPE_KPROBE:

nit, I'd keep KPROBE, TRACEPOINT and PERF_EVENT next to each other in
this switch (that will just grow larger over time), as they are pretty
closely related

otherwise lgtm:

Acked-by: Andrii Nakryiko <andrii@kernel.org>


> +               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
> +                   attach_type != BPF_TRACE_KPROBE_MULTI)
> +                       return -EINVAL;
> +               if (attach_type != BPF_PERF_EVENT &&
> +                   attach_type != BPF_TRACE_KPROBE_MULTI)
> +                       return -EINVAL;
> +               return 0;
> +       case BPF_PROG_TYPE_EXT:
> +               return 0;
> +       case BPF_PROG_TYPE_NETFILTER:
> +               if (attach_type != BPF_NETFILTER)
> +                       return -EINVAL;
> +               return 0;
> +       case BPF_PROG_TYPE_PERF_EVENT:
> +       case BPF_PROG_TYPE_TRACEPOINT:
> +               if (attach_type != BPF_PERF_EVENT)
> +                       return -EINVAL;
> +               return 0;
> +       default:
> +               ptype = attach_type_to_prog_type(attach_type);
> +               if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type)
> +                       return -EINVAL;
> +               return 0;
> +       }
> +}
> +
>  #define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd
>
>  #define BPF_F_ATTACH_MASK \
> @@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
>  #define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
>  static int link_create(union bpf_attr *attr, bpfptr_t uattr)
>  {
> -       enum bpf_prog_type ptype;
>         struct bpf_prog *prog;
>         int ret;
>
> @@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
>         if (ret)
>                 goto out;
>
> -       switch (prog->type) {
> -       case BPF_PROG_TYPE_EXT:
> -               break;
> -       case BPF_PROG_TYPE_NETFILTER:
> -               if (attr->link_create.attach_type != BPF_NETFILTER) {
> -                       ret = -EINVAL;
> -                       goto out;
> -               }
> -               break;
> -       case BPF_PROG_TYPE_PERF_EVENT:
> -       case BPF_PROG_TYPE_TRACEPOINT:
> -               if (attr->link_create.attach_type != BPF_PERF_EVENT) {
> -                       ret = -EINVAL;
> -                       goto out;
> -               }
> -               break;
> -       case BPF_PROG_TYPE_KPROBE:
> -               if (attr->link_create.attach_type != BPF_PERF_EVENT &&
> -                   attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI) {
> -                       ret = -EINVAL;
> -                       goto out;
> -               }
> -               break;
> -       default:
> -               ptype = attach_type_to_prog_type(attr->link_create.attach_type);
> -               if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) {
> -                       ret = -EINVAL;
> -                       goto out;
> -               }
> -               break;
> -       }
> -
>         switch (prog->type) {
>         case BPF_PROG_TYPE_CGROUP_SKB:
>         case BPF_PROG_TYPE_CGROUP_SOCK:
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 02/26] bpf: Add multi uprobe link

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding new multi uprobe link that allows to attach bpf program
> to multiple uprobes.
>
> Uprobes to attach are specified via new link_create uprobe_multi
> union:
>
>   struct {
>           __u32           flags;
>           __u32           cnt;
>           __aligned_u64   path;
>           __aligned_u64   offsets;
>           __aligned_u64   ref_ctr_offsets;
>   } uprobe_multi;
>
> Uprobes are defined for single binary specified in path and multiple
> calling sites specified in offsets array with optional reference
> counters specified in ref_ctr_offsets array. All specified arrays
> have length of 'cnt'.
>
> The 'flags' supports single bit for now that marks the uprobe as
> return probe.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  include/linux/trace_events.h   |   6 +
>  include/uapi/linux/bpf.h       |  14 ++
>  kernel/bpf/syscall.c           |  14 +-
>  kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
>  tools/include/uapi/linux/bpf.h |  14 ++
>  5 files changed, 282 insertions(+), 3 deletions(-)
>

overall LGTM, but I think there is path leak, please fix that and add my ack

Acked-by: Andrii Nakryiko <andrii@kernel.org>

> diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
> index 7c4a0b72334e..c71845e9d40a 100644
> --- a/include/linux/trace_events.h
> +++ b/include/linux/trace_events.h
> @@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id,
>                             u32 *fd_type, const char **buf,
>                             u64 *probe_offset, u64 *probe_addr);
>  int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
> +int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
>  #else
>  static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx)
>  {
> @@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
>  {
>         return -EOPNOTSUPP;
>  }
> +static inline int
> +bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
> +{
> +       return -EOPNOTSUPP;
> +}
>  #endif
>
>  enum {
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 60a9d59beeab..a236139f08ce 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -1036,6 +1036,7 @@ enum bpf_attach_type {
>         BPF_LSM_CGROUP,
>         BPF_STRUCT_OPS,
>         BPF_NETFILTER,
> +       BPF_TRACE_UPROBE_MULTI,
>         __MAX_BPF_ATTACH_TYPE
>  };
>
> @@ -1053,6 +1054,7 @@ enum bpf_link_type {
>         BPF_LINK_TYPE_KPROBE_MULTI = 8,
>         BPF_LINK_TYPE_STRUCT_OPS = 9,
>         BPF_LINK_TYPE_NETFILTER = 10,
> +       BPF_LINK_TYPE_UPROBE_MULTI = 11,
>
>         MAX_BPF_LINK_TYPE,
>  };
> @@ -1170,6 +1172,11 @@ enum bpf_link_type {
>   */
>  #define BPF_F_KPROBE_MULTI_RETURN      (1U << 0)
>
> +/* link_create.uprobe_multi.flags used in LINK_CREATE command for
> + * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
> + */
> +#define BPF_F_UPROBE_MULTI_RETURN      (1U << 0)
> +

any reason why we don't use anonymous ENUMs for all these UAPI
constants? When we need to use these flags from BPF side (e.g., for
BPF LSM), having them as #defines will be a PITA, as they won't be
present in vmlinux.h


>  /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
>   * the following extensions:
>   *
> @@ -1579,6 +1586,13 @@ union bpf_attr {
>                                 __s32           priority;
>                                 __u32           flags;
>                         } netfilter;
> +                       struct {
> +                               __u32           flags;
> +                               __u32           cnt;

total nit, but I'd move it after path/offsets/ref_ctr_offsets, and
make the order cnt (as it applies to previous two
offsets/ref_ctr_offsets) and then flags last. Seems like more logical
order, but totally subjective

> +                               __aligned_u64   path;
> +                               __aligned_u64   offsets;
> +                               __aligned_u64   ref_ctr_offsets;
> +                       } uprobe_multi;
>                 };
>         } link_create;
>
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 9046ad0f9b4e..3b0582a64ce4 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
>
>  /* Clean up bpf_link and corresponding anon_inode file and FD. After
>   * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
> - * anon_inode's release() call. This helper marksbpf_link as
> + * anon_inode's release() call. This helper marks bpf_link as
>   * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
>   * is not decremented, it's the responsibility of a calling code that failed
>   * to complete bpf_link initialization.
> + * This helper eventually calls link's dealloc callback, but does not call
> + * link's release callback.

Thanks for clarifying comments!

>   */
>  void bpf_link_cleanup(struct bpf_link_primer *primer)
>  {
> @@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
>                 if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
>                     attach_type != BPF_TRACE_KPROBE_MULTI)
>                         return -EINVAL;
> +               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
> +                   attach_type != BPF_TRACE_UPROBE_MULTI)
> +                       return -EINVAL;
>                 if (attach_type != BPF_PERF_EVENT &&
> -                   attach_type != BPF_TRACE_KPROBE_MULTI)
> +                   attach_type != BPF_TRACE_KPROBE_MULTI &&
> +                   attach_type != BPF_TRACE_UPROBE_MULTI)

if this keeps growing, we should think about having a switch in a
switch to not repeat BPF_TRACE_UPROBE_MULTI and BPF_TRACE_KPROBE_MULTI
twice

>                         return -EINVAL;
>                 return 0;
>         case BPF_PROG_TYPE_EXT:

[...]

> +       for (i = 0; i < cnt; i++) {
> +               if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
> +                       err = -EFAULT;
> +                       goto error_free;
> +               }
> +               if (__get_user(uprobes[i].offset, uoffsets + i)) {
> +                       err = -EFAULT;
> +                       goto error_free;
> +               }
> +
> +               uprobes[i].link = link;
> +
> +               if (flags & BPF_F_UPROBE_MULTI_RETURN)
> +                       uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
> +               else
> +                       uprobes[i].consumer.handler = uprobe_multi_link_handler;
> +       }
> +
> +       link->cnt = cnt;
> +       link->uprobes = uprobes;
> +       link->path = path;
> +
> +       bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI,
> +                     &bpf_uprobe_multi_link_lops, prog);
> +
> +       err = bpf_link_prime(&link->link, &link_primer);
> +       if (err)
> +               goto error_free;
> +
> +       for (i = 0; i < cnt; i++) {
> +               err = uprobe_register_refctr(d_real_inode(link->path.dentry),
> +                                            uprobes[i].offset,
> +                                            ref_ctr_offsets ? ref_ctr_offsets[i] : 0,
> +                                            &uprobes[i].consumer);
> +               if (err) {
> +                       bpf_uprobe_unregister(&path, uprobes, i);
> +                       bpf_link_cleanup(&link_primer);
> +                       kvfree(ref_ctr_offsets);

are we missing path_put() in this error handling path? so maybe goto
error_path_put here instead of return?

> +                       return err;
> +               }
> +       }
> +
> +       kvfree(ref_ctr_offsets);
> +       return bpf_link_settle(&link_primer);
> +
> +error_free:
> +       kvfree(ref_ctr_offsets);
> +       kvfree(uprobes);
> +       kfree(link);
> +error_path_put:
> +       path_put(&path);
> +       return err;
> +}

[...]

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding new elf object that will contain elf related functions.
> There's no functional change.
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/Build        |   2 +-
>  tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf.c     | 186 +---------------------------------
>  tools/lib/bpf/libbpf_elf.h |  11 +++
>  4 files changed, 211 insertions(+), 186 deletions(-)
>  create mode 100644 tools/lib/bpf/elf.c
>  create mode 100644 tools/lib/bpf/libbpf_elf.h
>

[...]

> diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
> new file mode 100644
> index 000000000000..1b652220fabf
> --- /dev/null
> +++ b/tools/lib/bpf/libbpf_elf.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
> +
> +#ifndef __LIBBPF_LIBBPF_ELF_H
> +#define __LIBBPF_LIBBPF_ELF_H
> +
> +#include <libelf.h>
> +
> +long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
> +long elf_find_func_offset_from_file(const char *binary_path, const char *name);
> +
> +#endif /* *__LIBBPF_LIBBPF_ELF_H */

we have libbpf_internal.h, let's put all this there for now, it's
already all the internal stuff together, I don't know if separate
header with few functions gives us much

> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding new elf object that will contain elf related functions.
> There's no functional change.
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/Build        |   2 +-
>  tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf.c     | 186 +---------------------------------
>  tools/lib/bpf/libbpf_elf.h |  11 +++
>  4 files changed, 211 insertions(+), 186 deletions(-)
>  create mode 100644 tools/lib/bpf/elf.c
>  create mode 100644 tools/lib/bpf/libbpf_elf.h
>
> diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build
> index b8b0a6369363..2d0c282c8588 100644
> --- a/tools/lib/bpf/Build
> +++ b/tools/lib/bpf/Build
> @@ -1,4 +1,4 @@
>  libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
>             netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
>             btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
> -           usdt.o zip.o
> +           usdt.o zip.o elf.o
> diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> new file mode 100644
> index 000000000000..2b62b4af28ce
> --- /dev/null
> +++ b/tools/lib/bpf/elf.c
> @@ -0,0 +1,198 @@
> +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> +
> +#include <libelf.h>
> +#include <gelf.h>
> +#include <fcntl.h>
> +#include <linux/kernel.h>

do you know why we need linux/kernel.h include? is it to get __u32 and
other typedefs?

> +
> +#include "libbpf_elf.h"
> +#include "libbpf_internal.h"
> +#include "str_error.h"
> +
> +#define STRERR_BUFSIZE  128
> +

[...]

Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding elf_open/elf_close functions and using it in
> elf_find_func_offset_from_file function. It will be
> used in following changes to save some common code.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
>  tools/lib/bpf/libbpf_elf.h |  8 ++++++
>  tools/lib/bpf/usdt.c       | 31 ++++++--------------
>  3 files changed, 56 insertions(+), 42 deletions(-)
>

one nit below

Acked-by: Andrii Nakryiko <andrii@kernel.org>

> diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> index 2b62b4af28ce..74e35071d22e 100644
> --- a/tools/lib/bpf/elf.c
> +++ b/tools/lib/bpf/elf.c
> @@ -11,6 +11,40 @@
>
>  #define STRERR_BUFSIZE  128
>
> +int elf_open(const char *binary_path, struct elf_fd *elf_fd)
> +{
> +       char errmsg[STRERR_BUFSIZE];
> +       int fd, ret;
> +       Elf *elf;
> +
> +       if (elf_version(EV_CURRENT) == EV_NONE) {
> +               pr_warn("elf: failed to init libelf for %s\n", binary_path);
> +               return -LIBBPF_ERRNO__LIBELF;
> +       }
> +       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
> +       if (fd < 0) {
> +               ret = -errno;
> +               pr_warn("elf: failed to open %s: %s\n", binary_path,
> +                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
> +               return ret;
> +       }
> +       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
> +       if (!elf) {
> +               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
> +               close(fd);
> +               return -LIBBPF_ERRNO__FORMAT;
> +       }
> +       elf_fd->fd = fd;
> +       elf_fd->elf = elf;
> +       return 0;
> +}
> +
> +void elf_close(struct elf_fd *elf_fd)
> +{
> +       elf_end(elf_fd->elf);
> +       close(elf_fd->fd);

nit: I'd make elf_close() work correctly with a) NULL elf_fd and b)
NULL elf_fd->elf, just to never have to think about this

> +}
> +
>  /* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
>  static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
>  {

[...]

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding elf symbol iterator object (and some functions) that follow
> open-coded iterator pattern and some functions to ease up iterating
> elf object symbols.
>
> The idea is to iterate single symbol section with:
>
>   struct elf_sym_iter iter;
>   struct elf_sym *sym;
>
>   if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))
>         goto error;
>
>   while ((sym = elf_sym_iter_next(&iter))) {
>         ...
>   }
>
> I considered opening the elf inside the iterator and iterate all symbol
> sections, but then it gets more complicated wrt user checks for when
> the next section is processed.
>
> Plus side is the we don't need 'exit' function, because caller/user is
> in charge of that.
>
> The returned iterated symbol object from elf_sym_iter_next function
> is placed inside the struct elf_sym_iter, so no extra allocation or
> argument is needed.
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
>  1 file changed, 117 insertions(+), 61 deletions(-)
>

A bunch of nits, but overall looks good. Please address nits, and add my ack

Acked-by: Andrii Nakryiko <andrii@kernel.org>

> diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> index 74e35071d22e..fcce4bd2478f 100644
> --- a/tools/lib/bpf/elf.c
> +++ b/tools/lib/bpf/elf.c
> @@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
>         return NULL;
>  }
>
> +struct elf_sym {
> +       const char *name;
> +       GElf_Sym sym;
> +       GElf_Shdr sh;
> +};
> +

if we want to use elf_sym_iter outside of elf.c, this should be in
libbpf_internal.h?

> +struct elf_sym_iter {
> +       Elf *elf;
> +       Elf_Data *syms;
> +       size_t nr_syms;
> +       size_t strtabidx;
> +       size_t next_sym_idx;
> +       struct elf_sym sym;
> +       int st_type;
> +};
> +
> +static int elf_sym_iter_new(struct elf_sym_iter *iter,
> +                           Elf *elf, const char *binary_path,
> +                           int sh_type, int st_type)
> +{
> +       Elf_Scn *scn = NULL;
> +       GElf_Ehdr ehdr;
> +       GElf_Shdr sh;
> +
> +       memset(iter, 0, sizeof(*iter));
> +
> +       if (!gelf_getehdr(elf, &ehdr)) {
> +               pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
> +               return -EINVAL;
> +       }
> +
> +       scn = elf_find_next_scn_by_type(elf, sh_type, NULL);
> +       if (!scn) {
> +               pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
> +                        binary_path);
> +               return -ENOENT;
> +       }
> +
> +       if (!gelf_getshdr(scn, &sh))
> +               return -EINVAL;
> +
> +       iter->strtabidx = sh.sh_link;
> +       iter->syms = elf_getdata(scn, 0);
> +       if (!iter->syms) {
> +               pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
> +                       binary_path, elf_errmsg(-1));
> +               return -EINVAL;
> +       }
> +       iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
> +       iter->elf = elf;
> +       iter->st_type = st_type;
> +       return 0;
> +}
> +
> +static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
> +{
> +       struct elf_sym *ret = &iter->sym;
> +       GElf_Sym *sym = &ret->sym;
> +       const char *name = NULL;
> +       Elf_Scn *sym_scn;
> +       size_t idx;
> +
> +       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
> +               if (!gelf_getsym(iter->syms, idx, sym))
> +                       continue;
> +               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
> +                       continue;
> +               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
> +               if (!name)
> +                       continue;
> +
> +               /* Transform symbol's virtual address (absolute for
> +                * binaries and relative for shared libs) into file
> +                * offset, which is what kernel is expecting for
> +                * uprobe/uretprobe attachment.
> +                * See Documentation/trace/uprobetracer.rst for more
> +                * details.
> +                * This is done by looking up symbol's containing
> +                * section's header and using iter's virtual address
> +                * (sh_addr) and corresponding file offset (sh_offset)
> +                * to transform sym.st_value (virtual address) into
> +                * desired final file offset.
> +                */

this comment is misplaced? we don't do the translation here

> +               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
> +               if (!sym_scn)
> +                       continue;
> +               if (!gelf_getshdr(sym_scn, &ret->sh))
> +                       continue;
> +
> +               iter->next_sym_idx = idx + 1;
> +               ret->name = name;
> +               return ret;
> +       }
> +
> +       return NULL;
> +}
> +
> +static unsigned long elf_sym_offset(struct elf_sym *sym)
> +{
> +       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
> +}
> +
>  /* Find offset of function name in the provided ELF object. "binary_path" is
>   * the path to the ELF binary represented by "elf", and only used for error
>   * reporting matters. "name" matches symbol name or name@@LIB for library
> @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
>          * reported as a warning/error.
>          */
>         for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> -               size_t nr_syms, strtabidx, idx;
> -               Elf_Data *symbols = NULL;
> -               Elf_Scn *scn = NULL;
> +               struct elf_sym_iter iter;
> +               struct elf_sym *sym;
>                 int last_bind = -1;
> -               const char *sname;
> -               GElf_Shdr sh;
> +               int curr_bind;

OCD nit:

$ rg 'curr(_|\b)' | wc -l
8
$ rg 'cur(_|\b)' | wc -l
148

and those 8 I consider an unfortunate accident ;) let's standardize on
using "cur" consistently

>
> -               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
> -               if (!scn) {
> -                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
> -                                binary_path);
> -                       continue;
> -               }
> -               if (!gelf_getshdr(scn, &sh))
> -                       continue;
> -               strtabidx = sh.sh_link;
> -               symbols = elf_getdata(scn, 0);
> -               if (!symbols) {
> -                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
> -                               binary_path, elf_errmsg(-1));
> -                       ret = -LIBBPF_ERRNO__FORMAT;
> +               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
> +               if (ret) {
> +                       if (ret == -ENOENT)
> +                               continue;
>                         goto out;

another styling nit: let's avoid unnecessary nesting of ifs:

if (ret == -ENOENT)
    continue;
if (ret)
    goto out;

simple and clean


>                 }
> -               nr_syms = symbols->d_size / sh.sh_entsize;
> -
> -               for (idx = 0; idx < nr_syms; idx++) {
> -                       int curr_bind;
> -                       GElf_Sym sym;
> -                       Elf_Scn *sym_scn;
> -                       GElf_Shdr sym_sh;
> -
> -                       if (!gelf_getsym(symbols, idx, &sym))
> -                               continue;
> -
> -                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
> -                               continue;
> -
> -                       sname = elf_strptr(elf, strtabidx, sym.st_name);
> -                       if (!sname)
> -                               continue;
> -
> -                       curr_bind = GELF_ST_BIND(sym.st_info);
>
> +               while ((sym = elf_sym_iter_next(&iter))) {
>                         /* User can specify func, func@@LIB or func@@LIB_VERSION. */
> -                       if (strncmp(sname, name, name_len) != 0)
> +                       if (strncmp(sym->name, name, name_len) != 0)
>                                 continue;
>                         /* ...but we don't want a search for "foo" to match 'foo2" also, so any
>                          * additional characters in sname should be of the form "@@LIB".
>                          */
> -                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
> +                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
>                                 continue;
>
> -                       if (ret >= 0) {
> +                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
> +
> +                       if (ret > 0) {

used to be >=, why the change?

>                                 /* handle multiple matches */
>                                 if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
>                                         /* Only accept one non-weak bind. */
>                                         pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
> -                                               sname, name, binary_path);
> +                                               sym->name, name, binary_path);
>                                         ret = -LIBBPF_ERRNO__FORMAT;
>                                         goto out;
>                                 } else if (curr_bind == STB_WEAK) {
> @@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
>                                 }
>                         }
>
> -                       /* Transform symbol's virtual address (absolute for
> -                        * binaries and relative for shared libs) into file
> -                        * offset, which is what kernel is expecting for
> -                        * uprobe/uretprobe attachment.
> -                        * See Documentation/trace/uprobetracer.rst for more
> -                        * details.
> -                        * This is done by looking up symbol's containing
> -                        * section's header and using it's virtual address
> -                        * (sh_addr) and corresponding file offset (sh_offset)
> -                        * to transform sym.st_value (virtual address) into
> -                        * desired final file offset.
> -                        */
> -                       sym_scn = elf_getscn(elf, sym.st_shndx);
> -                       if (!sym_scn)
> -                               continue;
> -                       if (!gelf_getshdr(sym_scn, &sym_sh))
> -                               continue;
> -
> -                       ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
> +                       ret = elf_sym_offset(sym);
>                         last_bind = curr_bind;
>                 }
>                 if (ret > 0)
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding elf_resolve_syms_offsets function that looks up
> offsets for symbols specified in syms array argument.
>
> Offsets are returned in allocated array with the 'cnt' size,
> that needs to be released by the caller.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf_elf.h |   2 +
>  2 files changed, 107 insertions(+)
>
> diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> index fcce4bd2478f..7e2f3b2e1fb6 100644
> --- a/tools/lib/bpf/elf.c
> +++ b/tools/lib/bpf/elf.c
> @@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
>         elf_close(&elf_fd);
>         return ret;
>  }
> +
> +struct symbol {
> +       const char *name;
> +       int bind;
> +       int idx;
> +};
> +
> +static int symbol_cmp(const void *_a, const void *_b)
> +{
> +       const struct symbol *a = _a;
> +       const struct symbol *b = _b;

please, let's not (over)use leading underscores, x/y, s1/s2, whatever

> +
> +       return strcmp(a->name, b->name);
> +}
> +

probably worth leaving a comment that the caller should free offsets on success?

> +int elf_resolve_syms_offsets(const char *binary_path, int cnt,
> +                            const char **syms, unsigned long **poffsets)
> +{
> +       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
> +       int err = 0, i, cnt_done = 0;
> +       unsigned long *offsets;
> +       struct symbol *symbols;
> +       struct elf_fd elf_fd;
> +
> +       err = elf_open(binary_path, &elf_fd);
> +       if (err)
> +               return err;
> +
> +       offsets = calloc(cnt, sizeof(*offsets));
> +       symbols = calloc(cnt, sizeof(*symbols));
> +
> +       if (!offsets || !symbols) {
> +               err = -ENOMEM;
> +               goto out;
> +       }
> +
> +       for (i = 0; i < cnt; i++) {
> +               symbols[i].name = syms[i];
> +               symbols[i].idx = i;
> +       }
> +
> +       qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
> +
> +       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> +               struct elf_sym_iter iter;
> +               struct elf_sym *sym;
> +
> +               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
> +               if (err) {
> +                       if (err == -ENOENT)
> +                               continue;
> +                       goto out;
> +               }

same nit, no need for nested ifs
> +
> +               while ((sym = elf_sym_iter_next(&iter))) {
> +                       int bind = GELF_ST_BIND(sym->sym.st_info);
> +                       struct symbol *found, tmp = {
> +                               .name = sym->name,
> +                       };
> +                       unsigned long *offset;
> +
> +                       found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
> +                       if (!found)
> +                               continue;
> +
> +                       offset = &offsets[found->idx];
> +                       if (*offset > 0) {
> +                               /* same offset, no problem */
> +                               if (*offset == elf_sym_offset(sym))
> +                                       continue;
> +                               /* handle multiple matches */
> +                               if (found->bind != STB_WEAK && bind != STB_WEAK) {
> +                                       /* Only accept one non-weak bind. */
> +                                       pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",

typo: found

but also wouldn't sym->name and found->name be always the same? Maybe
log sym->name, previous *offset and newly calculated
elf_sym_offset(sym) instead?

> +                                               sym->name, found->name, binary_path);
> +                                       err = -LIBBPF_ERRNO__FORMAT;

I'd minimize using those custom libbpf-only errors, why not -ESRCH here?

> +                                       goto out;
> +                               } else if (bind == STB_WEAK) {
> +                                       /* already have a non-weak bind, and
> +                                        * this is a weak bind, so ignore.
> +                                        */
> +                                       continue;
> +                               }
> +                       } else {
> +                               cnt_done++;
> +                       }
> +                       *offset = elf_sym_offset(sym);

maybe remember elf_sym_offset() result in a variable? you are using it
in two (and with my suggestion above it will be three) places already

> +                       found->bind = bind;
> +               }
> +       }
> +
> +       if (cnt != cnt_done) {
> +               err = -ENOENT;
> +               goto out;
> +       }
> +
> +       *poffsets = offsets;
> +
> +out:
> +       free(symbols);
> +       if (err)
> +               free(offsets);
> +       elf_close(&elf_fd);
> +       return err;
> +}
> diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
> index c763ac35a85e..026c7b378727 100644
> --- a/tools/lib/bpf/libbpf_elf.h
> +++ b/tools/lib/bpf/libbpf_elf.h
> @@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);
>  long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
>  long elf_find_func_offset_from_file(const char *binary_path, const char *name);
>
> +int elf_resolve_syms_offsets(const char *binary_path, int cnt,
> +                            const char **syms, unsigned long **poffsets);
>  #endif /* *__LIBBPF_LIBBPF_ELF_H */
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding elf_resolve_pattern_offsets function that looks up
> offsets for symbols specified by pattern argument.
>
> The 'pattern' argument allows wildcards (*?' supported).
>
> Offsets are returned in allocated array together with its
> size and needs to be released by the caller.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf.c          |  2 +-
>  tools/lib/bpf/libbpf_elf.h      |  3 ++
>  tools/lib/bpf/libbpf_internal.h |  1 +
>  4 files changed, 62 insertions(+), 1 deletion(-)
>
> diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644
> --- a/tools/lib/bpf/elf.c
> +++ b/tools/lib/bpf/elf.c
> @@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
>         elf_close(&elf_fd);
>         return err;
>  }
> +

same, leave comment that caller should free offsets on success?

> +int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
> +                               unsigned long **poffsets, size_t *pcnt)
> +{
> +       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
> +       unsigned long *offsets = NULL;
> +       size_t cap = 0, cnt = 0;
> +       struct elf_fd elf_fd;
> +       int err = 0, i;
> +
> +       err = elf_open(binary_path, &elf_fd);
> +       if (err)
> +               return err;
> +
> +       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> +               struct elf_sym_iter iter;
> +               struct elf_sym *sym;
> +
> +               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
> +               if (err) {
> +                       if (err == -ENOENT)
> +                               continue;
> +                       goto out;
> +               }

ditto, minimize nesting, please

> +
> +               while ((sym = elf_sym_iter_next(&iter))) {
> +                       if (!glob_match(sym->name, pattern))
> +                               continue;
> +
> +                       err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
> +                                               cnt + 1);
> +                       if (err)
> +                               goto out;
> +
> +                       offsets[cnt++] = elf_sym_offset(sym);
> +               }
> +
> +               /* If we found anything in the first symbol section,
> +                * do not search others to avoid duplicates.

DYNSYM is going to have only exposed symbols, so for this pattern
matching, maybe it's best to start with SYMTAB and only fallback to
DYNSYM if we didn't find anything in SYMTAB (more realistically it
would be that SYMTAB section is missing, so we fallback to DYNSYM;
otherwise neither DYNSYM nor SYMTAB will have matching symbols, most
probably, but that's minor)

other than that, LGTM

> +                */
> +               if (cnt)
> +                       break;
> +       }
> +
> +       if (cnt) {
> +               *poffsets = offsets;
> +               *pcnt = cnt;
> +       } else {
> +               err = -ENOENT;
> +       }
> +
> +out:
> +       if (err)
> +               free(offsets);
> +       elf_close(&elf_fd);
> +       return err;
> +}
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 093add8124d8..f33ef7cb1adc 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog,
>  }
>
>  /* Adapted from perf/util/string.c */
> -static bool glob_match(const char *str, const char *pat)
> +bool glob_match(const char *str, const char *pat)
>  {
>         while (*str && *pat && *pat != '*') {
>                 if (*pat == '?') {      /* Matches any single character */
> diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
> index 026c7b378727..0c75d3b2398b 100644
> --- a/tools/lib/bpf/libbpf_elf.h
> +++ b/tools/lib/bpf/libbpf_elf.h
> @@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);
>
>  int elf_resolve_syms_offsets(const char *binary_path, int cnt,
>                              const char **syms, unsigned long **poffsets);
> +
> +int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
> +                                unsigned long **poffsets, size_t *pcnt);
>  #endif /* *__LIBBPF_LIBBPF_ELF_H */
> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index e4d05662a96c..7d75b92e531a 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
> @@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)
>  #define PROG_LOAD_ATTEMPTS 5
>  int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);
>
> +bool glob_match(const char *str, const char *pat);
>  #endif /* __LIBBPF_LIBBPF_INTERNAL_H */
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding bpf_program__attach_uprobe_multi function that
> allows to attach multiple uprobes with uprobe_multi link.
>
> The user can specify uprobes with direct arguments:
>
>   binary_path/func_pattern/pid
>
> or with struct bpf_uprobe_multi_opts opts argument fields:
>
>   const char **syms;
>   const unsigned long *offsets;
>   const unsigned long *ref_ctr_offsets;
>   const __u64 *cookies;
>
> User can specify 2 mutually exclusive set of inputs:
>
>  1) use only path/func_pattern/pid arguments
>
>  2) use path/pid with allowed combinations of:
>     syms/offsets/ref_ctr_offsets/cookies/cnt
>
>     - syms and offsets are mutually exclusive
>     - ref_ctr_offsets and cookies are optional
>
> Any other usage results in error.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/libbpf.c   | 122 +++++++++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf.h   |  27 +++++++++
>  tools/lib/bpf/libbpf.map |   1 +
>  3 files changed, 150 insertions(+)
>
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index f33ef7cb1adc..b942f248038e 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)
>         return -ENOENT;
>  }
>
> +struct bpf_link *
> +bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
> +                                pid_t pid,
> +                                const char *path,
> +                                const char *func_pattern,
> +                                const struct bpf_uprobe_multi_opts *opts)
> +{
> +       const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL;
> +       LIBBPF_OPTS(bpf_link_create_opts, lopts);
> +       unsigned long *resolved_offsets = NULL;
> +       int err = 0, link_fd, prog_fd;
> +       struct bpf_link *link = NULL;
> +       char errmsg[STRERR_BUFSIZE];
> +       char full_path[PATH_MAX];
> +       const __u64 *cookies;
> +       const char **syms;
> +       bool has_pattern;
> +       bool retprobe;
> +       size_t cnt;
> +
> +       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
> +               return libbpf_err_ptr(-EINVAL);
> +
> +       syms = OPTS_GET(opts, syms, NULL);
> +       offsets = OPTS_GET(opts, offsets, NULL);
> +       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
> +       cookies = OPTS_GET(opts, cookies, NULL);
> +       cnt = OPTS_GET(opts, cnt, 0);
> +
> +       /*
> +        * User can specify 2 mutually exclusive set of inputs:
> +        *
> +        * 1) use only path/func_pattern/pid arguments
> +        *
> +        * 2) use path/pid with allowed combinations of:
> +        *    syms/offsets/ref_ctr_offsets/cookies/cnt
> +        *
> +        *    - syms and offsets are mutually exclusive
> +        *    - ref_ctr_offsets and cookies are optional
> +        *
> +        * Any other usage results in error.
> +        */
> +
> +       if (!path && !func_pattern && !cnt)

weird, I'd expect separate if (!path) return error (already bad,
regardless of func_pattern or cnt)

then if (!func_pattern && cnt == 0) return error

> +               return libbpf_err_ptr(-EINVAL);
> +       if (func_pattern && !path)
> +               return libbpf_err_ptr(-EINVAL);
> +
> +       has_pattern = path && func_pattern;

this and above check must be some leftovers from previous version.
path should always be present. and so you don't need has_pattern
variable, just use "func_pattern" check

> +
> +       if (has_pattern) {
> +               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
> +                       return libbpf_err_ptr(-EINVAL);
> +       } else {
> +               if (!cnt)
> +                       return libbpf_err_ptr(-EINVAL);
> +               if (!!syms == !!offsets)
> +                       return libbpf_err_ptr(-EINVAL);
> +       }
> +
> +       if (has_pattern) {
> +               if (!strchr(path, '/')) {
> +                       err = resolve_full_path(path, full_path, sizeof(full_path));
> +                       if (err) {
> +                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
> +                                       prog->name, path, err);
> +                               return libbpf_err_ptr(err);
> +                       }
> +                       path = full_path;
> +               }
> +
> +               err = elf_resolve_pattern_offsets(path, func_pattern,
> +                                                 &resolved_offsets, &cnt);
> +               if (err < 0)
> +                       return libbpf_err_ptr(err);
> +               offsets = resolved_offsets;
> +       } else if (syms) {
> +               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
> +               if (err < 0)
> +                       return libbpf_err_ptr(err);
> +               offsets = resolved_offsets;

you can extract this common error checking and `offsets =
resolved_offsets;` to after if, it's common for both branches
> +       }
> +
> +       retprobe = OPTS_GET(opts, retprobe, false);
> +
> +       lopts.uprobe_multi.path = path;
> +       lopts.uprobe_multi.offsets = offsets;
> +       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
> +       lopts.uprobe_multi.cookies = cookies;
> +       lopts.uprobe_multi.cnt = cnt;
> +       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;

retprobe is another unnecessary var, just inline check here to keep it simpler

> +
> +       if (pid == 0)
> +               pid = getpid();
> +       if (pid > 0)
> +               lopts.uprobe_multi.pid = pid;
> +
> +       link = calloc(1, sizeof(*link));
> +       if (!link) {
> +               err = -ENOMEM;
> +               goto error;
> +       }
> +       link->detach = &bpf_link__detach_fd;
> +
> +       prog_fd = bpf_program__fd(prog);
> +       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
> +       if (link_fd < 0) {
> +               err = -errno;
> +               pr_warn("prog '%s': failed to attach: %s\n",

"failed to attach multi-uprobe"? We probably should have added "failed
to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as
well?

> +                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
> +               goto error;
> +       }
> +       link->fd = link_fd;
> +       free(resolved_offsets);
> +       return link;
> +
> +error:
> +       free(resolved_offsets);
> +       free(link);
> +       return libbpf_err_ptr(err);
> +}
> +
>  LIBBPF_API struct bpf_link *
>  bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
>                                 const char *binary_path, size_t func_offset,
> diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
> index 754da73c643b..7c218f610210 100644
> --- a/tools/lib/bpf/libbpf.h
> +++ b/tools/lib/bpf/libbpf.h
> @@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
>                                       const char *pattern,
>                                       const struct bpf_kprobe_multi_opts *opts);
>
> +struct bpf_uprobe_multi_opts {
> +       /* size of this struct, for forward/backward compatibility */
> +       size_t sz;
> +       /* array of function symbols to attach */

attach to?

> +       const char **syms;
> +       /* array of function addresses to attach */

attach to?

> +       const unsigned long *offsets;
> +       /* array of refctr offsets to attach */

we don't really attach to ref counters, so maybe "optional, array of
associated ref counter offsets" or something along those lines ?

> +       const unsigned long *ref_ctr_offsets;
> +       /* array of user-provided values fetchable through bpf_get_attach_cookie */

"array of associated BPF cookies"? we can't keep explaining what BPF
cookie is in every possible API :)

> +       const __u64 *cookies;
> +       /* number of elements in syms/addrs/cookies arrays */
> +       size_t cnt;
> +       /* create return uprobes */
> +       bool retprobe;
> +       size_t :0;
> +};
> +
> +#define bpf_uprobe_multi_opts__last_field retprobe
> +
> +LIBBPF_API struct bpf_link *
> +bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
> +                                pid_t pid,
> +                                const char *binary_path,
> +                                const char *func_pattern,
> +                                const struct bpf_uprobe_multi_opts *opts);
> +

ok, let's be good citizens and add documentation for this new API.
Those comments about valid combinations belong here as well. Please
take a look at existing doccomments for the format and conventions.
Thanks!

>  struct bpf_ksyscall_opts {
>         /* size of this struct, for forward/backward compatibility */
>         size_t sz;
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index 7521a2fb7626..d8d11ea6c35e 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -395,4 +395,5 @@ LIBBPF_1.2.0 {
>  LIBBPF_1.3.0 {
>         global:
>                 bpf_obj_pin_opts;
> +               bpf_program__attach_uprobe_multi;
>  } LIBBPF_1.2.0;
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 14/26] libbpf: Add support for u[ret]probe.multi[.s] program sections

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding support for several uprobe_multi program sections
> to allow auto attach of multi_uprobe programs.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
>

LGTM

Acked-by: Andrii Nakryiko <andrii@kernel.org>

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index b942f248038e..06092b9752f1 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
>  static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
>  static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
>  static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
> +static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
>  static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
>  static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
>
> @@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {
>         SEC_DEF("uretprobe.s+",         KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
>         SEC_DEF("kprobe.multi+",        KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
>         SEC_DEF("kretprobe.multi+",     KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
> +       SEC_DEF("uprobe.multi+",        KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
> +       SEC_DEF("uretprobe.multi+",     KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
> +       SEC_DEF("uprobe.multi.s+",      KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
> +       SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
>         SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
>         SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
>         SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
> @@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
>         return libbpf_get_error(*link);
>  }
>
> +static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
> +{
> +       char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
> +       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
> +       int n, ret = -EINVAL;
> +
> +       *link = NULL;
> +
> +       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%ms",
> +                  &probe_type, &binary_path, &func_name);
> +       switch (n) {
> +       case 1:
> +               /* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */
> +               ret = 0;
> +               break;
> +       case 3:
> +               opts.retprobe = strcmp(probe_type, "uretprobe.multi") == 0;
> +               *link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
> +               ret = libbpf_get_error(*link);
> +               break;
> +       default:
> +               pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name,
> +                       prog->sec_name);
> +               break;
> +       }
> +       free(probe_type);
> +       free(binary_path);
> +       free(func_name);
> +       return ret;
> +}
> +
>  static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz,
>                                          const char *binary_path, uint64_t offset)
>  {
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding uprobe-multi link detection. It will be used later in
> bpf_program__attach_usdt function to check and use uprobe_multi
> link over standard uprobe links.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf_internal.h |  2 ++
>  2 files changed, 37 insertions(+)
>
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 06092b9752f1..4f61f9dc1748 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
>         return link_fd < 0 && err == -EBADF;
>  }
>
> +static int probe_uprobe_multi_link(void)
> +{
> +       LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
> +               .expected_attach_type = BPF_TRACE_UPROBE_MULTI,
> +       );
> +       LIBBPF_OPTS(bpf_link_create_opts, link_opts);
> +       struct bpf_insn insns[] = {
> +               BPF_MOV64_IMM(BPF_REG_0, 0),
> +               BPF_EXIT_INSN(),
> +       };
> +       unsigned long offset = 0;
> +       int prog_fd, link_fd;
> +
> +       prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
> +                               insns, ARRAY_SIZE(insns), &load_opts);
> +       if (prog_fd < 0)
> +               return -errno;
> +
> +       /* create single uprobe on offset 0 in current process */
> +       link_opts.uprobe_multi.path = "/proc/self/exe";
> +       link_opts.uprobe_multi.offsets = &offset;
> +       link_opts.uprobe_multi.cnt = 1;
> +
> +       link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
> +

so I'd like us to avoid successfully attaching anything. This might
have unintended consequences (e.g., unintentionally breaking backing
huge pages into normal pages, just because we happen to successfully
attach briefly). So let's work on feature detection that fails to
create a link, but does it in a way that we know that the feature
itself is supported by the kernel

Some ideas we could do:

1. Pass invalid file (e.g., just root, "/" as path), but modify
kernel-side logic to return not -EINVAL, but -EBADF (and I think it
would be good to do this anyway). Then expect -EBADF as a signal that
the feature is supported.

2. Also, we can return -EPROTO instead of -EINVAL on invalid
combination of paramers or something like that

I'd start with -EBADF change.

In general, we should write kernel-side code in such a way that allows
simple and efficient feature-detection. We shouldn't repeat the
nightmare of memcg-based mem accounting :(

> +       if (link_fd >= 0)
> +               close(link_fd);
> +       close(prog_fd);
> +
> +       return link_fd >= 0;
> +}
> +
>  static int probe_kern_bpf_cookie(void)
>  {
>         struct bpf_insn insns[] = {
> @@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
>         [FEAT_SYSCALL_WRAPPER] = {
>                 "Kernel using syscall wrapper", probe_kern_syscall_wrapper,
>         },
> +       [FEAT_UPROBE_MULTI_LINK] = {
> +               "BPF uprobe multi link support", probe_uprobe_multi_link,

nit: BPF multi-uprobe link support

> +       },
>  };
>
>  bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index 7d75b92e531a..9c04b3fe1207 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
> @@ -354,6 +354,8 @@ enum kern_feature_id {
>         FEAT_BTF_ENUM64,
>         /* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
>         FEAT_SYSCALL_WRAPPER,
> +       /* BPF uprobe_multi link support */

same, multi-uprobe link support



> +       FEAT_UPROBE_MULTI_LINK,
>         __FEAT_CNT,
>  };
>
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 02/26] bpf: Add multi uprobe link

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding new multi uprobe link that allows to attach bpf program
> to multiple uprobes.
>
> Uprobes to attach are specified via new link_create uprobe_multi
> union:
>
>   struct {
>           __u32           flags;
>           __u32           cnt;
>           __aligned_u64   path;
>           __aligned_u64   offsets;
>           __aligned_u64   ref_ctr_offsets;
>   } uprobe_multi;
>
> Uprobes are defined for single binary specified in path and multiple
> calling sites specified in offsets array with optional reference
> counters specified in ref_ctr_offsets array. All specified arrays
> have length of 'cnt'.
>
> The 'flags' supports single bit for now that marks the uprobe as
> return probe.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  include/linux/trace_events.h   |   6 +
>  include/uapi/linux/bpf.h       |  14 ++
>  kernel/bpf/syscall.c           |  14 +-
>  kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
>  tools/include/uapi/linux/bpf.h |  14 ++
>  5 files changed, 282 insertions(+), 3 deletions(-)
>

[...]

> +       flags = attr->link_create.uprobe_multi.flags;
> +       if (flags & ~BPF_F_UPROBE_MULTI_RETURN)
> +               return -EINVAL;
> +
> +       /*
> +        * path, offsets and cnt are mandatory,
> +        * ref_ctr_offsets is optional
> +        */
> +       upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
> +       uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
> +       cnt = attr->link_create.uprobe_multi.cnt;
> +
> +       if (!upath || !uoffsets || !cnt)
> +               return -EINVAL;

see below for -EBADF, but we can also, additionally, return -EPROTO
here, for example?

> +
> +       uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
> +
> +       name = strndup_user(upath, PATH_MAX);
> +       if (IS_ERR(name)) {
> +               err = PTR_ERR(name);
> +               return err;
> +       }
> +
> +       err = kern_path(name, LOOKUP_FOLLOW, &path);
> +       kfree(name);
> +       if (err)
> +               return err;
> +
> +       if (!d_is_reg(path.dentry)) {
> +               err = -EINVAL;

as I mentioned in another patch, -EBADF here for feature detection
(and it makes sense by itself, probably)

> +               goto error_path_put;
> +       }
> +
> +       err = -ENOMEM;
> +
> +       link = kzalloc(sizeof(*link), GFP_KERNEL);
> +       uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL);
> +
> +       if (!uprobes || !link)
> +               goto error_free;
> +
> +       if (uref_ctr_offsets) {
> +               ref_ctr_offsets = kvcalloc(cnt, sizeof(*ref_ctr_offsets), GFP_KERNEL);
> +               if (!ref_ctr_offsets)
> +                       goto error_free;
> +       }
> +

[...]

Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding support for usdt_manager_attach_usdt to use uprobe_multi
> link to attach to usdt probes.
>
> The uprobe_multi support is detected before the usdt program is
> loaded and its expected_attach_type is set accordingly.
>
> If uprobe_multi support is detected the usdt_manager_attach_usdt
> gathers uprobes info and calls bpf_program__attach_uprobe to
> create all needed uprobes.
>
> If uprobe_multi support is not detected the old behaviour stays.
>
> Also adding usdt.s program section for sleepable usdt probes.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/lib/bpf/libbpf.c | 13 +++++--
>  tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
>  2 files changed, 75 insertions(+), 16 deletions(-)
>
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 4f61f9dc1748..e234c2e860f9 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -365,6 +365,8 @@ enum sec_def_flags {
>         SEC_SLEEPABLE = 8,
>         /* BPF program support non-linear XDP buffer */
>         SEC_XDP_FRAGS = 16,
> +       /* Setup proper attach type for usdt probes. */
> +       SEC_USDT = 32,
>  };
>
>  struct bpf_sec_def {
> @@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
>         if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
>                 opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
>
> +       /* special check for usdt to use uprobe_multi link */
> +       if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))

please pass prog->obj to kernel_supports(), it will be especially
important later with BPF token stuff

> +               prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
> +
>         if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
>                 int btf_obj_fd = 0, btf_type_id = 0, err;
>                 const char *attach_name;
> @@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
>         if (!insns || !insns_cnt)
>                 return -EINVAL;
>
> -       load_attr.expected_attach_type = prog->expected_attach_type;
>         if (kernel_supports(obj, FEAT_PROG_NAME))
>                 prog_name = prog->name;
>         load_attr.attach_prog_fd = prog->attach_prog_fd;
> @@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
>                 insns_cnt = prog->insns_cnt;
>         }
>
> +       /* allow prog_prepare_load_fn to change expected_attach_type */
> +       load_attr.expected_attach_type = prog->expected_attach_type;
> +
>         if (obj->gen_loader) {
>                 bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
>                                    license, insns, insns_cnt, &load_attr,
> @@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
>         SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
>         SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
>         SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
> -       SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
> +       SEC_DEF("usdt+",                KPROBE, 0, SEC_USDT, attach_usdt),
> +       SEC_DEF("usdt.s+",              KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),

spaces around |

>         SEC_DEF("tc",                   SCHED_CLS, 0, SEC_NONE),
>         SEC_DEF("classifier",           SCHED_CLS, 0, SEC_NONE),
>         SEC_DEF("action",               SCHED_ACT, 0, SEC_NONE),
> diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
> index 9fa883ebc0bd..6ff66a8eaf85 100644
> --- a/tools/lib/bpf/usdt.c
> +++ b/tools/lib/bpf/usdt.c
> @@ -809,6 +809,8 @@ struct bpf_link_usdt {
>                 long abs_ip;
>                 struct bpf_link *link;
>         } *uprobes;
> +
> +       struct bpf_link *multi_link;
>  };
>
>  static int bpf_link_usdt_detach(struct bpf_link *link)
> @@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)
>         struct usdt_manager *man = usdt_link->usdt_man;
>         int i;
>
> +       /* When having multi_link, uprobe_cnt is 0 */

misplaced comment, move down to for() loop?

> +       bpf_link__destroy(usdt_link->multi_link);
> +
>         for (i = 0; i < usdt_link->uprobe_cnt; i++) {
>                 /* detach underlying uprobe link */
>                 bpf_link__destroy(usdt_link->uprobes[i].link);
> @@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
>                                           const char *usdt_provider, const char *usdt_name,
>                                           __u64 usdt_cookie)
>  {
> +       unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;
>         int i, err, spec_map_fd, ip_map_fd;
>         LIBBPF_OPTS(bpf_uprobe_opts, opts);
>         struct hashmap *specs_hash = NULL;
>         struct bpf_link_usdt *link = NULL;
>         struct usdt_target *targets = NULL;
> +       __u64 *cookies = NULL;
>         struct elf_fd elf_fd;
>         size_t target_cnt;
>
> @@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
>         link->link.detach = &bpf_link_usdt_detach;
>         link->link.dealloc = &bpf_link_usdt_dealloc;
>
> -       link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
> -       if (!link->uprobes) {
> -               err = -ENOMEM;
> -               goto err_out;
> +       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

see how we feature-detect has_sema_refcnt and has_bpf_cookie, let's do
the same with UPROBE_MULTI_LINK, detect once, remember, consistently
use it (it also matters later for BPF token)

> +               offsets = calloc(target_cnt, sizeof(*offsets));
> +               cookies = calloc(target_cnt, sizeof(*cookies));
> +               ref_ctr_offsets = calloc(target_cnt, sizeof(*ref_ctr_offsets));
> +
> +               if (!offsets || !ref_ctr_offsets || !cookies) {
> +                       err = -ENOMEM;
> +                       goto err_out;
> +               }
> +       } else {
> +               link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
> +               if (!link->uprobes) {
> +                       err = -ENOMEM;
> +                       goto err_out;
> +               }
>         }
>
>         for (i = 0; i < target_cnt; i++) {
> @@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
>                         goto err_out;
>                 }
>
> -               opts.ref_ctr_offset = target->sema_off;
> -               opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
> -               uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
> -                                                             target->rel_ip, &opts);
> -               err = libbpf_get_error(uprobe_link);
> +               if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
> +                       offsets[i] = target->rel_ip;
> +                       ref_ctr_offsets[i] = target->sema_off;
> +                       cookies[i] = spec_id;
> +               } else {
> +                       opts.ref_ctr_offset = target->sema_off;
> +                       opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
> +                       uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
> +                                                                     target->rel_ip, &opts);
> +                       err = libbpf_get_error(uprobe_link);
> +                       if (err) {
> +                               pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
> +                                       i, usdt_provider, usdt_name, path, err);
> +                               goto err_out;
> +                       }
> +
> +                       link->uprobes[i].link = uprobe_link;
> +                       link->uprobes[i].abs_ip = target->abs_ip;
> +                       link->uprobe_cnt++;
> +               }
> +       }
> +
> +       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

same as above, we should feature-detect once per usdt_manager while we
have associated bpf_object

> +               LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi,
> +                       .cnt = target_cnt,
> +                       .offsets = offsets,
> +                       .ref_ctr_offsets = ref_ctr_offsets,
> +                       .cookies = cookies,
> +               );
> +
> +               link->multi_link = bpf_program__attach_uprobe_multi(prog, pid, path,
> +                                                                   NULL, &opts_multi);
> +               err = libbpf_get_error(link->multi_link);

let's not use libbpf_get_error() in new code, there is no need, just
`err = -errno` and `if (!link->multi_link)`

>                 if (err) {
> -                       pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
> -                               i, usdt_provider, usdt_name, path, err);
> +                       pr_warn("usdt: failed to attach uprobe multi for '%s:%s' in '%s': %d\n",
> +                               usdt_provider, usdt_name, path, err);
>                         goto err_out;
>                 }
>
> -               link->uprobes[i].link = uprobe_link;
> -               link->uprobes[i].abs_ip = target->abs_ip;
> -               link->uprobe_cnt++;
> +               free(offsets);
> +               free(ref_ctr_offsets);
> +               free(cookies);
>         }
>
>         free(targets);
> @@ -1061,6 +1107,10 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
>         return &link->link;
>
>  err_out:
> +       free(offsets);
> +       free(ref_ctr_offsets);
> +       free(cookies);
> +
>         if (link)
>                 bpf_link__destroy(&link->link);
>         free(targets);
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding uprobe_multi test for bpf_program__attach_uprobe_multi
> attach function.
>
> Testing attachment using glob patterns and via bpf_uprobe_multi_opts
> paths/syms fields.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
>  1 file changed, 71 insertions(+)
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> index 5cd1116bbb62..f97a68871e73 100644
> --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> @@ -69,8 +69,79 @@ static void test_skel_api(void)
>         uprobe_multi__destroy(skel);
>  }
>
> +static void
> +test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
> +{
> +       struct bpf_link *link1 = NULL, *link2 = NULL;
> +       struct bpf_link *link3 = NULL, *link4 = NULL;
> +       struct uprobe_multi *skel = NULL;
> +
> +       skel = uprobe_multi__open_and_load();
> +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
> +               goto cleanup;
> +
> +       opts->retprobe = false;
> +       link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
> +                                                     binary, pattern, opts);
> +       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
> +               goto cleanup;
> +
> +       opts->retprobe = true;
> +       link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
> +                                                     binary, pattern, opts);
> +       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
> +               goto cleanup;
> +
> +       opts->retprobe = false;
> +       link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
> +                                                     binary, pattern, opts);
> +       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))

link3

> +               goto cleanup;
> +
> +       opts->retprobe = true;
> +       link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
> +                                                     binary, pattern, opts);
> +       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))

link4

> +               goto cleanup;
> +
> +       uprobe_multi_test_run(skel);
> +
> +cleanup:
> +       bpf_link__destroy(link4);
> +       bpf_link__destroy(link3);
> +       bpf_link__destroy(link2);
> +       bpf_link__destroy(link1);

you could have used
skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep}
"containers" to not manage destruction of these links manually

> +       uprobe_multi__destroy(skel);
> +}
> +
> +static void test_attach_api_pattern(void)
> +{
> +       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
> +
> +       test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
> +       test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
> +}
> +

[...]

Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding uprobe_multi test for bpf_link_create attach function.
>
> Testing attachment using the struct bpf_link_create_opts.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
>  1 file changed, 68 insertions(+)
>

[...]

> +       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
> +       prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
> +       link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> +       if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
> +               goto cleanup;
> +
> +       opts.kprobe_multi.flags = 0;
> +       prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
> +       link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> +       if (!ASSERT_GE(link1_fd, 0, "link3_fd"))

link3_fd

> +               goto cleanup;
> +
> +       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
> +       prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
> +       link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> +       if (!ASSERT_GE(link2_fd, 0, "link4_fd"))

link4_fd

> +               goto cleanup;
> +       uprobe_multi_test_run(skel);
> +
> +cleanup:
> +       if (link1_fd >= 0)
> +               close(link1_fd);
> +       if (link2_fd >= 0)
> +               close(link2_fd);
> +       if (link3_fd >= 0)
> +               close(link3_fd);
> +       if (link4_fd >= 0)
> +               close(link4_fd);
> +
> +       uprobe_multi__destroy(skel);
> +       free(offsets);
> +}
> +
>  void test_uprobe_multi_test(void)
>  {
>         if (test__start_subtest("skel_api"))
> @@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
>                 test_attach_api_pattern();
>         if (test__start_subtest("attach_api_syms"))
>                 test_attach_api_syms();
> +       if (test__start_subtest("link_api"))
> +               test_link_api();
>  }
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 20/26] selftests/bpf: Add uprobe_multi test program

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding uprobe_multi test program that defines 50k uprobe_multi_func_*
> functions and will serve as attach point for uprobe_multi bench test
> in following patch.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/testing/selftests/bpf/Makefile       |  5 ++
>  tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++
>  2 files changed, 58 insertions(+)
>  create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index ad6b585e0d7c..acf7c9a29082 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko      \
>                        $(OUTPUT)/liburandom_read.so                     \
>                        $(OUTPUT)/xdp_synproxy                           \
>                        $(OUTPUT)/sign-file                              \
> +                      $(OUTPUT)/uprobe_multi                           \

would adding all this to urandom_read be too bad in terms of size and
performance? I'm just not sure we won't yet more custom binaries and
rules in Makefile

>                        ima_setup.sh                                     \
>                        verify_sig_setup.sh                              \
>                        $(wildcard progs/btf_dump_test_case_*.c)         \
> @@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o
>         $(call msg,BINARY,,$@)
>         $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
>
> +$(OUTPUT)/uprobe_multi: uprobe_multi.c
> +       $(call msg,BINARY,,$@)
> +       $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
> +
>  EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \
>         prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
>         feature bpftool                                                 \
> diff --git a/tools/testing/selftests/bpf/uprobe_multi.c b/tools/testing/selftests/bpf/uprobe_multi.c
> new file mode 100644
> index 000000000000..115a7f6cebfa
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/uprobe_multi.c
> @@ -0,0 +1,53 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <stdio.h>
> +
> +#define __PASTE(a, b) a##b
> +#define PASTE(a, b) __PASTE(a, b)
> +
> +#define NAME(name, idx) PASTE(name, idx)
> +
> +#define DEF(name, idx)  int NAME(name, idx)(void) { return 0; }
> +#define CALL(name, idx) NAME(name, idx)();
> +
> +#define F(body, name, idx) body(name, idx)
> +
> +#define F10(body, name, idx) \
> +       F(body, PASTE(name, idx), 0) F(body, PASTE(name, idx), 1) F(body, PASTE(name, idx), 2) \
> +       F(body, PASTE(name, idx), 3) F(body, PASTE(name, idx), 4) F(body, PASTE(name, idx), 5) \
> +       F(body, PASTE(name, idx), 6) F(body, PASTE(name, idx), 7) F(body, PASTE(name, idx), 8) \
> +       F(body, PASTE(name, idx), 9)
> +
> +#define F100(body, name, idx) \
> +       F10(body, PASTE(name, idx), 0) F10(body, PASTE(name, idx), 1) F10(body, PASTE(name, idx), 2) \
> +       F10(body, PASTE(name, idx), 3) F10(body, PASTE(name, idx), 4) F10(body, PASTE(name, idx), 5) \
> +       F10(body, PASTE(name, idx), 6) F10(body, PASTE(name, idx), 7) F10(body, PASTE(name, idx), 8) \
> +       F10(body, PASTE(name, idx), 9)
> +
> +#define F1000(body, name, idx) \
> +       F100(body, PASTE(name, idx), 0) F100(body, PASTE(name, idx), 1) F100(body, PASTE(name, idx), 2) \
> +       F100(body, PASTE(name, idx), 3) F100(body, PASTE(name, idx), 4) F100(body, PASTE(name, idx), 5) \
> +       F100(body, PASTE(name, idx), 6) F100(body, PASTE(name, idx), 7) F100(body, PASTE(name, idx), 8) \
> +       F100(body, PASTE(name, idx), 9)
> +
> +#define F10000(body, name, idx) \
> +       F1000(body, PASTE(name, idx), 0) F1000(body, PASTE(name, idx), 1) F1000(body, PASTE(name, idx), 2) \
> +       F1000(body, PASTE(name, idx), 3) F1000(body, PASTE(name, idx), 4) F1000(body, PASTE(name, idx), 5) \
> +       F1000(body, PASTE(name, idx), 6) F1000(body, PASTE(name, idx), 7) F1000(body, PASTE(name, idx), 8) \
> +       F1000(body, PASTE(name, idx), 9)
> +
> +F10000(DEF, uprobe_multi_func_, 0)
> +F10000(DEF, uprobe_multi_func_, 1)
> +F10000(DEF, uprobe_multi_func_, 2)
> +F10000(DEF, uprobe_multi_func_, 3)
> +F10000(DEF, uprobe_multi_func_, 4)
> +
> +int main(void)
> +{
> +       F10000(CALL, uprobe_multi_func_, 0)
> +       F10000(CALL, uprobe_multi_func_, 1)
> +       F10000(CALL, uprobe_multi_func_, 2)
> +       F10000(CALL, uprobe_multi_func_, 3)
> +       F10000(CALL, uprobe_multi_func_, 4)
> +       return 0;
> +}
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:38 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding test that attaches 50k uprobes in uprobe_multi binary.
>
> After the attach is done we run the binary and make sure we
> get proper amount of hits.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
>  .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
>  2 files changed, 65 insertions(+)
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> index fd858636b8b0..547d46965d70 100644
> --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> @@ -202,6 +202,60 @@ static void test_link_api(void)
>         free(offsets);
>  }
>
> +static inline __u64 get_time_ns(void)
> +{
> +       struct timespec t;
> +
> +       clock_gettime(CLOCK_MONOTONIC, &t);
> +       return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
> +}
> +

hmm.. I would expect we have this helper somewhere in common headers.
If not, we should probably move all such helpers into one header and
use it everywhere

> +static void test_bench_attach_uprobe(void)
> +{
> +       long attach_start_ns, attach_end_ns;
> +       long detach_start_ns, detach_end_ns;
> +       double attach_delta, detach_delta;
> +       struct uprobe_multi *skel = NULL;
> +       struct bpf_program *prog;
> +       int err;
> +
> +       skel = uprobe_multi__open();
> +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
> +               goto cleanup;
> +
> +       bpf_object__for_each_program(prog, skel->obj)
> +               bpf_program__set_autoload(prog, false);
> +
> +       bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);

I don't get why you bothered adding this test_uprobe_bench into
progs/uprobe_multi and go through this manual auto-load
setting/resetting, instead of just having test_uprobe_bench in a
separate skeleton?...

> +
> +       err = uprobe_multi__load(skel);
> +       if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
> +               goto cleanup;
> +
> +       attach_start_ns = get_time_ns();
> +
> +       err = uprobe_multi__attach(skel);
> +       if (!ASSERT_OK(err, "uprobe_multi__attach"))
> +               goto cleanup;
> +
> +       attach_end_ns = get_time_ns();
> +
> +       system("./uprobe_multi");
> +
> +       ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
> +
> +cleanup:
> +       detach_start_ns = get_time_ns();
> +       uprobe_multi__destroy(skel);
> +       detach_end_ns = get_time_ns();
> +
> +       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
> +       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
> +
> +       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
> +       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);

and for us lazy folks, what are the numbers you see on your machine?

> +}
> +
>  void test_uprobe_multi_test(void)
>  {
>         if (test__start_subtest("skel_api"))
> @@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
>                 test_attach_api_syms();
>         if (test__start_subtest("link_api"))
>                 test_link_api();
> +       if (test__start_subtest("bench_uprobe"))
> +               test_bench_attach_uprobe();
>  }
> diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
> index 1eeb9b7b9cad..cd73139dc881 100644
> --- a/tools/testing/selftests/bpf/progs/uprobe_multi.c
> +++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
> @@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
>         uprobe_multi_check(ctx, true, true);
>         return 0;
>  }
> +
> +int count;
> +
> +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
> +int test_uprobe_bench(struct pt_regs *ctx)
> +{
> +       count++;
> +       return 0;
> +}
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 22/26] selftests/bpf: Add usdt_multi test program

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:38 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding usdt_multi test program that defines 50k usdts and will
> serve as attach point for uprobe_multi usdt bench test in
> following patch.
>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  tools/testing/selftests/bpf/Makefile     |  5 +++++
>  tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++
>  2 files changed, 29 insertions(+)
>  create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
>
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index acf7c9a29082..9762467cf0ba 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko      \
>                        $(OUTPUT)/xdp_synproxy                           \
>                        $(OUTPUT)/sign-file                              \
>                        $(OUTPUT)/uprobe_multi                           \
> +                      $(OUTPUT)/usdt_multi                             \

that's a bit too much, if you can't put everything into urandom_read,
let's at least combine uprobe_multi and usdt_multi

>                        ima_setup.sh                                     \
>                        verify_sig_setup.sh                              \
>                        $(wildcard progs/btf_dump_test_case_*.c)         \
> @@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c
>         $(call msg,BINARY,,$@)
>         $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
>
> +$(OUTPUT)/usdt_multi: usdt_multi.c
> +       $(call msg,BINARY,,$@)
> +       $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
> +
>  EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \
>         prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
>         feature bpftool                                                 \
> diff --git a/tools/testing/selftests/bpf/usdt_multi.c b/tools/testing/selftests/bpf/usdt_multi.c
> new file mode 100644
> index 000000000000..fedf856bad2b
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/usdt_multi.c
> @@ -0,0 +1,24 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <sdt.h>
> +
> +#define PROBE STAP_PROBE(test, usdt);
> +
> +#define PROBE10    PROBE PROBE PROBE PROBE PROBE \
> +                  PROBE PROBE PROBE PROBE PROBE
> +#define PROBE100   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \
> +                  PROBE10 PROBE10 PROBE10 PROBE10 PROBE10
> +#define PROBE1000  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \
> +                  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100
> +#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \
> +                  PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000
> +
> +int main(void)
> +{
> +       PROBE10000
> +       PROBE10000
> +       PROBE10000
> +       PROBE10000
> +       PROBE10000
> +       return 0;
> +}
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test

[Andrii Nakryiko]

On Fri, Jun 30, 2023 at 1:38 AM Jiri Olsa <jolsa@kernel.org> wrote:
>
> Adding test that attaches 50k usdt probes in usdt_multi binary.
>
> After the attach is done we run the binary and make sure we get
> proper amount of hits.
>
> With current uprobes:
>
>   # perf stat --null ./test_progs -n 254/6
>   #254/6   uprobe_multi_test/bench_usdt:OK
>   #254     uprobe_multi_test:OK
>   Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
>
>    Performance counter stats for './test_progs -n 254/6':
>
>       1353.659680562 seconds time elapsed
>

20 minutes, I commend your patience!

> With uprobe_multi link:
>
>   # perf stat --null ./test_progs -n 254/6
>   #254/6   uprobe_multi_test/bench_usdt:OK
>   #254     uprobe_multi_test:OK
>   Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
>
>    Performance counter stats for './test_progs -n 254/6':
>
>          0.322046364 seconds time elapsed

This is an impressive speed up, especially taking into account that
there is no batch attachment magic that we had to do for kprobes. Very
nice!

>
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  .../bpf/prog_tests/uprobe_multi_test.c        | 50 +++++++++++++++++++
>  .../selftests/bpf/progs/uprobe_multi_usdt.c   | 16 ++++++
>  2 files changed, 66 insertions(+)
>  create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> index 547d46965d70..b12dc1f992e5 100644
> --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> @@ -4,6 +4,7 @@
>  #include <test_progs.h>
>  #include "uprobe_multi.skel.h"
>  #include "bpf/libbpf_elf.h"
> +#include "uprobe_multi_usdt.skel.h"
>
>  static char test_data[] = "test_data";
>
> @@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)
>         printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
>  }
>
> +static void test_bench_attach_usdt(void)
> +{
> +       struct uprobe_multi_usdt *skel = NULL;
> +       long attach_start_ns, attach_end_ns;
> +       long detach_start_ns, detach_end_ns;
> +       double attach_delta, detach_delta;
> +       struct bpf_program *prog;
> +       int err;
> +
> +       skel = uprobe_multi_usdt__open();
> +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
> +               goto cleanup;
> +
> +       bpf_object__for_each_program(prog, skel->obj)
> +               bpf_program__set_autoload(prog, false);
> +
> +       bpf_program__set_autoload(skel->progs.usdt0, true);

there is nothing else in that skeleton, why this set_autoload() business?

> +
> +       err = uprobe_multi_usdt__load(skel);
> +       if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
> +               goto cleanup;
> +
> +       attach_start_ns = get_time_ns();
> +
> +       skel->links.usdt0 = bpf_program__attach_usdt(skel->progs.usdt0, -1, "./usdt_multi",
> +                                                    "test", "usdt", NULL);
> +       if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
> +               goto cleanup;
> +
> +       attach_end_ns = get_time_ns();
> +
> +       system("./usdt_multi");
> +
> +       ASSERT_EQ(skel->bss->count, 50000, "usdt_count");
> +
> +cleanup:
> +       detach_start_ns = get_time_ns();
> +       uprobe_multi_usdt__destroy(skel);
> +       detach_end_ns = get_time_ns();
> +
> +       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
> +       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
> +
> +       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
> +       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
> +}
> +
>  void test_uprobe_multi_test(void)
>  {
>         if (test__start_subtest("skel_api"))
> @@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)
>                 test_link_api();
>         if (test__start_subtest("bench_uprobe"))
>                 test_bench_attach_uprobe();
> +       if (test__start_subtest("bench_usdt"))
> +               test_bench_attach_usdt();
>  }
> diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
> new file mode 100644
> index 000000000000..9e1c33d0bd2f
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
> @@ -0,0 +1,16 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include "vmlinux.h"
> +#include <bpf/bpf_helpers.h>
> +#include <bpf/usdt.bpf.h>
> +
> +char _license[] SEC("license") = "GPL";
> +
> +int count;
> +
> +SEC("usdt")
> +int usdt0(struct pt_regs *ctx)
> +{
> +       count++;
> +       return 0;
> +}
> --
> 2.41.0
>

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Jiri Olsa]

On Thu, Jul 06, 2023 at 03:29:08PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding support for bpf_get_func_ip helper being called from
> > ebpf program attached by uprobe_multi link.
> >
> > It returns the ip of the uprobe.
> >
> > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
> >  1 file changed, 30 insertions(+), 3 deletions(-)
> >
> 
> A slight aside related to bpf_get_func_ip() support in
> uprobe/uretprobe. We just had a conversation with Alastair and Jordan
> (cc'ed) about bpftrace and using bpf_get_func_ip() there with
> uretprobes, and it seems like it doesn't work.
> 
> Is that intentional or we just missed that bpf_get_func_ip() doesn't
> work with uprobes/uretprobes? Do you think it would be hard to add
> support for them for bpf_get_func_ip()? It's a very useful helper,
> would be nice to have it working in all cases where it has meaningful
> behavior (and I think it does for uprobe and uretprobe).

I'm not sure we discussed at the time we added this helper,
but I see same problem as we had with kprobes where we need
to know if the probe is on the start of the symbol

we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's
get_func_ip helper version:

	BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) 
	{
		struct kprobe *kp = kprobe_running();

		if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
			return 0;

		return get_entry_ip((uintptr_t)kp->addr);
	}

I don't think we can have same flag for uprobe, we don't have
the binary symbol data as we have for kernel

I guess the app needs to store regs->ip and match it against symbol
addresses in user space

jirka


> 
> Thanks!
> 
> > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > index 4ef51fd0497f..f5a41c1604b8 100644
> > --- a/kernel/trace/bpf_trace.c
> > +++ b/kernel/trace/bpf_trace.c
> > @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
> >  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
> >
> 
> [...]

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Andrii Nakryiko]

On Mon, Jul 10, 2023 at 12:24 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Thu, Jul 06, 2023 at 03:29:08PM -0700, Andrii Nakryiko wrote:
> > On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > >
> > > Adding support for bpf_get_func_ip helper being called from
> > > ebpf program attached by uprobe_multi link.
> > >
> > > It returns the ip of the uprobe.
> > >
> > > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > ---
> > >  kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
> > >  1 file changed, 30 insertions(+), 3 deletions(-)
> > >
> >
> > A slight aside related to bpf_get_func_ip() support in
> > uprobe/uretprobe. We just had a conversation with Alastair and Jordan
> > (cc'ed) about bpftrace and using bpf_get_func_ip() there with
> > uretprobes, and it seems like it doesn't work.
> >
> > Is that intentional or we just missed that bpf_get_func_ip() doesn't
> > work with uprobes/uretprobes? Do you think it would be hard to add
> > support for them for bpf_get_func_ip()? It's a very useful helper,
> > would be nice to have it working in all cases where it has meaningful
> > behavior (and I think it does for uprobe and uretprobe).
>
> I'm not sure we discussed at the time we added this helper,
> but I see same problem as we had with kprobes where we need
> to know if the probe is on the start of the symbol
>
> we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's
> get_func_ip helper version:
>
>         BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
>         {
>                 struct kprobe *kp = kprobe_running();
>
>                 if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
>                         return 0;
>
>                 return get_entry_ip((uintptr_t)kp->addr);
>         }
>
> I don't think we can have same flag for uprobe, we don't have
> the binary symbol data as we have for kernel

what if we just return whatever was the IP where uprobe/uretprobe
interrupt instruction was installed at? I haven't tried what does
regs->ip contain for u*ret*probe, though, if it already has the IP
where we installed uprobe itself, it might be ok as is. But still,
feels like it would be nice to have bpf_get_func_ip() working for
uprobe/uretprobe (even if semantics might differ for uprobes not at
the entry of the function).

>
> I guess the app needs to store regs->ip and match it against symbol
> addresses in user space
>
> jirka
>
>
> >
> > Thanks!
> >
> > > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > > index 4ef51fd0497f..f5a41c1604b8 100644
> > > --- a/kernel/trace/bpf_trace.c
> > > +++ b/kernel/trace/bpf_trace.c
> > > @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
> > >  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
> > >
> >
> > [...]

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Jiri Olsa]

On Mon, Jul 10, 2023 at 10:55:36AM -0700, Andrii Nakryiko wrote:
> On Mon, Jul 10, 2023 at 12:24 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> >
> > On Thu, Jul 06, 2023 at 03:29:08PM -0700, Andrii Nakryiko wrote:
> > > On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > > >
> > > > Adding support for bpf_get_func_ip helper being called from
> > > > ebpf program attached by uprobe_multi link.
> > > >
> > > > It returns the ip of the uprobe.
> > > >
> > > > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> > > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > > ---
> > > >  kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
> > > >  1 file changed, 30 insertions(+), 3 deletions(-)
> > > >
> > >
> > > A slight aside related to bpf_get_func_ip() support in
> > > uprobe/uretprobe. We just had a conversation with Alastair and Jordan
> > > (cc'ed) about bpftrace and using bpf_get_func_ip() there with
> > > uretprobes, and it seems like it doesn't work.
> > >
> > > Is that intentional or we just missed that bpf_get_func_ip() doesn't
> > > work with uprobes/uretprobes? Do you think it would be hard to add
> > > support for them for bpf_get_func_ip()? It's a very useful helper,
> > > would be nice to have it working in all cases where it has meaningful
> > > behavior (and I think it does for uprobe and uretprobe).
> >
> > I'm not sure we discussed at the time we added this helper,
> > but I see same problem as we had with kprobes where we need
> > to know if the probe is on the start of the symbol
> >
> > we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's
> > get_func_ip helper version:
> >
> >         BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
> >         {
> >                 struct kprobe *kp = kprobe_running();
> >
> >                 if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
> >                         return 0;
> >
> >                 return get_entry_ip((uintptr_t)kp->addr);
> >         }
> >
> > I don't think we can have same flag for uprobe, we don't have
> > the binary symbol data as we have for kernel
> 
> what if we just return whatever was the IP where uprobe/uretprobe
> interrupt instruction was installed at? I haven't tried what does
> regs->ip contain for u*ret*probe, though, if it already has the IP
> where we installed uprobe itself, it might be ok as is. But still,

the entry uprobe gets the IP of the probed instruction, but the uretprobe
gets IP of the next instruction after probed func retun - like 4c57f0 in
the example below when uprobe is placed on trigger_func_get_func_ip

	  4c57eb:       e8 4b fe ff ff          call   4c563b <trigger_func_get_func_ip>
	  4c57f0:       eb 01                   jmp    4c57f3 <test_uprobe+0x1b1>

> feels like it would be nice to have bpf_get_func_ip() working for
> uprobe/uretprobe (even if semantics might differ for uprobes not at
> the entry of the function).

but it looks like we actually have the original uprobe address stored in
current->utask->vaddr before bpf program is executed (in uretprobe_dispatcher)
so we should be able to get that address from there in uretprobe's get_func_ip
helper function, I'll check

I don't mind the semantic change for uprobe's get_func_ip, because I think
it's unlikely we will be able to change it in future to return the real
function entry

jirka

> 
> >
> > I guess the app needs to store regs->ip and match it against symbol
> > addresses in user space
> >
> > jirka
> >
> >
> > >
> > > Thanks!
> > >
> > > > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > > > index 4ef51fd0497f..f5a41c1604b8 100644
> > > > --- a/kernel/trace/bpf_trace.c
> > > > +++ b/kernel/trace/bpf_trace.c
> > > > @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
> > > >  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
> > > >
> > >
> > > [...]

Re: [PATCHv3 bpf-next 02/26] bpf: Add multi uprobe link

[Jiri Olsa]

On Thu, Jul 06, 2023 at 03:34:10PM -0700, Andrii Nakryiko wrote:

SNIP

> > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> > index 60a9d59beeab..a236139f08ce 100644
> > --- a/include/uapi/linux/bpf.h
> > +++ b/include/uapi/linux/bpf.h
> > @@ -1036,6 +1036,7 @@ enum bpf_attach_type {
> >         BPF_LSM_CGROUP,
> >         BPF_STRUCT_OPS,
> >         BPF_NETFILTER,
> > +       BPF_TRACE_UPROBE_MULTI,
> >         __MAX_BPF_ATTACH_TYPE
> >  };
> >
> > @@ -1053,6 +1054,7 @@ enum bpf_link_type {
> >         BPF_LINK_TYPE_KPROBE_MULTI = 8,
> >         BPF_LINK_TYPE_STRUCT_OPS = 9,
> >         BPF_LINK_TYPE_NETFILTER = 10,
> > +       BPF_LINK_TYPE_UPROBE_MULTI = 11,
> >
> >         MAX_BPF_LINK_TYPE,
> >  };
> > @@ -1170,6 +1172,11 @@ enum bpf_link_type {
> >   */
> >  #define BPF_F_KPROBE_MULTI_RETURN      (1U << 0)
> >
> > +/* link_create.uprobe_multi.flags used in LINK_CREATE command for
> > + * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
> > + */
> > +#define BPF_F_UPROBE_MULTI_RETURN      (1U << 0)
> > +
> 
> any reason why we don't use anonymous ENUMs for all these UAPI
> constants? When we need to use these flags from BPF side (e.g., for
> BPF LSM), having them as #defines will be a PITA, as they won't be
> present in vmlinux.h

ugh right, we already did that before.. will change

> 
> 
> >  /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
> >   * the following extensions:
> >   *
> > @@ -1579,6 +1586,13 @@ union bpf_attr {
> >                                 __s32           priority;
> >                                 __u32           flags;
> >                         } netfilter;
> > +                       struct {
> > +                               __u32           flags;
> > +                               __u32           cnt;
> 
> total nit, but I'd move it after path/offsets/ref_ctr_offsets, and
> make the order cnt (as it applies to previous two
> offsets/ref_ctr_offsets) and then flags last. Seems like more logical
> order, but totally subjective

ok

> 
> > +                               __aligned_u64   path;
> > +                               __aligned_u64   offsets;
> > +                               __aligned_u64   ref_ctr_offsets;
> > +                       } uprobe_multi;
> >                 };
> >         } link_create;
> >
> > diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> > index 9046ad0f9b4e..3b0582a64ce4 100644
> > --- a/kernel/bpf/syscall.c
> > +++ b/kernel/bpf/syscall.c
> > @@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
> >
> >  /* Clean up bpf_link and corresponding anon_inode file and FD. After
> >   * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
> > - * anon_inode's release() call. This helper marksbpf_link as
> > + * anon_inode's release() call. This helper marks bpf_link as
> >   * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
> >   * is not decremented, it's the responsibility of a calling code that failed
> >   * to complete bpf_link initialization.
> > + * This helper eventually calls link's dealloc callback, but does not call
> > + * link's release callback.
> 
> Thanks for clarifying comments!
> 
> >   */
> >  void bpf_link_cleanup(struct bpf_link_primer *primer)
> >  {
> > @@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
> >                 if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
> >                     attach_type != BPF_TRACE_KPROBE_MULTI)
> >                         return -EINVAL;
> > +               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
> > +                   attach_type != BPF_TRACE_UPROBE_MULTI)
> > +                       return -EINVAL;
> >                 if (attach_type != BPF_PERF_EVENT &&
> > -                   attach_type != BPF_TRACE_KPROBE_MULTI)
> > +                   attach_type != BPF_TRACE_KPROBE_MULTI &&
> > +                   attach_type != BPF_TRACE_UPROBE_MULTI)
> 
> if this keeps growing, we should think about having a switch in a
> switch to not repeat BPF_TRACE_UPROBE_MULTI and BPF_TRACE_KPROBE_MULTI
> twice

ok

SNIP

> > +       for (i = 0; i < cnt; i++) {
> > +               err = uprobe_register_refctr(d_real_inode(link->path.dentry),
> > +                                            uprobes[i].offset,
> > +                                            ref_ctr_offsets ? ref_ctr_offsets[i] : 0,
> > +                                            &uprobes[i].consumer);
> > +               if (err) {
> > +                       bpf_uprobe_unregister(&path, uprobes, i);
> > +                       bpf_link_cleanup(&link_primer);
> > +                       kvfree(ref_ctr_offsets);
> 
> are we missing path_put() in this error handling path? so maybe goto
> error_path_put here instead of return?

aaaah right path_put needs to go to dealloc callback :-\ will change, thanks

jirka

Re: [PATCHv3 bpf-next 02/26] bpf: Add multi uprobe link

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:22:29PM -0700, Andrii Nakryiko wrote:

SNIP

> > +       flags = attr->link_create.uprobe_multi.flags;
> > +       if (flags & ~BPF_F_UPROBE_MULTI_RETURN)
> > +               return -EINVAL;
> > +
> > +       /*
> > +        * path, offsets and cnt are mandatory,
> > +        * ref_ctr_offsets is optional
> > +        */
> > +       upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
> > +       uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
> > +       cnt = attr->link_create.uprobe_multi.cnt;
> > +
> > +       if (!upath || !uoffsets || !cnt)
> > +               return -EINVAL;
> 
> see below for -EBADF, but we can also, additionally, return -EPROTO
> here, for example?
> 
> > +
> > +       uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
> > +
> > +       name = strndup_user(upath, PATH_MAX);
> > +       if (IS_ERR(name)) {
> > +               err = PTR_ERR(name);
> > +               return err;
> > +       }
> > +
> > +       err = kern_path(name, LOOKUP_FOLLOW, &path);
> > +       kfree(name);
> > +       if (err)
> > +               return err;
> > +
> > +       if (!d_is_reg(path.dentry)) {
> > +               err = -EINVAL;
> 
> as I mentioned in another patch, -EBADF here for feature detection
> (and it makes sense by itself, probably)

yes, I like this place better, also -EBADF error makes more sense in here

thanks,
jirka

> 
> > +               goto error_path_put;
> > +       }
> > +
> > +       err = -ENOMEM;
> > +
> > +       link = kzalloc(sizeof(*link), GFP_KERNEL);
> > +       uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL);
> > +
> > +       if (!uprobes || !link)
> > +               goto error_free;
> > +
> > +       if (uref_ctr_offsets) {
> > +               ref_ctr_offsets = kvcalloc(cnt, sizeof(*ref_ctr_offsets), GFP_KERNEL);
> > +               if (!ref_ctr_offsets)
> > +                       goto error_free;
> > +       }
> > +
> 
> [...]

Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions

[Jiri Olsa]

On Thu, Jul 06, 2023 at 04:09:29PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding elf_open/elf_close functions and using it in
> > elf_find_func_offset_from_file function. It will be
> > used in following changes to save some common code.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
> >  tools/lib/bpf/libbpf_elf.h |  8 ++++++
> >  tools/lib/bpf/usdt.c       | 31 ++++++--------------
> >  3 files changed, 56 insertions(+), 42 deletions(-)
> >
> 
> one nit below
> 
> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> 
> > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > index 2b62b4af28ce..74e35071d22e 100644
> > --- a/tools/lib/bpf/elf.c
> > +++ b/tools/lib/bpf/elf.c
> > @@ -11,6 +11,40 @@
> >
> >  #define STRERR_BUFSIZE  128
> >
> > +int elf_open(const char *binary_path, struct elf_fd *elf_fd)
> > +{
> > +       char errmsg[STRERR_BUFSIZE];
> > +       int fd, ret;
> > +       Elf *elf;
> > +
> > +       if (elf_version(EV_CURRENT) == EV_NONE) {
> > +               pr_warn("elf: failed to init libelf for %s\n", binary_path);
> > +               return -LIBBPF_ERRNO__LIBELF;
> > +       }
> > +       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
> > +       if (fd < 0) {
> > +               ret = -errno;
> > +               pr_warn("elf: failed to open %s: %s\n", binary_path,
> > +                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
> > +               return ret;
> > +       }
> > +       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
> > +       if (!elf) {
> > +               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
> > +               close(fd);
> > +               return -LIBBPF_ERRNO__FORMAT;
> > +       }
> > +       elf_fd->fd = fd;
> > +       elf_fd->elf = elf;
> > +       return 0;
> > +}
> > +
> > +void elf_close(struct elf_fd *elf_fd)
> > +{
> > +       elf_end(elf_fd->elf);
> > +       close(elf_fd->fd);
> 
> nit: I'd make elf_close() work correctly with a) NULL elf_fd and b)

right, ok

> NULL elf_fd->elf, just to never have to think about this

there's NULL check in elf_end

thanks,
jirka

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

[Jiri Olsa]

On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:

SNIP

> > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
> >  1 file changed, 117 insertions(+), 61 deletions(-)
> >
> 
> A bunch of nits, but overall looks good. Please address nits, and add my ack
> 
> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> 
> > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > index 74e35071d22e..fcce4bd2478f 100644
> > --- a/tools/lib/bpf/elf.c
> > +++ b/tools/lib/bpf/elf.c
> > @@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
> >         return NULL;
> >  }
> >
> > +struct elf_sym {
> > +       const char *name;
> > +       GElf_Sym sym;
> > +       GElf_Shdr sh;
> > +};
> > +
> 
> if we want to use elf_sym_iter outside of elf.c, this should be in
> libbpf_internal.h?

yes eventually, but all the helper functions using elf_sym_iter that
I added later are in elf.c, so there's no need atm

SNIP

> > +
> > +static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
> > +{
> > +       struct elf_sym *ret = &iter->sym;
> > +       GElf_Sym *sym = &ret->sym;
> > +       const char *name = NULL;
> > +       Elf_Scn *sym_scn;
> > +       size_t idx;
> > +
> > +       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
> > +               if (!gelf_getsym(iter->syms, idx, sym))
> > +                       continue;
> > +               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
> > +                       continue;
> > +               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
> > +               if (!name)
> > +                       continue;
> > +
> > +               /* Transform symbol's virtual address (absolute for
> > +                * binaries and relative for shared libs) into file
> > +                * offset, which is what kernel is expecting for
> > +                * uprobe/uretprobe attachment.
> > +                * See Documentation/trace/uprobetracer.rst for more
> > +                * details.
> > +                * This is done by looking up symbol's containing
> > +                * section's header and using iter's virtual address
> > +                * (sh_addr) and corresponding file offset (sh_offset)
> > +                * to transform sym.st_value (virtual address) into
> > +                * desired final file offset.
> > +                */
> 
> this comment is misplaced? we don't do the translation here

right, should be placed at the elf_sym_offset function

> 
> > +               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
> > +               if (!sym_scn)
> > +                       continue;
> > +               if (!gelf_getshdr(sym_scn, &ret->sh))
> > +                       continue;
> > +
> > +               iter->next_sym_idx = idx + 1;
> > +               ret->name = name;
> > +               return ret;
> > +       }
> > +
> > +       return NULL;
> > +}
> > +
> > +static unsigned long elf_sym_offset(struct elf_sym *sym)
> > +{
> > +       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
> > +}
> > +
> >  /* Find offset of function name in the provided ELF object. "binary_path" is
> >   * the path to the ELF binary represented by "elf", and only used for error
> >   * reporting matters. "name" matches symbol name or name@@LIB for library
> > @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
> >          * reported as a warning/error.
> >          */
> >         for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> > -               size_t nr_syms, strtabidx, idx;
> > -               Elf_Data *symbols = NULL;
> > -               Elf_Scn *scn = NULL;
> > +               struct elf_sym_iter iter;
> > +               struct elf_sym *sym;
> >                 int last_bind = -1;
> > -               const char *sname;
> > -               GElf_Shdr sh;
> > +               int curr_bind;
> 
> OCD nit:
> 
> $ rg 'curr(_|\b)' | wc -l
> 8
> $ rg 'cur(_|\b)' | wc -l
> 148
> 
> and those 8 I consider an unfortunate accident ;) let's standardize on
> using "cur" consistently

ok.. probably easier to make that change than treat ocd ;-)

> 
> >
> > -               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
> > -               if (!scn) {
> > -                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
> > -                                binary_path);
> > -                       continue;
> > -               }
> > -               if (!gelf_getshdr(scn, &sh))
> > -                       continue;
> > -               strtabidx = sh.sh_link;
> > -               symbols = elf_getdata(scn, 0);
> > -               if (!symbols) {
> > -                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
> > -                               binary_path, elf_errmsg(-1));
> > -                       ret = -LIBBPF_ERRNO__FORMAT;
> > +               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
> > +               if (ret) {
> > +                       if (ret == -ENOENT)
> > +                               continue;
> >                         goto out;
> 
> another styling nit: let's avoid unnecessary nesting of ifs:
> 
> if (ret == -ENOENT)
>     continue;
> if (ret)
>     goto out;
> 
> simple and clean

ok

> 
> 
> >                 }
> > -               nr_syms = symbols->d_size / sh.sh_entsize;
> > -
> > -               for (idx = 0; idx < nr_syms; idx++) {
> > -                       int curr_bind;
> > -                       GElf_Sym sym;
> > -                       Elf_Scn *sym_scn;
> > -                       GElf_Shdr sym_sh;
> > -
> > -                       if (!gelf_getsym(symbols, idx, &sym))
> > -                               continue;
> > -
> > -                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
> > -                               continue;
> > -
> > -                       sname = elf_strptr(elf, strtabidx, sym.st_name);
> > -                       if (!sname)
> > -                               continue;
> > -
> > -                       curr_bind = GELF_ST_BIND(sym.st_info);
> >
> > +               while ((sym = elf_sym_iter_next(&iter))) {
> >                         /* User can specify func, func@@LIB or func@@LIB_VERSION. */
> > -                       if (strncmp(sname, name, name_len) != 0)
> > +                       if (strncmp(sym->name, name, name_len) != 0)
> >                                 continue;
> >                         /* ...but we don't want a search for "foo" to match 'foo2" also, so any
> >                          * additional characters in sname should be of the form "@@LIB".
> >                          */
> > -                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
> > +                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
> >                                 continue;
> >
> > -                       if (ret >= 0) {
> > +                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
> > +
> > +                       if (ret > 0) {
> 
> used to be >=, why the change?

the original code initialized ret with -ENOENT and did not change
its value till this point, so the condition never triggered for
the first loop, but it did for the new code because we now use ret
earlier in:

	ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);

also the check makes sense to me only if ret > 0 .. when we find
a duplicate value for symbol

jirka

Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:20:42PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding uprobe-multi link detection. It will be used later in
> > bpf_program__attach_usdt function to check and use uprobe_multi
> > link over standard uprobe links.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
> >  tools/lib/bpf/libbpf_internal.h |  2 ++
> >  2 files changed, 37 insertions(+)
> >
> > diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> > index 06092b9752f1..4f61f9dc1748 100644
> > --- a/tools/lib/bpf/libbpf.c
> > +++ b/tools/lib/bpf/libbpf.c
> > @@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
> >         return link_fd < 0 && err == -EBADF;
> >  }
> >
> > +static int probe_uprobe_multi_link(void)
> > +{
> > +       LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
> > +               .expected_attach_type = BPF_TRACE_UPROBE_MULTI,
> > +       );
> > +       LIBBPF_OPTS(bpf_link_create_opts, link_opts);
> > +       struct bpf_insn insns[] = {
> > +               BPF_MOV64_IMM(BPF_REG_0, 0),
> > +               BPF_EXIT_INSN(),
> > +       };
> > +       unsigned long offset = 0;
> > +       int prog_fd, link_fd;
> > +
> > +       prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
> > +                               insns, ARRAY_SIZE(insns), &load_opts);
> > +       if (prog_fd < 0)
> > +               return -errno;
> > +
> > +       /* create single uprobe on offset 0 in current process */
> > +       link_opts.uprobe_multi.path = "/proc/self/exe";
> > +       link_opts.uprobe_multi.offsets = &offset;
> > +       link_opts.uprobe_multi.cnt = 1;
> > +
> > +       link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
> > +
> 
> so I'd like us to avoid successfully attaching anything. This might
> have unintended consequences (e.g., unintentionally breaking backing
> huge pages into normal pages, just because we happen to successfully
> attach briefly). So let's work on feature detection that fails to
> create a link, but does it in a way that we know that the feature
> itself is supported by the kernel
> 
> Some ideas we could do:
> 
> 1. Pass invalid file (e.g., just root, "/" as path), but modify
> kernel-side logic to return not -EINVAL, but -EBADF (and I think it
> would be good to do this anyway). Then expect -EBADF as a signal that
> the feature is supported.

ah ok, so -EBADF from inside uprobe_multi link setup code would mean
it's supported, anything else means it's not.. should work also for
old kernels, I don't think we have -EBADF in related paths, will check

> 
> 2. Also, we can return -EPROTO instead of -EINVAL on invalid
> combination of paramers or something like that
> 
> I'd start with -EBADF change.
> 
> In general, we should write kernel-side code in such a way that allows
> simple and efficient feature-detection. We shouldn't repeat the
> nightmare of memcg-based mem accounting :(
> 
> > +       if (link_fd >= 0)
> > +               close(link_fd);
> > +       close(prog_fd);
> > +
> > +       return link_fd >= 0;
> > +}
> > +
> >  static int probe_kern_bpf_cookie(void)
> >  {
> >         struct bpf_insn insns[] = {
> > @@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
> >         [FEAT_SYSCALL_WRAPPER] = {
> >                 "Kernel using syscall wrapper", probe_kern_syscall_wrapper,
> >         },
> > +       [FEAT_UPROBE_MULTI_LINK] = {
> > +               "BPF uprobe multi link support", probe_uprobe_multi_link,
> 
> nit: BPF multi-uprobe link support
> 
> > +       },
> >  };
> >
> >  bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
> > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > index 7d75b92e531a..9c04b3fe1207 100644
> > --- a/tools/lib/bpf/libbpf_internal.h
> > +++ b/tools/lib/bpf/libbpf_internal.h
> > @@ -354,6 +354,8 @@ enum kern_feature_id {
> >         FEAT_BTF_ENUM64,
> >         /* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
> >         FEAT_SYSCALL_WRAPPER,
> > +       /* BPF uprobe_multi link support */
> 
> same, multi-uprobe link support

ok, thanks

jirka

> 
> 
> 
> > +       FEAT_UPROBE_MULTI_LINK,
> >         __FEAT_CNT,
> >  };
> >
> > --
> > 2.41.0
> >

Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:29:35PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding support for usdt_manager_attach_usdt to use uprobe_multi
> > link to attach to usdt probes.
> >
> > The uprobe_multi support is detected before the usdt program is
> > loaded and its expected_attach_type is set accordingly.
> >
> > If uprobe_multi support is detected the usdt_manager_attach_usdt
> > gathers uprobes info and calls bpf_program__attach_uprobe to
> > create all needed uprobes.
> >
> > If uprobe_multi support is not detected the old behaviour stays.
> >
> > Also adding usdt.s program section for sleepable usdt probes.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/libbpf.c | 13 +++++--
> >  tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
> >  2 files changed, 75 insertions(+), 16 deletions(-)
> >
> > diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> > index 4f61f9dc1748..e234c2e860f9 100644
> > --- a/tools/lib/bpf/libbpf.c
> > +++ b/tools/lib/bpf/libbpf.c
> > @@ -365,6 +365,8 @@ enum sec_def_flags {
> >         SEC_SLEEPABLE = 8,
> >         /* BPF program support non-linear XDP buffer */
> >         SEC_XDP_FRAGS = 16,
> > +       /* Setup proper attach type for usdt probes. */
> > +       SEC_USDT = 32,
> >  };
> >
> >  struct bpf_sec_def {
> > @@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
> >         if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
> >                 opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
> >
> > +       /* special check for usdt to use uprobe_multi link */
> > +       if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))
> 
> please pass prog->obj to kernel_supports(), it will be especially
> important later with BPF token stuff

did not realize I have it in prog->obj ;-) ok

> 
> > +               prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
> > +
> >         if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
> >                 int btf_obj_fd = 0, btf_type_id = 0, err;
> >                 const char *attach_name;
> > @@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
> >         if (!insns || !insns_cnt)
> >                 return -EINVAL;
> >
> > -       load_attr.expected_attach_type = prog->expected_attach_type;
> >         if (kernel_supports(obj, FEAT_PROG_NAME))
> >                 prog_name = prog->name;
> >         load_attr.attach_prog_fd = prog->attach_prog_fd;
> > @@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
> >                 insns_cnt = prog->insns_cnt;
> >         }
> >
> > +       /* allow prog_prepare_load_fn to change expected_attach_type */
> > +       load_attr.expected_attach_type = prog->expected_attach_type;
> > +
> >         if (obj->gen_loader) {
> >                 bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
> >                                    license, insns, insns_cnt, &load_attr,
> > @@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
> >         SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
> >         SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
> >         SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
> > -       SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
> > +       SEC_DEF("usdt+",                KPROBE, 0, SEC_USDT, attach_usdt),
> > +       SEC_DEF("usdt.s+",              KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),
> 
> spaces around |

ook

> 
> >         SEC_DEF("tc",                   SCHED_CLS, 0, SEC_NONE),
> >         SEC_DEF("classifier",           SCHED_CLS, 0, SEC_NONE),
> >         SEC_DEF("action",               SCHED_ACT, 0, SEC_NONE),
> > diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
> > index 9fa883ebc0bd..6ff66a8eaf85 100644
> > --- a/tools/lib/bpf/usdt.c
> > +++ b/tools/lib/bpf/usdt.c
> > @@ -809,6 +809,8 @@ struct bpf_link_usdt {
> >                 long abs_ip;
> >                 struct bpf_link *link;
> >         } *uprobes;
> > +
> > +       struct bpf_link *multi_link;
> >  };
> >
> >  static int bpf_link_usdt_detach(struct bpf_link *link)
> > @@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)
> >         struct usdt_manager *man = usdt_link->usdt_man;
> >         int i;
> >
> > +       /* When having multi_link, uprobe_cnt is 0 */
> 
> misplaced comment, move down to for() loop?

right, will move

> 
> > +       bpf_link__destroy(usdt_link->multi_link);
> > +
> >         for (i = 0; i < usdt_link->uprobe_cnt; i++) {
> >                 /* detach underlying uprobe link */
> >                 bpf_link__destroy(usdt_link->uprobes[i].link);
> > @@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
> >                                           const char *usdt_provider, const char *usdt_name,
> >                                           __u64 usdt_cookie)
> >  {
> > +       unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;
> >         int i, err, spec_map_fd, ip_map_fd;
> >         LIBBPF_OPTS(bpf_uprobe_opts, opts);
> >         struct hashmap *specs_hash = NULL;
> >         struct bpf_link_usdt *link = NULL;
> >         struct usdt_target *targets = NULL;
> > +       __u64 *cookies = NULL;
> >         struct elf_fd elf_fd;
> >         size_t target_cnt;
> >
> > @@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
> >         link->link.detach = &bpf_link_usdt_detach;
> >         link->link.dealloc = &bpf_link_usdt_dealloc;
> >
> > -       link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
> > -       if (!link->uprobes) {
> > -               err = -ENOMEM;
> > -               goto err_out;
> > +       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
> 
> see how we feature-detect has_sema_refcnt and has_bpf_cookie, let's do
> the same with UPROBE_MULTI_LINK, detect once, remember, consistently
> use it (it also matters later for BPF token)

ok

> 
> > +               offsets = calloc(target_cnt, sizeof(*offsets));
> > +               cookies = calloc(target_cnt, sizeof(*cookies));
> > +               ref_ctr_offsets = calloc(target_cnt, sizeof(*ref_ctr_offsets));
> > +
> > +               if (!offsets || !ref_ctr_offsets || !cookies) {
> > +                       err = -ENOMEM;
> > +                       goto err_out;
> > +               }
> > +       } else {
> > +               link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
> > +               if (!link->uprobes) {
> > +                       err = -ENOMEM;
> > +                       goto err_out;
> > +               }
> >         }
> >
> >         for (i = 0; i < target_cnt; i++) {
> > @@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
> >                         goto err_out;
> >                 }
> >
> > -               opts.ref_ctr_offset = target->sema_off;
> > -               opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
> > -               uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
> > -                                                             target->rel_ip, &opts);
> > -               err = libbpf_get_error(uprobe_link);
> > +               if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
> > +                       offsets[i] = target->rel_ip;
> > +                       ref_ctr_offsets[i] = target->sema_off;
> > +                       cookies[i] = spec_id;
> > +               } else {
> > +                       opts.ref_ctr_offset = target->sema_off;
> > +                       opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
> > +                       uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
> > +                                                                     target->rel_ip, &opts);
> > +                       err = libbpf_get_error(uprobe_link);
> > +                       if (err) {
> > +                               pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
> > +                                       i, usdt_provider, usdt_name, path, err);
> > +                               goto err_out;
> > +                       }
> > +
> > +                       link->uprobes[i].link = uprobe_link;
> > +                       link->uprobes[i].abs_ip = target->abs_ip;
> > +                       link->uprobe_cnt++;
> > +               }
> > +       }
> > +
> > +       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
> 
> same as above, we should feature-detect once per usdt_manager while we
> have associated bpf_object

ook

> 
> > +               LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi,
> > +                       .cnt = target_cnt,
> > +                       .offsets = offsets,
> > +                       .ref_ctr_offsets = ref_ctr_offsets,
> > +                       .cookies = cookies,
> > +               );
> > +
> > +               link->multi_link = bpf_program__attach_uprobe_multi(prog, pid, path,
> > +                                                                   NULL, &opts_multi);
> > +               err = libbpf_get_error(link->multi_link);
> 
> let's not use libbpf_get_error() in new code, there is no need, just
> `err = -errno` and `if (!link->multi_link)`

ah right, did not see its comment that says it's no longer recomended,
will change

thanks,
jirka

Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function

[Jiri Olsa]

On Thu, Jul 06, 2023 at 08:48:13PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding elf_resolve_syms_offsets function that looks up
> > offsets for symbols specified in syms array argument.
> >
> > Offsets are returned in allocated array with the 'cnt' size,
> > that needs to be released by the caller.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
> >  tools/lib/bpf/libbpf_elf.h |   2 +
> >  2 files changed, 107 insertions(+)
> >
> > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > index fcce4bd2478f..7e2f3b2e1fb6 100644
> > --- a/tools/lib/bpf/elf.c
> > +++ b/tools/lib/bpf/elf.c
> > @@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
> >         elf_close(&elf_fd);
> >         return ret;
> >  }
> > +
> > +struct symbol {
> > +       const char *name;
> > +       int bind;
> > +       int idx;
> > +};
> > +
> > +static int symbol_cmp(const void *_a, const void *_b)
> > +{
> > +       const struct symbol *a = _a;
> > +       const struct symbol *b = _b;
> 
> please, let's not (over)use leading underscores, x/y, s1/s2, whatever

ok

> 
> > +
> > +       return strcmp(a->name, b->name);
> > +}
> > +
> 
> probably worth leaving a comment that the caller should free offsets on success?

ook

> 
> > +int elf_resolve_syms_offsets(const char *binary_path, int cnt,
> > +                            const char **syms, unsigned long **poffsets)
> > +{
> > +       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
> > +       int err = 0, i, cnt_done = 0;
> > +       unsigned long *offsets;
> > +       struct symbol *symbols;
> > +       struct elf_fd elf_fd;
> > +
> > +       err = elf_open(binary_path, &elf_fd);
> > +       if (err)
> > +               return err;
> > +
> > +       offsets = calloc(cnt, sizeof(*offsets));
> > +       symbols = calloc(cnt, sizeof(*symbols));
> > +
> > +       if (!offsets || !symbols) {
> > +               err = -ENOMEM;
> > +               goto out;
> > +       }
> > +
> > +       for (i = 0; i < cnt; i++) {
> > +               symbols[i].name = syms[i];
> > +               symbols[i].idx = i;
> > +       }
> > +
> > +       qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
> > +
> > +       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> > +               struct elf_sym_iter iter;
> > +               struct elf_sym *sym;
> > +
> > +               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
> > +               if (err) {
> > +                       if (err == -ENOENT)
> > +                               continue;
> > +                       goto out;
> > +               }
> 
> same nit, no need for nested ifs

ok

> > +
> > +               while ((sym = elf_sym_iter_next(&iter))) {
> > +                       int bind = GELF_ST_BIND(sym->sym.st_info);
> > +                       struct symbol *found, tmp = {
> > +                               .name = sym->name,
> > +                       };
> > +                       unsigned long *offset;
> > +
> > +                       found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
> > +                       if (!found)
> > +                               continue;
> > +
> > +                       offset = &offsets[found->idx];
> > +                       if (*offset > 0) {
> > +                               /* same offset, no problem */
> > +                               if (*offset == elf_sym_offset(sym))
> > +                                       continue;
> > +                               /* handle multiple matches */
> > +                               if (found->bind != STB_WEAK && bind != STB_WEAK) {
> > +                                       /* Only accept one non-weak bind. */
> > +                                       pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",
> 
> typo: found
> 
> but also wouldn't sym->name and found->name be always the same? Maybe
> log sym->name, previous *offset and newly calculated
> elf_sym_offset(sym) instead?

ok

> 
> > +                                               sym->name, found->name, binary_path);
> > +                                       err = -LIBBPF_ERRNO__FORMAT;
> 
> I'd minimize using those custom libbpf-only errors, why not -ESRCH here?

ok

> 
> > +                                       goto out;
> > +                               } else if (bind == STB_WEAK) {
> > +                                       /* already have a non-weak bind, and
> > +                                        * this is a weak bind, so ignore.
> > +                                        */
> > +                                       continue;
> > +                               }
> > +                       } else {
> > +                               cnt_done++;
> > +                       }
> > +                       *offset = elf_sym_offset(sym);
> 
> maybe remember elf_sym_offset() result in a variable? you are using it
> in two (and with my suggestion above it will be three) places already

ok


jirka

Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function

[Jiri Olsa]

On Thu, Jul 06, 2023 at 08:52:13PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:36 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding elf_resolve_pattern_offsets function that looks up
> > offsets for symbols specified by pattern argument.
> >
> > The 'pattern' argument allows wildcards (*?' supported).
> >
> > Offsets are returned in allocated array together with its
> > size and needs to be released by the caller.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
> >  tools/lib/bpf/libbpf.c          |  2 +-
> >  tools/lib/bpf/libbpf_elf.h      |  3 ++
> >  tools/lib/bpf/libbpf_internal.h |  1 +
> >  4 files changed, 62 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644
> > --- a/tools/lib/bpf/elf.c
> > +++ b/tools/lib/bpf/elf.c
> > @@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
> >         elf_close(&elf_fd);
> >         return err;
> >  }
> > +
> 
> same, leave comment that caller should free offsets on success?

yes, will add

> 
> > +int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
> > +                               unsigned long **poffsets, size_t *pcnt)
> > +{
> > +       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
> > +       unsigned long *offsets = NULL;
> > +       size_t cap = 0, cnt = 0;
> > +       struct elf_fd elf_fd;
> > +       int err = 0, i;
> > +
> > +       err = elf_open(binary_path, &elf_fd);
> > +       if (err)
> > +               return err;
> > +
> > +       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> > +               struct elf_sym_iter iter;
> > +               struct elf_sym *sym;
> > +
> > +               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
> > +               if (err) {
> > +                       if (err == -ENOENT)
> > +                               continue;
> > +                       goto out;
> > +               }
> 
> ditto, minimize nesting, please

ok

> 
> > +
> > +               while ((sym = elf_sym_iter_next(&iter))) {
> > +                       if (!glob_match(sym->name, pattern))
> > +                               continue;
> > +
> > +                       err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
> > +                                               cnt + 1);
> > +                       if (err)
> > +                               goto out;
> > +
> > +                       offsets[cnt++] = elf_sym_offset(sym);
> > +               }
> > +
> > +               /* If we found anything in the first symbol section,
> > +                * do not search others to avoid duplicates.
> 
> DYNSYM is going to have only exposed symbols, so for this pattern
> matching, maybe it's best to start with SYMTAB and only fallback to
> DYNSYM if we didn't find anything in SYMTAB (more realistically it
> would be that SYMTAB section is missing, so we fallback to DYNSYM;
> otherwise neither DYNSYM nor SYMTAB will have matching symbols, most
> probably, but that's minor)

makes sense, will switch

> 
> other than that, LGTM

thanks,
jirka

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:

SNIP

> > +       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
> > +               return libbpf_err_ptr(-EINVAL);
> > +
> > +       syms = OPTS_GET(opts, syms, NULL);
> > +       offsets = OPTS_GET(opts, offsets, NULL);
> > +       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
> > +       cookies = OPTS_GET(opts, cookies, NULL);
> > +       cnt = OPTS_GET(opts, cnt, 0);
> > +
> > +       /*
> > +        * User can specify 2 mutually exclusive set of inputs:
> > +        *
> > +        * 1) use only path/func_pattern/pid arguments
> > +        *
> > +        * 2) use path/pid with allowed combinations of:
> > +        *    syms/offsets/ref_ctr_offsets/cookies/cnt
> > +        *
> > +        *    - syms and offsets are mutually exclusive
> > +        *    - ref_ctr_offsets and cookies are optional
> > +        *
> > +        * Any other usage results in error.
> > +        */
> > +
> > +       if (!path && !func_pattern && !cnt)
> 
> weird, I'd expect separate if (!path) return error (already bad,
> regardless of func_pattern or cnt)
> 
> then if (!func_pattern && cnt == 0) return error
> 
> > +               return libbpf_err_ptr(-EINVAL);
> > +       if (func_pattern && !path)
> > +               return libbpf_err_ptr(-EINVAL);
> > +
> > +       has_pattern = path && func_pattern;
> 
> this and above check must be some leftovers from previous version.
> path should always be present. and so you don't need has_pattern
> variable, just use "func_pattern" check

hum, right, previous version had 2 paths, now there's just one,
I'll change that together with the suggested change above

> 
> > +
> > +       if (has_pattern) {
> > +               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
> > +                       return libbpf_err_ptr(-EINVAL);
> > +       } else {
> > +               if (!cnt)
> > +                       return libbpf_err_ptr(-EINVAL);
> > +               if (!!syms == !!offsets)
> > +                       return libbpf_err_ptr(-EINVAL);
> > +       }
> > +
> > +       if (has_pattern) {
> > +               if (!strchr(path, '/')) {
> > +                       err = resolve_full_path(path, full_path, sizeof(full_path));
> > +                       if (err) {
> > +                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
> > +                                       prog->name, path, err);
> > +                               return libbpf_err_ptr(err);
> > +                       }
> > +                       path = full_path;
> > +               }
> > +
> > +               err = elf_resolve_pattern_offsets(path, func_pattern,
> > +                                                 &resolved_offsets, &cnt);
> > +               if (err < 0)
> > +                       return libbpf_err_ptr(err);
> > +               offsets = resolved_offsets;
> > +       } else if (syms) {
> > +               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
> > +               if (err < 0)
> > +                       return libbpf_err_ptr(err);
> > +               offsets = resolved_offsets;
> 
> you can extract this common error checking and `offsets =
> resolved_offsets;` to after if, it's common for both branches

not sure what you mean in here, offsets can be also provided
by OPTS_GET(opts, offsets, NULL) earlier

> > +       }
> > +
> > +       retprobe = OPTS_GET(opts, retprobe, false);
> > +
> > +       lopts.uprobe_multi.path = path;
> > +       lopts.uprobe_multi.offsets = offsets;
> > +       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
> > +       lopts.uprobe_multi.cookies = cookies;
> > +       lopts.uprobe_multi.cnt = cnt;
> > +       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;
> 
> retprobe is another unnecessary var, just inline check here to keep it simpler

ok

> 
> > +
> > +       if (pid == 0)
> > +               pid = getpid();
> > +       if (pid > 0)
> > +               lopts.uprobe_multi.pid = pid;
> > +
> > +       link = calloc(1, sizeof(*link));
> > +       if (!link) {
> > +               err = -ENOMEM;
> > +               goto error;
> > +       }
> > +       link->detach = &bpf_link__detach_fd;
> > +
> > +       prog_fd = bpf_program__fd(prog);
> > +       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
> > +       if (link_fd < 0) {
> > +               err = -errno;
> > +               pr_warn("prog '%s': failed to attach: %s\n",
> 
> "failed to attach multi-uprobe"? We probably should have added "failed
> to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as
> well?

ook, will add

> 
> > +                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
> > +               goto error;
> > +       }
> > +       link->fd = link_fd;
> > +       free(resolved_offsets);
> > +       return link;
> > +
> > +error:
> > +       free(resolved_offsets);
> > +       free(link);
> > +       return libbpf_err_ptr(err);
> > +}
> > +
> >  LIBBPF_API struct bpf_link *
> >  bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
> >                                 const char *binary_path, size_t func_offset,
> > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
> > index 754da73c643b..7c218f610210 100644
> > --- a/tools/lib/bpf/libbpf.h
> > +++ b/tools/lib/bpf/libbpf.h
> > @@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
> >                                       const char *pattern,
> >                                       const struct bpf_kprobe_multi_opts *opts);
> >
> > +struct bpf_uprobe_multi_opts {
> > +       /* size of this struct, for forward/backward compatibility */
> > +       size_t sz;
> > +       /* array of function symbols to attach */
> 
> attach to?

ok

> 
> > +       const char **syms;
> > +       /* array of function addresses to attach */
> 
> attach to?

ook

> 
> > +       const unsigned long *offsets;
> > +       /* array of refctr offsets to attach */
> 
> we don't really attach to ref counters, so maybe "optional, array of
> associated ref counter offsets" or something along those lines ?

ok

> 
> > +       const unsigned long *ref_ctr_offsets;
> > +       /* array of user-provided values fetchable through bpf_get_attach_cookie */
> 
> "array of associated BPF cookies"? we can't keep explaining what BPF
> cookie is in every possible API :)

ook

> 
> > +       const __u64 *cookies;
> > +       /* number of elements in syms/addrs/cookies arrays */
> > +       size_t cnt;
> > +       /* create return uprobes */
> > +       bool retprobe;
> > +       size_t :0;
> > +};
> > +
> > +#define bpf_uprobe_multi_opts__last_field retprobe
> > +
> > +LIBBPF_API struct bpf_link *
> > +bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
> > +                                pid_t pid,
> > +                                const char *binary_path,
> > +                                const char *func_pattern,
> > +                                const struct bpf_uprobe_multi_opts *opts);
> > +
> 
> ok, let's be good citizens and add documentation for this new API.
> Those comments about valid combinations belong here as well. Please
> take a look at existing doccomments for the format and conventions.
> Thanks!

ok, will add

thanks,
jirka

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Jiri Olsa]

On Thu, Jul 06, 2023 at 04:02:22PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding new elf object that will contain elf related functions.
> > There's no functional change.
> >
> > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/Build        |   2 +-
> >  tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
> >  tools/lib/bpf/libbpf.c     | 186 +---------------------------------
> >  tools/lib/bpf/libbpf_elf.h |  11 +++
> >  4 files changed, 211 insertions(+), 186 deletions(-)
> >  create mode 100644 tools/lib/bpf/elf.c
> >  create mode 100644 tools/lib/bpf/libbpf_elf.h
> >
> 
> [...]
> 
> > diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
> > new file mode 100644
> > index 000000000000..1b652220fabf
> > --- /dev/null
> > +++ b/tools/lib/bpf/libbpf_elf.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
> > +
> > +#ifndef __LIBBPF_LIBBPF_ELF_H
> > +#define __LIBBPF_LIBBPF_ELF_H
> > +
> > +#include <libelf.h>
> > +
> > +long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
> > +long elf_find_func_offset_from_file(const char *binary_path, const char *name);
> > +
> > +#endif /* *__LIBBPF_LIBBPF_ELF_H */
> 
> we have libbpf_internal.h, let's put all this there for now, it's
> already all the internal stuff together, I don't know if separate
> header with few functions gives us much

there's more functions coming later in the patchset

	struct elf_fd {
		Elf *elf;
		int fd;
	};

	int elf_open(const char *binary_path, struct elf_fd *elf_fd);
	void elf_close(struct elf_fd *elf_fd);

	int elf_resolve_syms_offsets(const char *binary_path, int cnt,
				     const char **syms, unsigned long **poffsets);

	int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
					 unsigned long **poffsets, size_t *pcnt);


and there's probably more elf helpers to eventually move in:

	libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off);
	libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off);
	libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx);
	libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name);
	libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx);
	libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx);

	usdt.c:static int find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn)

	'struct elf_seg' stuff

	usdt.c:static int cmp_elf_segs(const void *_a, const void *_b)
	usdt.c:static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt)
	usdt.c:static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt)
	usdt.c:static struct elf_seg *find_elf_seg(struct elf_seg *segs, size_t seg_cnt, long virtaddr)
	usdt.c:static struct elf_seg *find_vma_seg(struct elf_seg *segs, size_t seg_cnt, long offset)


but I can add the new header file later in follow up changes when
we have more elf functions in

jirka

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Jiri Olsa]

On Thu, Jul 06, 2023 at 04:03:22PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding new elf object that will contain elf related functions.
> > There's no functional change.
> >
> > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  tools/lib/bpf/Build        |   2 +-
> >  tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
> >  tools/lib/bpf/libbpf.c     | 186 +---------------------------------
> >  tools/lib/bpf/libbpf_elf.h |  11 +++
> >  4 files changed, 211 insertions(+), 186 deletions(-)
> >  create mode 100644 tools/lib/bpf/elf.c
> >  create mode 100644 tools/lib/bpf/libbpf_elf.h
> >
> > diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build
> > index b8b0a6369363..2d0c282c8588 100644
> > --- a/tools/lib/bpf/Build
> > +++ b/tools/lib/bpf/Build
> > @@ -1,4 +1,4 @@
> >  libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
> >             netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
> >             btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
> > -           usdt.o zip.o
> > +           usdt.o zip.o elf.o
> > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > new file mode 100644
> > index 000000000000..2b62b4af28ce
> > --- /dev/null
> > +++ b/tools/lib/bpf/elf.c
> > @@ -0,0 +1,198 @@
> > +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> > +
> > +#include <libelf.h>
> > +#include <gelf.h>
> > +#include <fcntl.h>
> > +#include <linux/kernel.h>
> 
> do you know why we need linux/kernel.h include? is it to get __u32 and
> other typedefs?

it's for the ARRAY_SIZE macro

jirka

> 
> > +
> > +#include "libbpf_elf.h"
> > +#include "libbpf_internal.h"
> > +#include "str_error.h"
> > +
> > +#define STRERR_BUFSIZE  128
> > +
> 
> [...]

Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:32:15PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding uprobe_multi test for bpf_program__attach_uprobe_multi
> > attach function.
> >
> > Testing attachment using glob patterns and via bpf_uprobe_multi_opts
> > paths/syms fields.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
> >  1 file changed, 71 insertions(+)
> >
> > diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > index 5cd1116bbb62..f97a68871e73 100644
> > --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > @@ -69,8 +69,79 @@ static void test_skel_api(void)
> >         uprobe_multi__destroy(skel);
> >  }
> >
> > +static void
> > +test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
> > +{
> > +       struct bpf_link *link1 = NULL, *link2 = NULL;
> > +       struct bpf_link *link3 = NULL, *link4 = NULL;
> > +       struct uprobe_multi *skel = NULL;
> > +
> > +       skel = uprobe_multi__open_and_load();
> > +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
> > +               goto cleanup;
> > +
> > +       opts->retprobe = false;
> > +       link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
> > +                                                     binary, pattern, opts);
> > +       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
> > +               goto cleanup;
> > +
> > +       opts->retprobe = true;
> > +       link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
> > +                                                     binary, pattern, opts);
> > +       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
> > +               goto cleanup;
> > +
> > +       opts->retprobe = false;
> > +       link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
> > +                                                     binary, pattern, opts);
> > +       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
> 
> link3
> 
> > +               goto cleanup;
> > +
> > +       opts->retprobe = true;
> > +       link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
> > +                                                     binary, pattern, opts);
> > +       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
> 
> link4
> 
> > +               goto cleanup;
> > +
> > +       uprobe_multi_test_run(skel);
> > +
> > +cleanup:
> > +       bpf_link__destroy(link4);
> > +       bpf_link__destroy(link3);
> > +       bpf_link__destroy(link2);
> > +       bpf_link__destroy(link1);
> 
> you could have used
> skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep}
> "containers" to not manage destruction of these links manually

ah right, I keep forgetting we have those variables ready in there :-\
will change

thanks,
jirka

> 
> > +       uprobe_multi__destroy(skel);
> > +}
> > +
> > +static void test_attach_api_pattern(void)
> > +{
> > +       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
> > +
> > +       test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
> > +       test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
> > +}
> > +
> 
> [...]

Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:33:40PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:37 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding uprobe_multi test for bpf_link_create attach function.
> >
> > Testing attachment using the struct bpf_link_create_opts.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
> >  1 file changed, 68 insertions(+)
> >
> 
> [...]
> 
> > +       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
> > +       prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
> > +       link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> > +       if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
> > +               goto cleanup;
> > +
> > +       opts.kprobe_multi.flags = 0;
> > +       prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
> > +       link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> > +       if (!ASSERT_GE(link1_fd, 0, "link3_fd"))
> 
> link3_fd
> 
> > +               goto cleanup;
> > +
> > +       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
> > +       prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
> > +       link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
> > +       if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
> 
> link4_fd

right, will change

thanks,
jirka

> 
> > +               goto cleanup;
> > +       uprobe_multi_test_run(skel);
> > +
> > +cleanup:
> > +       if (link1_fd >= 0)
> > +               close(link1_fd);
> > +       if (link2_fd >= 0)
> > +               close(link2_fd);
> > +       if (link3_fd >= 0)
> > +               close(link3_fd);
> > +       if (link4_fd >= 0)
> > +               close(link4_fd);
> > +
> > +       uprobe_multi__destroy(skel);
> > +       free(offsets);
> > +}
> > +
> >  void test_uprobe_multi_test(void)
> >  {
> >         if (test__start_subtest("skel_api"))
> > @@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
> >                 test_attach_api_pattern();
> >         if (test__start_subtest("attach_api_syms"))
> >                 test_attach_api_syms();
> > +       if (test__start_subtest("link_api"))
> > +               test_link_api();
> >  }
> > --
> > 2.41.0
> >

Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:38:20PM -0700, Andrii Nakryiko wrote:
> On Fri, Jun 30, 2023 at 1:38 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > Adding test that attaches 50k uprobes in uprobe_multi binary.
> >
> > After the attach is done we run the binary and make sure we
> > get proper amount of hits.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
> >  .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
> >  2 files changed, 65 insertions(+)
> >
> > diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > index fd858636b8b0..547d46965d70 100644
> > --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
> > @@ -202,6 +202,60 @@ static void test_link_api(void)
> >         free(offsets);
> >  }
> >
> > +static inline __u64 get_time_ns(void)
> > +{
> > +       struct timespec t;
> > +
> > +       clock_gettime(CLOCK_MONOTONIC, &t);
> > +       return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
> > +}
> > +
> 
> hmm.. I would expect we have this helper somewhere in common headers.
> If not, we should probably move all such helpers into one header and
> use it everywhere

hum, now I see it's also defined in bench.h, I'll try to use it from
there ;-) also perhaps in kprobe multi test

> 
> > +static void test_bench_attach_uprobe(void)
> > +{
> > +       long attach_start_ns, attach_end_ns;
> > +       long detach_start_ns, detach_end_ns;
> > +       double attach_delta, detach_delta;
> > +       struct uprobe_multi *skel = NULL;
> > +       struct bpf_program *prog;
> > +       int err;
> > +
> > +       skel = uprobe_multi__open();
> > +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
> > +               goto cleanup;
> > +
> > +       bpf_object__for_each_program(prog, skel->obj)
> > +               bpf_program__set_autoload(prog, false);
> > +
> > +       bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);
> 
> I don't get why you bothered adding this test_uprobe_bench into
> progs/uprobe_multi and go through this manual auto-load
> setting/resetting, instead of just having test_uprobe_bench in a
> separate skeleton?...

ok, will add separate file for that.. will be simpler

> 
> > +
> > +       err = uprobe_multi__load(skel);
> > +       if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
> > +               goto cleanup;
> > +
> > +       attach_start_ns = get_time_ns();
> > +
> > +       err = uprobe_multi__attach(skel);
> > +       if (!ASSERT_OK(err, "uprobe_multi__attach"))
> > +               goto cleanup;
> > +
> > +       attach_end_ns = get_time_ns();
> > +
> > +       system("./uprobe_multi");
> > +
> > +       ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
> > +
> > +cleanup:
> > +       detach_start_ns = get_time_ns();
> > +       uprobe_multi__destroy(skel);
> > +       detach_end_ns = get_time_ns();
> > +
> > +       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
> > +       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
> > +
> > +       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
> > +       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
> 
> and for us lazy folks, what are the numbers you see on your machine?

right, will put it to the changelog

	# ./test_progs -n 260/5 -v
	bpf_testmod.ko is already unloaded.
	Loading bpf_testmod.ko...
	Successfully loaded bpf_testmod.ko.
	test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec
	test_bench_attach_uprobe:PASS:uprobe_multi__load 0 nsec
	test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec
	test_bench_attach_uprobe:PASS:uprobes_count 0 nsec
	test_bench_attach_uprobe: attached in   0.398s
	test_bench_attach_uprobe: detached in   0.443s
	#260/5   uprobe_multi_test/bench_uprobe:OK
	#260     uprobe_multi_test:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
	Successfully unloaded bpf_testmod.ko.

thanks,
jirka

> 
> > +}
> > +
> >  void test_uprobe_multi_test(void)
> >  {
> >         if (test__start_subtest("skel_api"))
> > @@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
> >                 test_attach_api_syms();
> >         if (test__start_subtest("link_api"))
> >                 test_link_api();
> > +       if (test__start_subtest("bench_uprobe"))
> > +               test_bench_attach_uprobe();
> >  }
> > diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c
> > index 1eeb9b7b9cad..cd73139dc881 100644
> > --- a/tools/testing/selftests/bpf/progs/uprobe_multi.c
> > +++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c
> > @@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
> >         uprobe_multi_check(ctx, true, true);
> >         return 0;
> >  }
> > +
> > +int count;
> > +
> > +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
> > +int test_uprobe_bench(struct pt_regs *ctx)
> > +{
> > +       count++;
> > +       return 0;
> > +}
> > --
> > 2.41.0
> >

Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test

[Jiri Olsa]

On Thu, Jul 06, 2023 at 09:42:05PM -0700, Andrii Nakryiko wrote:

SNIP

> > +static void test_bench_attach_usdt(void)
> > +{
> > +       struct uprobe_multi_usdt *skel = NULL;
> > +       long attach_start_ns, attach_end_ns;
> > +       long detach_start_ns, detach_end_ns;
> > +       double attach_delta, detach_delta;
> > +       struct bpf_program *prog;
> > +       int err;
> > +
> > +       skel = uprobe_multi_usdt__open();
> > +       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
> > +               goto cleanup;
> > +
> > +       bpf_object__for_each_program(prog, skel->obj)
> > +               bpf_program__set_autoload(prog, false);
> > +
> > +       bpf_program__set_autoload(skel->progs.usdt0, true);
> 
> there is nothing else in that skeleton, why this set_autoload() business?

nah copy&paste error from the uprobe bench test.. will change ;-)

jirka

SNIP

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

[Andrii Nakryiko]

On Tue, Jul 11, 2023 at 1:28 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Mon, Jul 10, 2023 at 10:55:36AM -0700, Andrii Nakryiko wrote:
> > On Mon, Jul 10, 2023 at 12:24 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> > >
> > > On Thu, Jul 06, 2023 at 03:29:08PM -0700, Andrii Nakryiko wrote:
> > > > On Fri, Jun 30, 2023 at 1:34 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > > > >
> > > > > Adding support for bpf_get_func_ip helper being called from
> > > > > ebpf program attached by uprobe_multi link.
> > > > >
> > > > > It returns the ip of the uprobe.
> > > > >
> > > > > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> > > > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > > > ---
> > > > >  kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
> > > > >  1 file changed, 30 insertions(+), 3 deletions(-)
> > > > >
> > > >
> > > > A slight aside related to bpf_get_func_ip() support in
> > > > uprobe/uretprobe. We just had a conversation with Alastair and Jordan
> > > > (cc'ed) about bpftrace and using bpf_get_func_ip() there with
> > > > uretprobes, and it seems like it doesn't work.
> > > >
> > > > Is that intentional or we just missed that bpf_get_func_ip() doesn't
> > > > work with uprobes/uretprobes? Do you think it would be hard to add
> > > > support for them for bpf_get_func_ip()? It's a very useful helper,
> > > > would be nice to have it working in all cases where it has meaningful
> > > > behavior (and I think it does for uprobe and uretprobe).
> > >
> > > I'm not sure we discussed at the time we added this helper,
> > > but I see same problem as we had with kprobes where we need
> > > to know if the probe is on the start of the symbol
> > >
> > > we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's
> > > get_func_ip helper version:
> > >
> > >         BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
> > >         {
> > >                 struct kprobe *kp = kprobe_running();
> > >
> > >                 if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
> > >                         return 0;
> > >
> > >                 return get_entry_ip((uintptr_t)kp->addr);
> > >         }
> > >
> > > I don't think we can have same flag for uprobe, we don't have
> > > the binary symbol data as we have for kernel
> >
> > what if we just return whatever was the IP where uprobe/uretprobe
> > interrupt instruction was installed at? I haven't tried what does
> > regs->ip contain for u*ret*probe, though, if it already has the IP
> > where we installed uprobe itself, it might be ok as is. But still,
>
> the entry uprobe gets the IP of the probed instruction, but the uretprobe
> gets IP of the next instruction after probed func retun - like 4c57f0 in
> the example below when uprobe is placed on trigger_func_get_func_ip
>
>           4c57eb:       e8 4b fe ff ff          call   4c563b <trigger_func_get_func_ip>
>           4c57f0:       eb 01                   jmp    4c57f3 <test_uprobe+0x1b1>
>
> > feels like it would be nice to have bpf_get_func_ip() working for
> > uprobe/uretprobe (even if semantics might differ for uprobes not at
> > the entry of the function).
>
> but it looks like we actually have the original uprobe address stored in
> current->utask->vaddr before bpf program is executed (in uretprobe_dispatcher)
> so we should be able to get that address from there in uretprobe's get_func_ip
> helper function, I'll check

ok, sounds great, seems like we have a pretty logical behavior for
both uprobe and uretprobe.

>
> I don't mind the semantic change for uprobe's get_func_ip, because I think
> it's unlikely we will be able to change it in future to return the real
> function entry
>

Yep, I don't think kernel can easily know where the function even
begins. But that's ok. We can clarify helper's behavior in UAPI
comments: uprobe/uretprobe always work, but return not function IP,
but the traced location. Thanks!


> jirka
>
> >
> > >
> > > I guess the app needs to store regs->ip and match it against symbol
> > > addresses in user space
> > >
> > > jirka
> > >
> > >
> > > >
> > > > Thanks!
> > > >
> > > > > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> > > > > index 4ef51fd0497f..f5a41c1604b8 100644
> > > > > --- a/kernel/trace/bpf_trace.c
> > > > > +++ b/kernel/trace/bpf_trace.c
> > > > > @@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
> > > > >  static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
> > > > >
> > > >
> > > > [...]

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

[Andrii Nakryiko]

On Tue, Jul 11, 2023 at 2:03 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:
>
> SNIP
>
> > > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > ---
> > >  tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
> > >  1 file changed, 117 insertions(+), 61 deletions(-)
> > >
> >
> > A bunch of nits, but overall looks good. Please address nits, and add my ack
> >
> > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> >
> > > diff --git a/tools/lib/bpf/elf.c b/tools/lib/bpf/elf.c
> > > index 74e35071d22e..fcce4bd2478f 100644
> > > --- a/tools/lib/bpf/elf.c
> > > +++ b/tools/lib/bpf/elf.c
> > > @@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
> > >         return NULL;
> > >  }
> > >
> > > +struct elf_sym {
> > > +       const char *name;
> > > +       GElf_Sym sym;
> > > +       GElf_Shdr sh;
> > > +};
> > > +
> >
> > if we want to use elf_sym_iter outside of elf.c, this should be in
> > libbpf_internal.h?
>
> yes eventually, but all the helper functions using elf_sym_iter that
> I added later are in elf.c, so there's no need atm
>
> SNIP
>
> > > +
> > > +static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
> > > +{
> > > +       struct elf_sym *ret = &iter->sym;
> > > +       GElf_Sym *sym = &ret->sym;
> > > +       const char *name = NULL;
> > > +       Elf_Scn *sym_scn;
> > > +       size_t idx;
> > > +
> > > +       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
> > > +               if (!gelf_getsym(iter->syms, idx, sym))
> > > +                       continue;
> > > +               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
> > > +                       continue;
> > > +               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
> > > +               if (!name)
> > > +                       continue;
> > > +
> > > +               /* Transform symbol's virtual address (absolute for
> > > +                * binaries and relative for shared libs) into file
> > > +                * offset, which is what kernel is expecting for
> > > +                * uprobe/uretprobe attachment.
> > > +                * See Documentation/trace/uprobetracer.rst for more
> > > +                * details.
> > > +                * This is done by looking up symbol's containing
> > > +                * section's header and using iter's virtual address
> > > +                * (sh_addr) and corresponding file offset (sh_offset)
> > > +                * to transform sym.st_value (virtual address) into
> > > +                * desired final file offset.
> > > +                */
> >
> > this comment is misplaced? we don't do the translation here
>
> right, should be placed at the elf_sym_offset function
>
> >
> > > +               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
> > > +               if (!sym_scn)
> > > +                       continue;
> > > +               if (!gelf_getshdr(sym_scn, &ret->sh))
> > > +                       continue;
> > > +
> > > +               iter->next_sym_idx = idx + 1;
> > > +               ret->name = name;
> > > +               return ret;
> > > +       }
> > > +
> > > +       return NULL;
> > > +}
> > > +
> > > +static unsigned long elf_sym_offset(struct elf_sym *sym)
> > > +{
> > > +       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
> > > +}
> > > +
> > >  /* Find offset of function name in the provided ELF object. "binary_path" is
> > >   * the path to the ELF binary represented by "elf", and only used for error
> > >   * reporting matters. "name" matches symbol name or name@@LIB for library
> > > @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
> > >          * reported as a warning/error.
> > >          */
> > >         for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
> > > -               size_t nr_syms, strtabidx, idx;
> > > -               Elf_Data *symbols = NULL;
> > > -               Elf_Scn *scn = NULL;
> > > +               struct elf_sym_iter iter;
> > > +               struct elf_sym *sym;
> > >                 int last_bind = -1;
> > > -               const char *sname;
> > > -               GElf_Shdr sh;
> > > +               int curr_bind;
> >
> > OCD nit:
> >
> > $ rg 'curr(_|\b)' | wc -l
> > 8
> > $ rg 'cur(_|\b)' | wc -l
> > 148
> >
> > and those 8 I consider an unfortunate accident ;) let's standardize on
> > using "cur" consistently
>
> ok.. probably easier to make that change than treat ocd ;-)
>

thanks :)


> >
> > >
> > > -               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
> > > -               if (!scn) {
> > > -                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
> > > -                                binary_path);
> > > -                       continue;
> > > -               }
> > > -               if (!gelf_getshdr(scn, &sh))
> > > -                       continue;
> > > -               strtabidx = sh.sh_link;
> > > -               symbols = elf_getdata(scn, 0);
> > > -               if (!symbols) {
> > > -                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
> > > -                               binary_path, elf_errmsg(-1));
> > > -                       ret = -LIBBPF_ERRNO__FORMAT;
> > > +               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
> > > +               if (ret) {
> > > +                       if (ret == -ENOENT)
> > > +                               continue;
> > >                         goto out;
> >
> > another styling nit: let's avoid unnecessary nesting of ifs:
> >
> > if (ret == -ENOENT)
> >     continue;
> > if (ret)
> >     goto out;
> >
> > simple and clean
>
> ok
>
> >
> >
> > >                 }
> > > -               nr_syms = symbols->d_size / sh.sh_entsize;
> > > -
> > > -               for (idx = 0; idx < nr_syms; idx++) {
> > > -                       int curr_bind;
> > > -                       GElf_Sym sym;
> > > -                       Elf_Scn *sym_scn;
> > > -                       GElf_Shdr sym_sh;
> > > -
> > > -                       if (!gelf_getsym(symbols, idx, &sym))
> > > -                               continue;
> > > -
> > > -                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
> > > -                               continue;
> > > -
> > > -                       sname = elf_strptr(elf, strtabidx, sym.st_name);
> > > -                       if (!sname)
> > > -                               continue;
> > > -
> > > -                       curr_bind = GELF_ST_BIND(sym.st_info);
> > >
> > > +               while ((sym = elf_sym_iter_next(&iter))) {
> > >                         /* User can specify func, func@@LIB or func@@LIB_VERSION. */
> > > -                       if (strncmp(sname, name, name_len) != 0)
> > > +                       if (strncmp(sym->name, name, name_len) != 0)
> > >                                 continue;
> > >                         /* ...but we don't want a search for "foo" to match 'foo2" also, so any
> > >                          * additional characters in sname should be of the form "@@LIB".
> > >                          */
> > > -                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
> > > +                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
> > >                                 continue;
> > >
> > > -                       if (ret >= 0) {
> > > +                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
> > > +
> > > +                       if (ret > 0) {
> >
> > used to be >=, why the change?
>
> the original code initialized ret with -ENOENT and did not change
> its value till this point, so the condition never triggered for
> the first loop, but it did for the new code because we now use ret
> earlier in:
>
>         ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
>
> also the check makes sense to me only if ret > 0 .. when we find
> a duplicate value for symbol

there is a subtle difference if we have some unresolved func with addr
0 or something. But you are right, it's probably not very sensical, we
can keep it as > 0.

>
> jirka

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

[Andrii Nakryiko]

On Tue, Jul 11, 2023 at 2:05 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Thu, Jul 06, 2023 at 04:02:22PM -0700, Andrii Nakryiko wrote:
> > On Fri, Jun 30, 2023 at 1:35 AM Jiri Olsa <jolsa@kernel.org> wrote:
> > >
> > > Adding new elf object that will contain elf related functions.
> > > There's no functional change.
> > >
> > > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > ---
> > >  tools/lib/bpf/Build        |   2 +-
> > >  tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
> > >  tools/lib/bpf/libbpf.c     | 186 +---------------------------------
> > >  tools/lib/bpf/libbpf_elf.h |  11 +++
> > >  4 files changed, 211 insertions(+), 186 deletions(-)
> > >  create mode 100644 tools/lib/bpf/elf.c
> > >  create mode 100644 tools/lib/bpf/libbpf_elf.h
> > >
> >
> > [...]
> >
> > > diff --git a/tools/lib/bpf/libbpf_elf.h b/tools/lib/bpf/libbpf_elf.h
> > > new file mode 100644
> > > index 000000000000..1b652220fabf
> > > --- /dev/null
> > > +++ b/tools/lib/bpf/libbpf_elf.h
> > > @@ -0,0 +1,11 @@
> > > +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
> > > +
> > > +#ifndef __LIBBPF_LIBBPF_ELF_H
> > > +#define __LIBBPF_LIBBPF_ELF_H
> > > +
> > > +#include <libelf.h>
> > > +
> > > +long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
> > > +long elf_find_func_offset_from_file(const char *binary_path, const char *name);
> > > +
> > > +#endif /* *__LIBBPF_LIBBPF_ELF_H */
> >
> > we have libbpf_internal.h, let's put all this there for now, it's
> > already all the internal stuff together, I don't know if separate
> > header with few functions gives us much
>
> there's more functions coming later in the patchset
>
>         struct elf_fd {
>                 Elf *elf;
>                 int fd;
>         };
>
>         int elf_open(const char *binary_path, struct elf_fd *elf_fd);
>         void elf_close(struct elf_fd *elf_fd);
>
>         int elf_resolve_syms_offsets(const char *binary_path, int cnt,
>                                      const char **syms, unsigned long **poffsets);
>
>         int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
>                                          unsigned long **poffsets, size_t *pcnt);
>
>
> and there's probably more elf helpers to eventually move in:
>
>         libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off);
>         libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off);
>         libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx);
>         libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name);
>         libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn);
>         libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn);
>         libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn);
>         libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx);
>         libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx);
>

yep, I was anticipating that these will move as well. But I think it's
fine if they all stay in libbpf_internal.h, IMO. I'd rather not have
many small internal header for no good reason (like we have
str_error.h right now, with a single func declaration)

>         usdt.c:static int find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn)
>
>         'struct elf_seg' stuff
>
>         usdt.c:static int cmp_elf_segs(const void *_a, const void *_b)
>         usdt.c:static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt)
>         usdt.c:static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt)
>         usdt.c:static struct elf_seg *find_elf_seg(struct elf_seg *segs, size_t seg_cnt, long virtaddr)
>         usdt.c:static struct elf_seg *find_vma_seg(struct elf_seg *segs, size_t seg_cnt, long offset)
>
>
> but I can add the new header file later in follow up changes when
> we have more elf functions in

see above, I don't think we should, let's stick to libbpf_internal.h for now

>
> jirka

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

[Andrii Nakryiko]

On Tue, Jul 11, 2023 at 2:05 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:
>
> SNIP
>
> > > +       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
> > > +               return libbpf_err_ptr(-EINVAL);
> > > +
> > > +       syms = OPTS_GET(opts, syms, NULL);
> > > +       offsets = OPTS_GET(opts, offsets, NULL);
> > > +       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
> > > +       cookies = OPTS_GET(opts, cookies, NULL);
> > > +       cnt = OPTS_GET(opts, cnt, 0);
> > > +
> > > +       /*
> > > +        * User can specify 2 mutually exclusive set of inputs:
> > > +        *
> > > +        * 1) use only path/func_pattern/pid arguments
> > > +        *
> > > +        * 2) use path/pid with allowed combinations of:
> > > +        *    syms/offsets/ref_ctr_offsets/cookies/cnt
> > > +        *
> > > +        *    - syms and offsets are mutually exclusive
> > > +        *    - ref_ctr_offsets and cookies are optional
> > > +        *
> > > +        * Any other usage results in error.
> > > +        */
> > > +
> > > +       if (!path && !func_pattern && !cnt)
> >
> > weird, I'd expect separate if (!path) return error (already bad,
> > regardless of func_pattern or cnt)
> >
> > then if (!func_pattern && cnt == 0) return error
> >
> > > +               return libbpf_err_ptr(-EINVAL);
> > > +       if (func_pattern && !path)
> > > +               return libbpf_err_ptr(-EINVAL);
> > > +
> > > +       has_pattern = path && func_pattern;
> >
> > this and above check must be some leftovers from previous version.
> > path should always be present. and so you don't need has_pattern
> > variable, just use "func_pattern" check
>
> hum, right, previous version had 2 paths, now there's just one,
> I'll change that together with the suggested change above
>
> >
> > > +
> > > +       if (has_pattern) {
> > > +               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
> > > +                       return libbpf_err_ptr(-EINVAL);
> > > +       } else {
> > > +               if (!cnt)
> > > +                       return libbpf_err_ptr(-EINVAL);
> > > +               if (!!syms == !!offsets)
> > > +                       return libbpf_err_ptr(-EINVAL);
> > > +       }
> > > +
> > > +       if (has_pattern) {
> > > +               if (!strchr(path, '/')) {
> > > +                       err = resolve_full_path(path, full_path, sizeof(full_path));
> > > +                       if (err) {
> > > +                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
> > > +                                       prog->name, path, err);
> > > +                               return libbpf_err_ptr(err);
> > > +                       }
> > > +                       path = full_path;
> > > +               }
> > > +
> > > +               err = elf_resolve_pattern_offsets(path, func_pattern,
> > > +                                                 &resolved_offsets, &cnt);
> > > +               if (err < 0)
> > > +                       return libbpf_err_ptr(err);
> > > +               offsets = resolved_offsets;
> > > +       } else if (syms) {
> > > +               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
> > > +               if (err < 0)
> > > +                       return libbpf_err_ptr(err);
> > > +               offsets = resolved_offsets;
> >
> > you can extract this common error checking and `offsets =
> > resolved_offsets;` to after if, it's common for both branches
>
> not sure what you mean in here, offsets can be also provided
> by OPTS_GET(opts, offsets, NULL) earlier

ah, I missed that it's else if, not just else. Never mind, it's good as is then.

>
> > > +       }
> > > +
> > > +       retprobe = OPTS_GET(opts, retprobe, false);
> > > +
> > > +       lopts.uprobe_multi.path = path;
> > > +       lopts.uprobe_multi.offsets = offsets;
> > > +       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
> > > +       lopts.uprobe_multi.cookies = cookies;
> > > +       lopts.uprobe_multi.cnt = cnt;
> > > +       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;
> >
> > retprobe is another unnecessary var, just inline check here to keep it simpler
>
> ok
>
> >
> > > +
> > > +       if (pid == 0)
> > > +               pid = getpid();
> > > +       if (pid > 0)
> > > +               lopts.uprobe_multi.pid = pid;
> > > +
> > > +       link = calloc(1, sizeof(*link));
> > > +       if (!link) {
> > > +               err = -ENOMEM;
> > > +               goto error;
> > > +       }
> > > +       link->detach = &bpf_link__detach_fd;
> > > +
> > > +       prog_fd = bpf_program__fd(prog);
> > > +       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
> > > +       if (link_fd < 0) {
> > > +               err = -errno;
> > > +               pr_warn("prog '%s': failed to attach: %s\n",
> >
> > "failed to attach multi-uprobe"? We probably should have added "failed
> > to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as
> > well?
>
> ook, will add
>
> >
> > > +                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
> > > +               goto error;
> > > +       }
> > > +       link->fd = link_fd;
> > > +       free(resolved_offsets);
> > > +       return link;
> > > +
> > > +error:
> > > +       free(resolved_offsets);
> > > +       free(link);
> > > +       return libbpf_err_ptr(err);
> > > +}
> > > +
> > >  LIBBPF_API struct bpf_link *
> > >  bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
> > >                                 const char *binary_path, size_t func_offset,
> > > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
> > > index 754da73c643b..7c218f610210 100644
> > > --- a/tools/lib/bpf/libbpf.h
> > > +++ b/tools/lib/bpf/libbpf.h
> > > @@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
> > >                                       const char *pattern,
> > >                                       const struct bpf_kprobe_multi_opts *opts);
> > >
> > > +struct bpf_uprobe_multi_opts {
> > > +       /* size of this struct, for forward/backward compatibility */
> > > +       size_t sz;
> > > +       /* array of function symbols to attach */
> >
> > attach to?
>
> ok
>
> >
> > > +       const char **syms;
> > > +       /* array of function addresses to attach */
> >
> > attach to?
>
> ook
>
> >
> > > +       const unsigned long *offsets;
> > > +       /* array of refctr offsets to attach */
> >
> > we don't really attach to ref counters, so maybe "optional, array of
> > associated ref counter offsets" or something along those lines ?
>
> ok
>
> >
> > > +       const unsigned long *ref_ctr_offsets;
> > > +       /* array of user-provided values fetchable through bpf_get_attach_cookie */
> >
> > "array of associated BPF cookies"? we can't keep explaining what BPF
> > cookie is in every possible API :)
>
> ook
>
> >
> > > +       const __u64 *cookies;
> > > +       /* number of elements in syms/addrs/cookies arrays */
> > > +       size_t cnt;
> > > +       /* create return uprobes */
> > > +       bool retprobe;
> > > +       size_t :0;
> > > +};
> > > +
> > > +#define bpf_uprobe_multi_opts__last_field retprobe
> > > +
> > > +LIBBPF_API struct bpf_link *
> > > +bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
> > > +                                pid_t pid,
> > > +                                const char *binary_path,
> > > +                                const char *func_pattern,
> > > +                                const struct bpf_uprobe_multi_opts *opts);
> > > +
> >
> > ok, let's be good citizens and add documentation for this new API.
> > Those comments about valid combinations belong here as well. Please
> > take a look at existing doccomments for the format and conventions.
> > Thanks!
>
> ok, will add
>
> thanks,
> jirka