[PATCH bpf-next v3 00/11] libbpf: Fix accessing syscall arguments

[PATCH bpf-next v3 00/11] libbpf: Fix accessing syscall arguments

[Ilya Leoshkevich]

libbpf now has macros to access syscall arguments in an
architecture-agnostic manner, but unfortunately they have a number of
issues on non-Intel arches, which this series aims to fix.

v1: https://lore.kernel.org/bpf/20220201234200.1836443-1-iii@linux.ibm.com/
v1 -> v2:
* Put orig_gpr2 in place of args[1] on s390 (Vasily).
* Fix arm64, powerpc and riscv (Heiko).

v2: https://lore.kernel.org/bpf/20220204041955.1958263-1-iii@linux.ibm.com/
v2 -> v3:
* Undo args[1] change (Andrii).
* Rename PT_REGS_SYSCALL to PT_REGS_SYSCALL_REGS (Andrii).
* Split the riscv patch (Andrii).

+cc Naveen.

Ilya Leoshkevich (11):
  arm64/bpf: Add orig_x0 to user_pt_regs
  s390/bpf: Add orig_gpr2 to user_pt_regs
  selftests/bpf: Fix an endianness issue in bpf_syscall_macro test
  libbpf: Add __PT_PARM1_REG_SYSCALL macro
  libbpf: Add PT_REGS_SYSCALL_REGS macro
  selftests/bpf: Use PT_REGS_SYSCALL_REGS in bpf_syscall_macro
  libbpf: Fix accessing the first syscall argument on arm64
  libbpf: Fix accessing syscall arguments on powerpc
  libbpf: Fix accessing program counter on riscv
  libbpf: Fix accessing syscall arguments on riscv
  libbpf: Fix accessing the first syscall argument on s390

 arch/arm64/include/asm/ptrace.h               |  2 +-
 arch/arm64/include/uapi/asm/ptrace.h          |  1 +
 arch/s390/include/asm/ptrace.h                |  2 +-
 arch/s390/include/uapi/asm/ptrace.h           |  1 +
 tools/lib/bpf/bpf_tracing.h                   | 23 ++++++++++++++++++-
 .../selftests/bpf/progs/bpf_syscall_macro.c   |  7 ++++--
 6 files changed, 31 insertions(+), 5 deletions(-)

-- 
2.34.1

[PATCH bpf-next v3 01/11] arm64/bpf: Add orig_x0 to user_pt_regs

[Ilya Leoshkevich]

orig_x0 is needed in order to access the first syscall argument from
eBPF programs.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/arm64/include/asm/ptrace.h      | 2 +-
 arch/arm64/include/uapi/asm/ptrace.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index 41b332c054ab..1be22f7870f8 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -183,9 +183,9 @@ struct pt_regs {
 			u64 sp;
 			u64 pc;
 			u64 pstate;
+			u64 orig_x0;
 		};
 	};
-	u64 orig_x0;
 #ifdef __AARCH64EB__
 	u32 unused2;
 	s32 syscallno;
diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
index 758ae984ff97..3c118c5b0893 100644
--- a/arch/arm64/include/uapi/asm/ptrace.h
+++ b/arch/arm64/include/uapi/asm/ptrace.h
@@ -90,6 +90,7 @@ struct user_pt_regs {
 	__u64		sp;
 	__u64		pc;
 	__u64		pstate;
+	__u64		orig_x0;
 };
 
 struct user_fpsimd_state {
-- 
2.34.1

[PATCH bpf-next v3 02/11] s390/bpf: Add orig_gpr2 to user_pt_regs

[Ilya Leoshkevich]

user_pt_regs is used by eBPF in order to access userspace registers -
see commit 466698e654e8 ("s390/bpf: correct broken uapi for
BPF_PROG_TYPE_PERF_EVENT program type"). In order to access the first
syscall argument from eBPF programs, we need to export orig_gpr2.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 arch/s390/include/asm/ptrace.h      | 2 +-
 arch/s390/include/uapi/asm/ptrace.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h
index 4ffa8e7f0ed3..c8698e643904 100644
--- a/arch/s390/include/asm/ptrace.h
+++ b/arch/s390/include/asm/ptrace.h
@@ -83,9 +83,9 @@ struct pt_regs {
 			unsigned long args[1];
 			psw_t psw;
 			unsigned long gprs[NUM_GPRS];
+			unsigned long orig_gpr2;
 		};
 	};
-	unsigned long orig_gpr2;
 	union {
 		struct {
 			unsigned int int_code;
diff --git a/arch/s390/include/uapi/asm/ptrace.h b/arch/s390/include/uapi/asm/ptrace.h
index ad64d673b5e6..b3dec603f507 100644
--- a/arch/s390/include/uapi/asm/ptrace.h
+++ b/arch/s390/include/uapi/asm/ptrace.h
@@ -295,6 +295,7 @@ typedef struct {
 	unsigned long args[1];
 	psw_t psw;
 	unsigned long gprs[NUM_GPRS];
+	unsigned long orig_gpr2;
 } user_pt_regs;
 
 /*
-- 
2.34.1

[PATCH bpf-next v3 03/11] selftests/bpf: Fix an endianness issue in bpf_syscall_macro test

[Ilya Leoshkevich]

bpf_syscall_macro reads a long argument into an int variable, which
produces a wrong value on big-endian systems. Fix by reading the
argument into an intermediate long variable first.

Fixes: 77fc0330dfe5 ("selftests/bpf: Add a test to confirm PT_REGS_PARM4_SYSCALL")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/testing/selftests/bpf/progs/bpf_syscall_macro.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c b/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
index c8e60220cda8..f5c6ef2ff6d1 100644
--- a/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
+++ b/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
@@ -28,6 +28,7 @@ int BPF_KPROBE(handle_sys_prctl)
 {
 	struct pt_regs *real_regs;
 	pid_t pid = bpf_get_current_pid_tgid() >> 32;
+	unsigned long tmp;
 
 	if (pid != filter_pid)
 		return 0;
@@ -35,7 +36,9 @@ int BPF_KPROBE(handle_sys_prctl)
 	real_regs = (struct pt_regs *)PT_REGS_PARM1(ctx);
 
 	/* test for PT_REGS_PARM */
-	bpf_probe_read_kernel(&arg1, sizeof(arg1), &PT_REGS_PARM1_SYSCALL(real_regs));
+
+	bpf_probe_read_kernel(&tmp, sizeof(tmp), &PT_REGS_PARM1_SYSCALL(real_regs));
+	arg1 = tmp;
 	bpf_probe_read_kernel(&arg2, sizeof(arg2), &PT_REGS_PARM2_SYSCALL(real_regs));
 	bpf_probe_read_kernel(&arg3, sizeof(arg3), &PT_REGS_PARM3_SYSCALL(real_regs));
 	bpf_probe_read_kernel(&arg4_cx, sizeof(arg4_cx), &PT_REGS_PARM4(real_regs));
-- 
2.34.1

[PATCH bpf-next v3 04/11] libbpf: Add __PT_PARM1_REG_SYSCALL macro

[Ilya Leoshkevich]

Some architectures have a special way to access the first syscall
argument. There already exists __PT_PARM4_REG_SYSCALL for the
fourth argument, so define a similar macro for the first one.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 032ba809f3e5..30f0964f8c9e 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -265,7 +265,11 @@ struct pt_regs;
 
 #endif
 
+#ifdef __PT_PARM1_REG_SYSCALL
+#define PT_REGS_PARM1_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM1_REG_SYSCALL)
+#else /* __PT_PARM1_REG_SYSCALL */
 #define PT_REGS_PARM1_SYSCALL(x) PT_REGS_PARM1(x)
+#endif
 #define PT_REGS_PARM2_SYSCALL(x) PT_REGS_PARM2(x)
 #define PT_REGS_PARM3_SYSCALL(x) PT_REGS_PARM3(x)
 #ifdef __PT_PARM4_REG_SYSCALL
@@ -275,7 +279,11 @@ struct pt_regs;
 #endif
 #define PT_REGS_PARM5_SYSCALL(x) PT_REGS_PARM5(x)
 
+#ifdef __PT_PARM1_REG_SYSCALL
+#define PT_REGS_PARM1_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM1_REG_SYSCALL)
+#else /* __PT_PARM1_REG_SYSCALL */
 #define PT_REGS_PARM1_CORE_SYSCALL(x) PT_REGS_PARM1_CORE(x)
+#endif
 #define PT_REGS_PARM2_CORE_SYSCALL(x) PT_REGS_PARM2_CORE(x)
 #define PT_REGS_PARM3_CORE_SYSCALL(x) PT_REGS_PARM3_CORE(x)
 #ifdef __PT_PARM4_REG_SYSCALL
-- 
2.34.1

[PATCH bpf-next v3 05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro

[Ilya Leoshkevich]

Some architectures pass a pointer to struct pt_regs to syscall
handlers, others unpack it into individual function parameters.
Introduce a macro to describe what a particular arch does, using
`passing pt_regs *` as a default.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 30f0964f8c9e..08d2990c006f 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -334,6 +334,15 @@ struct pt_regs;
 
 #endif /* defined(bpf_target_defined) */
 
+/*
+ * When invoked from a syscall handler kprobe, returns a pointer to a
+ * struct pt_regs containing syscall arguments and suitable for passing to
+ * PT_REGS_PARMn_SYSCALL() and PT_REGS_PARMn_CORE_SYSCALL().
+ */
+#ifndef PT_REGS_SYSCALL_REGS
+#define PT_REGS_SYSCALL_REGS(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
+#endif
+
 #ifndef ___bpf_concat
 #define ___bpf_concat(a, b) a ## b
 #endif
-- 
2.34.1

[PATCH bpf-next v3 06/11] selftests/bpf: Use PT_REGS_SYSCALL_REGS in bpf_syscall_macro

[Ilya Leoshkevich]

Ensure that PT_REGS_SYSCALL_REGS works correctly.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/testing/selftests/bpf/progs/bpf_syscall_macro.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c b/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
index f5c6ef2ff6d1..e7c622cb6a39 100644
--- a/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
+++ b/tools/testing/selftests/bpf/progs/bpf_syscall_macro.c
@@ -33,7 +33,7 @@ int BPF_KPROBE(handle_sys_prctl)
 	if (pid != filter_pid)
 		return 0;
 
-	real_regs = (struct pt_regs *)PT_REGS_PARM1(ctx);
+	real_regs = PT_REGS_SYSCALL_REGS(ctx);
 
 	/* test for PT_REGS_PARM */
 
-- 
2.34.1

[PATCH bpf-next v3 07/11] libbpf: Fix accessing the first syscall argument on arm64

[Ilya Leoshkevich]

On arm64, the first syscall argument should be accessed via orig_x0
(see arch/arm64/include/asm/syscall.h). Currently regs[0] is used
instead, leading to bpf_syscall_macro test failure.

Reported-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 08d2990c006f..317bee0fd8e4 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -145,6 +145,7 @@
 /* arm64 provides struct user_pt_regs instead of struct pt_regs to userspace */
 #define __PT_REGS_CAST(x) ((const struct user_pt_regs *)(x))
 #define __PT_PARM1_REG regs[0]
+#define __PT_PARM1_REG_SYSCALL orig_x0
 #define __PT_PARM2_REG regs[1]
 #define __PT_PARM3_REG regs[2]
 #define __PT_PARM4_REG regs[3]
-- 
2.34.1

[PATCH bpf-next v3 08/11] libbpf: Fix accessing syscall arguments on powerpc

[Ilya Leoshkevich]

powerpc's syscall handlers get "unpacked" arguments instead of a
struct pt_regs pointer. Indicate this to libbpf using
PT_REGS_SYSCALL_REGS macro.

Reported-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 317bee0fd8e4..f9d22ea0af97 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -181,6 +181,7 @@
 #define __PT_RC_REG gpr[3]
 #define __PT_SP_REG sp
 #define __PT_IP_REG nip
+#define PT_REGS_SYSCALL_REGS(ctx) ctx
 
 #elif defined(bpf_target_sparc)
 
-- 
2.34.1

[PATCH bpf-next v3 09/11] libbpf: Fix accessing program counter on riscv

[Ilya Leoshkevich]

riscv registers are accessed via struct user_regs_struct, not struct
pt_regs. The program counter member in this struct is called pc, not
epc.

Fixes: 3cc31d794097 ("libbpf: Normalize PT_REGS_xxx() macro definitions")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index f9d22ea0af97..8629441304df 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -213,7 +213,7 @@
 #define __PT_FP_REG fp
 #define __PT_RC_REG a5
 #define __PT_SP_REG sp
-#define __PT_IP_REG epc
+#define __PT_IP_REG pc
 
 #endif
 
-- 
2.34.1

[PATCH bpf-next v3 10/11] libbpf: Fix accessing syscall arguments on riscv

[Ilya Leoshkevich]

riscv's syscall handlers get "unpacked" arguments instead of a
struct pt_regs pointer. Indicate this to libbpf using
PT_REGS_SYSCALL_REGS macro.

Reported-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 8629441304df..169d47aa0a79 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -214,6 +214,7 @@
 #define __PT_RC_REG a5
 #define __PT_SP_REG sp
 #define __PT_IP_REG pc
+#define PT_REGS_SYSCALL_REGS(ctx) ctx
 
 #endif
 
-- 
2.34.1

[PATCH bpf-next v3 11/11] libbpf: Fix accessing the first syscall argument on s390

[Ilya Leoshkevich]

On s390, the first syscall argument should be accessed via orig_gpr2
(see arch/s390/include/asm/syscall.h). Currently gpr[2] is used
instead, leading to bpf_syscall_macro test failure.

Reported-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 169d47aa0a79..cf980e54d331 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -117,6 +117,7 @@
 /* s390 provides user_pt_regs instead of struct pt_regs to userspace */
 #define __PT_REGS_CAST(x) ((const user_pt_regs *)(x))
 #define __PT_PARM1_REG gprs[2]
+#define __PT_PARM1_REG_SYSCALL orig_gpr2
 #define __PT_PARM2_REG gprs[3]
 #define __PT_PARM3_REG gprs[4]
 #define __PT_PARM4_REG gprs[5]
-- 
2.34.1

Re: [PATCH bpf-next v3 04/11] libbpf: Add __PT_PARM1_REG_SYSCALL macro

[Naveen N. Rao]

Ilya Leoshkevich wrote:
> Some architectures have a special way to access the first syscall
> argument. There already exists __PT_PARM4_REG_SYSCALL for the
> fourth argument, so define a similar macro for the first one.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  tools/lib/bpf/bpf_tracing.h | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
> index 032ba809f3e5..30f0964f8c9e 100644
> --- a/tools/lib/bpf/bpf_tracing.h
> +++ b/tools/lib/bpf/bpf_tracing.h
> @@ -265,7 +265,11 @@ struct pt_regs;
> 
>  #endif
> 
> +#ifdef __PT_PARM1_REG_SYSCALL
> +#define PT_REGS_PARM1_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM1_REG_SYSCALL)
> +#else /* __PT_PARM1_REG_SYSCALL */
>  #define PT_REGS_PARM1_SYSCALL(x) PT_REGS_PARM1(x)
> +#endif
>  #define PT_REGS_PARM2_SYSCALL(x) PT_REGS_PARM2(x)
>  #define PT_REGS_PARM3_SYSCALL(x) PT_REGS_PARM3(x)
>  #ifdef __PT_PARM4_REG_SYSCALL
> @@ -275,7 +279,11 @@ struct pt_regs;
>  #endif
>  #define PT_REGS_PARM5_SYSCALL(x) PT_REGS_PARM5(x)
> 
> +#ifdef __PT_PARM1_REG_SYSCALL
> +#define PT_REGS_PARM1_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM1_REG_SYSCALL)
> +#else /* __PT_PARM1_REG_SYSCALL */
>  #define PT_REGS_PARM1_CORE_SYSCALL(x) PT_REGS_PARM1_CORE(x)
> +#endif
>  #define PT_REGS_PARM2_CORE_SYSCALL(x) PT_REGS_PARM2_CORE(x)
>  #define PT_REGS_PARM3_CORE_SYSCALL(x) PT_REGS_PARM3_CORE(x)
>  #ifdef __PT_PARM4_REG_SYSCALL

When I was contemplating implementing this for powerpc, I came up with 
the below patch which looked cleaner to me, and also makes it easy for 
architectures to over-ride any other syscall parameter in future. Feel 
free to include this in your series if it makes sense.

- Naveen

--
libbpf: Generalize overriding syscall parameter access macros

Instead of conditionally overriding PT_REGS_PARM4_SYSCALL, provide
default fallback for all _REG_SYSCALL macros so that architectures can
simply override a specific syscall parameter macro.

Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 tools/lib/bpf/bpf_tracing.h | 40 ++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 16 deletions(-)

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 032ba809f3e57a..2e2f057c7ec7c5 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -265,25 +265,33 @@ struct pt_regs;
 
 #endif
 
-#define PT_REGS_PARM1_SYSCALL(x) PT_REGS_PARM1(x)
-#define PT_REGS_PARM2_SYSCALL(x) PT_REGS_PARM2(x)
-#define PT_REGS_PARM3_SYSCALL(x) PT_REGS_PARM3(x)
-#ifdef __PT_PARM4_REG_SYSCALL
-#define PT_REGS_PARM4_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM4_REG_SYSCALL)
-#else /* __PT_PARM4_REG_SYSCALL */
-#define PT_REGS_PARM4_SYSCALL(x) PT_REGS_PARM4(x)
+#ifndef __PT_PARM1_REG_SYSCALL
+#define __PT_PARM1_REG_SYSCALL __PT_PARM1_REG
+#endif
+#ifndef __PT_PARM2_REG_SYSCALL
+#define __PT_PARM2_REG_SYSCALL __PT_PARM2_REG
+#endif
+#ifndef __PT_PARM3_REG_SYSCALL
+#define __PT_PARM3_REG_SYSCALL __PT_PARM3_REG
+#endif
+#ifndef __PT_PARM4_REG_SYSCALL
+#define __PT_PARM4_REG_SYSCALL __PT_PARM4_REG
 #endif
-#define PT_REGS_PARM5_SYSCALL(x) PT_REGS_PARM5(x)
+#ifndef __PT_PARM5_REG_SYSCALL
+#define __PT_PARM5_REG_SYSCALL __PT_PARM5_REG
+#endif
+
+#define PT_REGS_PARM1_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM1_REG_SYSCALL)
+#define PT_REGS_PARM2_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM2_REG_SYSCALL)
+#define PT_REGS_PARM3_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM3_REG_SYSCALL)
+#define PT_REGS_PARM4_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM4_REG_SYSCALL)
+#define PT_REGS_PARM5_SYSCALL(x) (__PT_REGS_CAST(x)->__PT_PARM5_REG_SYSCALL)
 
-#define PT_REGS_PARM1_CORE_SYSCALL(x) PT_REGS_PARM1_CORE(x)
-#define PT_REGS_PARM2_CORE_SYSCALL(x) PT_REGS_PARM2_CORE(x)
-#define PT_REGS_PARM3_CORE_SYSCALL(x) PT_REGS_PARM3_CORE(x)
-#ifdef __PT_PARM4_REG_SYSCALL
+#define PT_REGS_PARM1_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM1_REG_SYSCALL)
+#define PT_REGS_PARM2_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM2_REG_SYSCALL)
+#define PT_REGS_PARM3_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM3_REG_SYSCALL)
 #define PT_REGS_PARM4_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM4_REG_SYSCALL)
-#else /* __PT_PARM4_REG_SYSCALL */
-#define PT_REGS_PARM4_CORE_SYSCALL(x) PT_REGS_PARM4_CORE(x)
-#endif
-#define PT_REGS_PARM5_CORE_SYSCALL(x) PT_REGS_PARM5_CORE(x)
+#define PT_REGS_PARM5_CORE_SYSCALL(x) BPF_CORE_READ(__PT_REGS_CAST(x), __PT_PARM5_REG_SYSCALL)
 
 #else /* defined(bpf_target_defined) */
 
-- 
2.34.1

Re: [PATCH bpf-next v3 05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro

[Naveen N. Rao]

Ilya Leoshkevich wrote:
> Some architectures pass a pointer to struct pt_regs to syscall
> handlers, others unpack it into individual function parameters.

I think that is just dependent on ARCH_HAS_SYSCALL_WRAPPER, so only x86, 
arm64 and s390 pass pointers to pt_regs to syscall entry points.

> Introduce a macro to describe what a particular arch does, using
> `passing pt_regs *` as a default.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  tools/lib/bpf/bpf_tracing.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
> index 30f0964f8c9e..08d2990c006f 100644
> --- a/tools/lib/bpf/bpf_tracing.h
> +++ b/tools/lib/bpf/bpf_tracing.h
> @@ -334,6 +334,15 @@ struct pt_regs;
> 
>  #endif /* defined(bpf_target_defined) */
> 
> +/*
> + * When invoked from a syscall handler kprobe, returns a pointer to a
> + * struct pt_regs containing syscall arguments and suitable for passing to
> + * PT_REGS_PARMn_SYSCALL() and PT_REGS_PARMn_CORE_SYSCALL().
> + */
> +#ifndef PT_REGS_SYSCALL_REGS
> +#define PT_REGS_SYSCALL_REGS(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
> +#endif
> +

I think that name is misleading if an architecture doesn't implement syscall 
wrappers, since you are simply getting access to the kprobe pt_regs, rather 
than the syscall pt_regs. This can perhaps be named PT_REGS_SYSCALL_UNWRAP() or 
such to make that clear.

Also, should this just be keyed off a simpler HAS_SYSCALL_WRAPPER or such, 
rather than the other way around?

diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
index 032ba809f3e57a..c72f285578d3fc 100644
--- a/tools/lib/bpf/bpf_tracing.h
+++ b/tools/lib/bpf/bpf_tracing.h
@@ -110,6 +110,8 @@
 
 #endif /* __i386__ */
 
+#define HAS_SYSCALL_WRAPPER
+
 #endif /* __KERNEL__ || __VMLINUX_H__ */
 
 #elif defined(bpf_target_s390)
@@ -126,6 +128,7 @@
 #define __PT_RC_REG gprs[2]
 #define __PT_SP_REG gprs[15]
 #define __PT_IP_REG psw.addr
+#define HAS_SYSCALL_WRAPPER
 
 #elif defined(bpf_target_arm)
 
@@ -154,6 +157,7 @@
 #define __PT_RC_REG regs[0]
 #define __PT_SP_REG sp
 #define __PT_IP_REG pc
+#define HAS_SYSCALL_WRAPPER
 
 #elif defined(bpf_target_mips)
 

We can then simply do:

#ifdef HAS_SYSCALL_WRAPPER
#define PT_REGS_SYSCALL_UNWRAP(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
#else
#define PT_REGS_SYSCALL_unwRAP(ctx) ((struct pt_regs *)(ctx))
#endif


Taking this a bit further, it would be nice if we can fold in progs/bpf_misc.h 
into bpf_traching.h by also including SYS_PREFIX.


- Naveen

Re: [PATCH bpf-next v3 05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro

[Andrii Nakryiko]

On Fri, Feb 4, 2022 at 8:46 AM Naveen N. Rao
<naveen.n.rao@linux.vnet.ibm.com> wrote:
>
> Ilya Leoshkevich wrote:
> > Some architectures pass a pointer to struct pt_regs to syscall
> > handlers, others unpack it into individual function parameters.
>
> I think that is just dependent on ARCH_HAS_SYSCALL_WRAPPER, so only x86,
> arm64 and s390 pass pointers to pt_regs to syscall entry points.
>
> > Introduce a macro to describe what a particular arch does, using
> > `passing pt_regs *` as a default.
> >
> > Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> > ---
> >  tools/lib/bpf/bpf_tracing.h | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
> > index 30f0964f8c9e..08d2990c006f 100644
> > --- a/tools/lib/bpf/bpf_tracing.h
> > +++ b/tools/lib/bpf/bpf_tracing.h
> > @@ -334,6 +334,15 @@ struct pt_regs;
> >
> >  #endif /* defined(bpf_target_defined) */
> >
> > +/*
> > + * When invoked from a syscall handler kprobe, returns a pointer to a
> > + * struct pt_regs containing syscall arguments and suitable for passing to
> > + * PT_REGS_PARMn_SYSCALL() and PT_REGS_PARMn_CORE_SYSCALL().
> > + */
> > +#ifndef PT_REGS_SYSCALL_REGS
> > +#define PT_REGS_SYSCALL_REGS(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
> > +#endif
> > +
>
> I think that name is misleading if an architecture doesn't implement syscall
> wrappers, since you are simply getting access to the kprobe pt_regs, rather
> than the syscall pt_regs. This can perhaps be named PT_REGS_SYSCALL_UNWRAP() or
> such to make that clear.

UNWRAP implies that there is something to unwrap, always. In case of
s390x, for example, there is nothing to unwrap. So I think
PT_REGS_SYSCALL_REGS() makes more sense, it just fetches correct
pt_regs to work with to get syscall input arguments (and it might be
exactly the same pt_regs that are passed in).

I think in practice most users won't ever have to use this, as we'll
add BPF_KPROBE_SYSCALL() macro, similar to BPF_KPROBE that we have
now, but specific to syscall kprobe.

>
> Also, should this just be keyed off a simpler HAS_SYSCALL_WRAPPER or such,
> rather than the other way around?

I think the way Ilya did it is totally fine.

>
> diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
> index 032ba809f3e57a..c72f285578d3fc 100644
> --- a/tools/lib/bpf/bpf_tracing.h
> +++ b/tools/lib/bpf/bpf_tracing.h
> @@ -110,6 +110,8 @@
>
>  #endif /* __i386__ */
>
> +#define HAS_SYSCALL_WRAPPER
> +
>  #endif /* __KERNEL__ || __VMLINUX_H__ */
>
>  #elif defined(bpf_target_s390)
> @@ -126,6 +128,7 @@
>  #define __PT_RC_REG gprs[2]
>  #define __PT_SP_REG gprs[15]
>  #define __PT_IP_REG psw.addr
> +#define HAS_SYSCALL_WRAPPER
>
>  #elif defined(bpf_target_arm)
>
> @@ -154,6 +157,7 @@
>  #define __PT_RC_REG regs[0]
>  #define __PT_SP_REG sp
>  #define __PT_IP_REG pc
> +#define HAS_SYSCALL_WRAPPER
>
>  #elif defined(bpf_target_mips)
>
>
> We can then simply do:
>
> #ifdef HAS_SYSCALL_WRAPPER
> #define PT_REGS_SYSCALL_UNWRAP(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
> #else
> #define PT_REGS_SYSCALL_unwRAP(ctx) ((struct pt_regs *)(ctx))
> #endif
>
>
> Taking this a bit further, it would be nice if we can fold in progs/bpf_misc.h
> into bpf_traching.h by also including SYS_PREFIX.

As far as I know, SYS_PREFIX depends not just on architecture but also
on kernel version (older versions of x86-64 kernels didn't need that
prefix). For selftests, given they follow the latest version of kernel
it's ok to always append SYS_PREFIX, but generally speaking for user
BPF apps, they would need to be more careful and check whether they
need SYS_PREFIX or not. So I don't want to add SYS_PREFIX to
bpf_tracing.h because it's misleading.

>
>
> - Naveen
>

Re: [PATCH bpf-next v3 02/11] s390/bpf: Add orig_gpr2 to user_pt_regs

[Heiko Carstens]

On Fri, Feb 04, 2022 at 03:50:09PM +0100, Ilya Leoshkevich wrote:
> user_pt_regs is used by eBPF in order to access userspace registers -
> see commit 466698e654e8 ("s390/bpf: correct broken uapi for
> BPF_PROG_TYPE_PERF_EVENT program type"). In order to access the first
> syscall argument from eBPF programs, we need to export orig_gpr2.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  arch/s390/include/asm/ptrace.h      | 2 +-
>  arch/s390/include/uapi/asm/ptrace.h | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)

Acked-by: Heiko Carstens <hca@linux.ibm.com>

Re: [PATCH bpf-next v3 05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro

[Naveen N. Rao]

Andrii Nakryiko wrote:
> On Fri, Feb 4, 2022 at 8:46 AM Naveen N. Rao
> <naveen.n.rao@linux.vnet.ibm.com> wrote:
>>
>> Ilya Leoshkevich wrote:
>> > Some architectures pass a pointer to struct pt_regs to syscall
>> > handlers, others unpack it into individual function parameters.
>>
>> I think that is just dependent on ARCH_HAS_SYSCALL_WRAPPER, so only x86,
>> arm64 and s390 pass pointers to pt_regs to syscall entry points.
>>
>> > Introduce a macro to describe what a particular arch does, using
>> > `passing pt_regs *` as a default.
>> >
>> > Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
>> > ---
>> >  tools/lib/bpf/bpf_tracing.h | 9 +++++++++
>> >  1 file changed, 9 insertions(+)
>> >
>> > diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
>> > index 30f0964f8c9e..08d2990c006f 100644
>> > --- a/tools/lib/bpf/bpf_tracing.h
>> > +++ b/tools/lib/bpf/bpf_tracing.h
>> > @@ -334,6 +334,15 @@ struct pt_regs;
>> >
>> >  #endif /* defined(bpf_target_defined) */
>> >
>> > +/*
>> > + * When invoked from a syscall handler kprobe, returns a pointer to a
>> > + * struct pt_regs containing syscall arguments and suitable for passing to
>> > + * PT_REGS_PARMn_SYSCALL() and PT_REGS_PARMn_CORE_SYSCALL().
>> > + */
>> > +#ifndef PT_REGS_SYSCALL_REGS
>> > +#define PT_REGS_SYSCALL_REGS(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
>> > +#endif
>> > +
>>
>> I think that name is misleading if an architecture doesn't implement syscall
>> wrappers, since you are simply getting access to the kprobe pt_regs, rather
>> than the syscall pt_regs. This can perhaps be named PT_REGS_SYSCALL_UNWRAP() or
>> such to make that clear.
> 
> UNWRAP implies that there is something to unwrap, always. In case of
> s390x, for example, there is nothing to unwrap. So I think
> PT_REGS_SYSCALL_REGS() makes more sense, it just fetches correct
> pt_regs to work with to get syscall input arguments (and it might be
> exactly the same pt_regs that are passed in).
> 
> I think in practice most users won't ever have to use this, as we'll
> add BPF_KPROBE_SYSCALL() macro, similar to BPF_KPROBE that we have
> now, but specific to syscall kprobe.

That will be very nice.

> 
>>
>> Also, should this just be keyed off a simpler HAS_SYSCALL_WRAPPER or such,
>> rather than the other way around?
> 
> I think the way Ilya did it is totally fine.
> 
>>
>> diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
>> index 032ba809f3e57a..c72f285578d3fc 100644
>> --- a/tools/lib/bpf/bpf_tracing.h
>> +++ b/tools/lib/bpf/bpf_tracing.h
>> @@ -110,6 +110,8 @@
>>
>>  #endif /* __i386__ */
>>
>> +#define HAS_SYSCALL_WRAPPER
>> +
>>  #endif /* __KERNEL__ || __VMLINUX_H__ */
>>
>>  #elif defined(bpf_target_s390)
>> @@ -126,6 +128,7 @@
>>  #define __PT_RC_REG gprs[2]
>>  #define __PT_SP_REG gprs[15]
>>  #define __PT_IP_REG psw.addr
>> +#define HAS_SYSCALL_WRAPPER
>>
>>  #elif defined(bpf_target_arm)
>>
>> @@ -154,6 +157,7 @@
>>  #define __PT_RC_REG regs[0]
>>  #define __PT_SP_REG sp
>>  #define __PT_IP_REG pc
>> +#define HAS_SYSCALL_WRAPPER
>>
>>  #elif defined(bpf_target_mips)
>>
>>
>> We can then simply do:
>>
>> #ifdef HAS_SYSCALL_WRAPPER
>> #define PT_REGS_SYSCALL_UNWRAP(ctx) ((struct pt_regs *)PT_REGS_PARM1(ctx))
>> #else
>> #define PT_REGS_SYSCALL_unwRAP(ctx) ((struct pt_regs *)(ctx))
>> #endif
>>
>>
>> Taking this a bit further, it would be nice if we can fold in progs/bpf_misc.h
>> into bpf_traching.h by also including SYS_PREFIX.
> 
> As far as I know, SYS_PREFIX depends not just on architecture but also
> on kernel version (older versions of x86-64 kernels didn't need that
> prefix). For selftests, given they follow the latest version of kernel
> it's ok to always append SYS_PREFIX, but generally speaking for user
> BPF apps, they would need to be more careful and check whether they
> need SYS_PREFIX or not. So I don't want to add SYS_PREFIX to
> bpf_tracing.h because it's misleading.

That makes sense, thanks.


- Naveen

Re: [PATCH bpf-next v3 08/11] libbpf: Fix accessing syscall arguments on powerpc

[Naveen N. Rao]

Ilya Leoshkevich wrote:
> powerpc's syscall handlers get "unpacked" arguments instead of a
> struct pt_regs pointer. Indicate this to libbpf using
> PT_REGS_SYSCALL_REGS macro.
> 
> Reported-by: Heiko Carstens <hca@linux.ibm.com>
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  tools/lib/bpf/bpf_tracing.h | 1 +
>  1 file changed, 1 insertion(+)

Tested-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>

> 
> diff --git a/tools/lib/bpf/bpf_tracing.h b/tools/lib/bpf/bpf_tracing.h
> index 317bee0fd8e4..f9d22ea0af97 100644
> --- a/tools/lib/bpf/bpf_tracing.h
> +++ b/tools/lib/bpf/bpf_tracing.h
> @@ -181,6 +181,7 @@
>  #define __PT_RC_REG gpr[3]
>  #define __PT_SP_REG sp
>  #define __PT_IP_REG nip
> +#define PT_REGS_SYSCALL_REGS(ctx) ctx
> 
>  #elif defined(bpf_target_sparc)
> 
> -- 
> 2.34.1
> 
> 

Re: [PATCH bpf-next v3 02/11] s390/bpf: Add orig_gpr2 to user_pt_regs

[Vasily Gorbik]

On Fri, Feb 04, 2022 at 03:50:09PM +0100, Ilya Leoshkevich wrote:
> user_pt_regs is used by eBPF in order to access userspace registers -
> see commit 466698e654e8 ("s390/bpf: correct broken uapi for
> BPF_PROG_TYPE_PERF_EVENT program type"). In order to access the first
> syscall argument from eBPF programs, we need to export orig_gpr2.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>  arch/s390/include/asm/ptrace.h      | 2 +-
>  arch/s390/include/uapi/asm/ptrace.h | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)

Acked-by: Vasily Gorbik <gor@linux.ibm.com>

Re: [PATCH bpf-next v3 00/11] libbpf: Fix accessing syscall arguments

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Fri,  4 Feb 2022 15:50:07 +0100 you wrote:
> libbpf now has macros to access syscall arguments in an
> architecture-agnostic manner, but unfortunately they have a number of
> issues on non-Intel arches, which this series aims to fix.
> 
> v1: https://lore.kernel.org/bpf/20220201234200.1836443-1-iii@linux.ibm.com/
> v1 -> v2:
> * Put orig_gpr2 in place of args[1] on s390 (Vasily).
> * Fix arm64, powerpc and riscv (Heiko).
> 
> [...]

Here is the summary with links:
  - [bpf-next,v3,01/11] arm64/bpf: Add orig_x0 to user_pt_regs
    https://git.kernel.org/bpf/bpf-next/c/d473f4062165
  - [bpf-next,v3,02/11] s390/bpf: Add orig_gpr2 to user_pt_regs
    https://git.kernel.org/bpf/bpf-next/c/61f88e88f263
  - [bpf-next,v3,03/11] selftests/bpf: Fix an endianness issue in bpf_syscall_macro test
    https://git.kernel.org/bpf/bpf-next/c/a936c141cbe4
  - [bpf-next,v3,04/11] libbpf: Add __PT_PARM1_REG_SYSCALL macro
    https://git.kernel.org/bpf/bpf-next/c/3a9d84aafb8c
  - [bpf-next,v3,05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro
    https://git.kernel.org/bpf/bpf-next/c/b62a862d42f5
  - [bpf-next,v3,06/11] selftests/bpf: Use PT_REGS_SYSCALL_REGS in bpf_syscall_macro
    https://git.kernel.org/bpf/bpf-next/c/730809c15ac2
  - [bpf-next,v3,07/11] libbpf: Fix accessing the first syscall argument on arm64
    https://git.kernel.org/bpf/bpf-next/c/8b9b06ad4726
  - [bpf-next,v3,08/11] libbpf: Fix accessing syscall arguments on powerpc
    https://git.kernel.org/bpf/bpf-next/c/f5af16d0ae28
  - [bpf-next,v3,09/11] libbpf: Fix accessing program counter on riscv
    https://git.kernel.org/bpf/bpf-next/c/27870c91b5c7
  - [bpf-next,v3,10/11] libbpf: Fix accessing syscall arguments on riscv
    https://git.kernel.org/bpf/bpf-next/c/5860b82236c6
  - [bpf-next,v3,11/11] libbpf: Fix accessing the first syscall argument on s390
    https://git.kernel.org/bpf/bpf-next/c/088d6aafd5bb

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH bpf-next v3 00/11] libbpf: Fix accessing syscall arguments

[Andrii Nakryiko]

On Sat, Feb 5, 2022 at 12:30 PM <patchwork-bot+netdevbpf@kernel.org> wrote:
>
> Hello:
>
> This series was applied to bpf/bpf-next.git (master)
> by Andrii Nakryiko <andrii@kernel.org>:
>
> On Fri,  4 Feb 2022 15:50:07 +0100 you wrote:
> > libbpf now has macros to access syscall arguments in an
> > architecture-agnostic manner, but unfortunately they have a number of
> > issues on non-Intel arches, which this series aims to fix.
> >
> > v1: https://lore.kernel.org/bpf/20220201234200.1836443-1-iii@linux.ibm.com/
> > v1 -> v2:
> > * Put orig_gpr2 in place of args[1] on s390 (Vasily).
> > * Fix arm64, powerpc and riscv (Heiko).
> >
> > [...]
>
> Here is the summary with links:
>   - [bpf-next,v3,01/11] arm64/bpf: Add orig_x0 to user_pt_regs
>     https://git.kernel.org/bpf/bpf-next/c/d473f4062165
>   - [bpf-next,v3,02/11] s390/bpf: Add orig_gpr2 to user_pt_regs
>     https://git.kernel.org/bpf/bpf-next/c/61f88e88f263
>   - [bpf-next,v3,03/11] selftests/bpf: Fix an endianness issue in bpf_syscall_macro test
>     https://git.kernel.org/bpf/bpf-next/c/a936c141cbe4
>   - [bpf-next,v3,04/11] libbpf: Add __PT_PARM1_REG_SYSCALL macro
>     https://git.kernel.org/bpf/bpf-next/c/3a9d84aafb8c
>   - [bpf-next,v3,05/11] libbpf: Add PT_REGS_SYSCALL_REGS macro
>     https://git.kernel.org/bpf/bpf-next/c/b62a862d42f5
>   - [bpf-next,v3,06/11] selftests/bpf: Use PT_REGS_SYSCALL_REGS in bpf_syscall_macro
>     https://git.kernel.org/bpf/bpf-next/c/730809c15ac2
>   - [bpf-next,v3,07/11] libbpf: Fix accessing the first syscall argument on arm64
>     https://git.kernel.org/bpf/bpf-next/c/8b9b06ad4726
>   - [bpf-next,v3,08/11] libbpf: Fix accessing syscall arguments on powerpc
>     https://git.kernel.org/bpf/bpf-next/c/f5af16d0ae28
>   - [bpf-next,v3,09/11] libbpf: Fix accessing program counter on riscv
>     https://git.kernel.org/bpf/bpf-next/c/27870c91b5c7
>   - [bpf-next,v3,10/11] libbpf: Fix accessing syscall arguments on riscv
>     https://git.kernel.org/bpf/bpf-next/c/5860b82236c6
>   - [bpf-next,v3,11/11] libbpf: Fix accessing the first syscall argument on s390
>     https://git.kernel.org/bpf/bpf-next/c/088d6aafd5bb
>

I've backed out this patch set from bpf-next for now while we are
deciding the best way to deal with s390x and arm64 arches without
breaking UAPIs.

> You are awesome, thank you!
> --
> Deet-doot-dot, I am a bot.
> https://korg.docs.kernel.org/patchwork/pwbot.html
>
>