[PATCH bpf-next v3] bpf, docs: Explain helper functions

[PATCH bpf-next v3] bpf, docs: Explain helper functions

[Dave Thaler]

From: Dave Thaler <dthaler@microsoft.com>

Add text explaining helper functions.
Note that text about runtime functions (kfuncs) is part of a separate patch,
not this one.

Signed-off-by: Dave Thaler <dthaler@microsoft.com>
---
V1 -> V2: addressed comments from Alexei and Stanislav

V2 -> V3: addressed comments from David Vernet
---
 Documentation/bpf/clang-notes.rst     |  6 ++++++
 Documentation/bpf/instruction-set.rst | 19 ++++++++++++++++++-
 Documentation/bpf/linux-notes.rst     |  8 ++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/Documentation/bpf/clang-notes.rst b/Documentation/bpf/clang-notes.rst
index 528feddf2db..2c872a1ee08 100644
--- a/Documentation/bpf/clang-notes.rst
+++ b/Documentation/bpf/clang-notes.rst
@@ -20,6 +20,12 @@ Arithmetic instructions
 For CPU versions prior to 3, Clang v7.0 and later can enable ``BPF_ALU`` support with
 ``-Xclang -target-feature -Xclang +alu32``.  In CPU version 3, support is automatically included.
 
+Jump instructions
+=================
+
+If ``-O0`` is used, Clang will generate the ``BPF_CALL | BPF_X | BPF_JMP`` (0x8d)
+instruction, which is not supported by the Linux kernel verifier.
+
 Atomic operations
 =================
 
diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
index af515de5fc3..148dd2a2e39 100644
--- a/Documentation/bpf/instruction-set.rst
+++ b/Documentation/bpf/instruction-set.rst
@@ -239,7 +239,7 @@ BPF_JSET  0x40   PC += off if dst & src
 BPF_JNE   0x50   PC += off if dst != src
 BPF_JSGT  0x60   PC += off if dst > src     signed
 BPF_JSGE  0x70   PC += off if dst >= src    signed
-BPF_CALL  0x80   function call
+BPF_CALL  0x80   function call              see `Helper functions`_
 BPF_EXIT  0x90   function / program return  BPF_JMP only
 BPF_JLT   0xa0   PC += off if dst < src     unsigned
 BPF_JLE   0xb0   PC += off if dst <= src    unsigned
@@ -250,6 +250,23 @@ BPF_JSLE  0xd0   PC += off if dst <= src    signed
 The eBPF program needs to store the return value into register R0 before doing a
 BPF_EXIT.
 
+Helper functions
+~~~~~~~~~~~~~~~~
+
+Helper functions are a concept whereby BPF programs can call into a
+set of function calls exposed by the runtime.  Each helper
+function is identified by an integer used in a ``BPF_CALL`` instruction.
+The available helper functions may differ for each program type.
+
+Conceptually, each helper function is implemented with a commonly shared function
+signature defined as:
+
+  u64 function(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
+
+In actuality, each helper function is defined as taking between 0 and 5 arguments,
+with the remaining registers being ignored.  The definition of a helper function
+is responsible for specifying the type (e.g., integer, pointer, etc.) of the value returned,
+the number of arguments, and the type of each argument.
 
 Load and store instructions
 ===========================
diff --git a/Documentation/bpf/linux-notes.rst b/Documentation/bpf/linux-notes.rst
index 956b0c86699..f43b9c797bc 100644
--- a/Documentation/bpf/linux-notes.rst
+++ b/Documentation/bpf/linux-notes.rst
@@ -12,6 +12,14 @@ Byte swap instructions
 
 ``BPF_FROM_LE`` and ``BPF_FROM_BE`` exist as aliases for ``BPF_TO_LE`` and ``BPF_TO_BE`` respectively.
 
+Jump instructions
+=================
+
+``BPF_CALL | BPF_X | BPF_JMP`` (0x8d), where the helper function
+integer would be read from a specified register, is not currently supported
+by the verifier.  Any programs with this instruction will fail to load
+until such support is added.
+
 Legacy BPF Packet access instructions
 =====================================
 
-- 
2.33.4

Re: [Bpf] [PATCH bpf-next v3] bpf, docs: Explain helper functions

[Alexei Starovoitov]

On Mon, Feb 20, 2023 at 2:52 PM Dave Thaler
<dthaler1968=40googlemail.com@dmarc.ietf.org> wrote:
>
> From: Dave Thaler <dthaler@microsoft.com>
>
> Add text explaining helper functions.
> Note that text about runtime functions (kfuncs) is part of a separate patch,
> not this one.
>
> Signed-off-by: Dave Thaler <dthaler@microsoft.com>
> ---
> V1 -> V2: addressed comments from Alexei and Stanislav
>
> V2 -> V3: addressed comments from David Vernet
> ---
>  Documentation/bpf/clang-notes.rst     |  6 ++++++
>  Documentation/bpf/instruction-set.rst | 19 ++++++++++++++++++-
>  Documentation/bpf/linux-notes.rst     |  8 ++++++++
>  3 files changed, 32 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/bpf/clang-notes.rst b/Documentation/bpf/clang-notes.rst
> index 528feddf2db..2c872a1ee08 100644
> --- a/Documentation/bpf/clang-notes.rst
> +++ b/Documentation/bpf/clang-notes.rst
> @@ -20,6 +20,12 @@ Arithmetic instructions
>  For CPU versions prior to 3, Clang v7.0 and later can enable ``BPF_ALU`` support with
>  ``-Xclang -target-feature -Xclang +alu32``.  In CPU version 3, support is automatically included.
>
> +Jump instructions
> +=================
> +
> +If ``-O0`` is used, Clang will generate the ``BPF_CALL | BPF_X | BPF_JMP`` (0x8d)
> +instruction, which is not supported by the Linux kernel verifier.

This is fine here.

> +
>  Atomic operations
>  =================
>
> diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
> index af515de5fc3..148dd2a2e39 100644
> --- a/Documentation/bpf/instruction-set.rst
> +++ b/Documentation/bpf/instruction-set.rst
> @@ -239,7 +239,7 @@ BPF_JSET  0x40   PC += off if dst & src
>  BPF_JNE   0x50   PC += off if dst != src
>  BPF_JSGT  0x60   PC += off if dst > src     signed
>  BPF_JSGE  0x70   PC += off if dst >= src    signed
> -BPF_CALL  0x80   function call
> +BPF_CALL  0x80   function call              see `Helper functions`_
>  BPF_EXIT  0x90   function / program return  BPF_JMP only
>  BPF_JLT   0xa0   PC += off if dst < src     unsigned
>  BPF_JLE   0xb0   PC += off if dst <= src    unsigned
> @@ -250,6 +250,23 @@ BPF_JSLE  0xd0   PC += off if dst <= src    signed
>  The eBPF program needs to store the return value into register R0 before doing a
>  BPF_EXIT.
>
> +Helper functions
> +~~~~~~~~~~~~~~~~
> +
> +Helper functions are a concept whereby BPF programs can call into a
> +set of function calls exposed by the runtime.  Each helper
> +function is identified by an integer used in a ``BPF_CALL`` instruction.
> +The available helper functions may differ for each program type.
> +
> +Conceptually, each helper function is implemented with a commonly shared function
> +signature defined as:
> +
> +  u64 function(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
> +
> +In actuality, each helper function is defined as taking between 0 and 5 arguments,
> +with the remaining registers being ignored.  The definition of a helper function
> +is responsible for specifying the type (e.g., integer, pointer, etc.) of the value returned,
> +the number of arguments, and the type of each argument.

Above is correct, but it aims to describe the calling convention
which should be done in a separate BPF psABI doc and not in
instruction-set.rst.
And if we start describing calling convention we should talk
about promotion rules, sign extensions, expectations for return values,
for passing structs by value, etc.

>  Load and store instructions
>  ===========================
> diff --git a/Documentation/bpf/linux-notes.rst b/Documentation/bpf/linux-notes.rst
> index 956b0c86699..f43b9c797bc 100644
> --- a/Documentation/bpf/linux-notes.rst
> +++ b/Documentation/bpf/linux-notes.rst
> @@ -12,6 +12,14 @@ Byte swap instructions
>
>  ``BPF_FROM_LE`` and ``BPF_FROM_BE`` exist as aliases for ``BPF_TO_LE`` and ``BPF_TO_BE`` respectively.
>
> +Jump instructions
> +=================
> +
> +``BPF_CALL | BPF_X | BPF_JMP`` (0x8d), where the helper function
> +integer would be read from a specified register, is not currently supported
> +by the verifier.  Any programs with this instruction will fail to load
> +until such support is added.

This is fine here as well.

RE: [Bpf] [PATCH bpf-next v3] bpf, docs: Explain helper functions

[Dave Thaler]

Alexei Starovoitov <alexei.starovoitov@gmail.com> writes:
[...]
> > +Helper functions
> > +~~~~~~~~~~~~~~~~
> > +
> > +Helper functions are a concept whereby BPF programs can call into a
> > +set of function calls exposed by the runtime.  Each helper function
> > +is identified by an integer used in a ``BPF_CALL`` instruction.
> > +The available helper functions may differ for each program type.
> > +
> > +Conceptually, each helper function is implemented with a commonly
> > +shared function signature defined as:
> > +
> > +  u64 function(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
> > +
> > +In actuality, each helper function is defined as taking between 0 and
> > +5 arguments, with the remaining registers being ignored.  The
> > +definition of a helper function is responsible for specifying the
> > +type (e.g., integer, pointer, etc.) of the value returned, the number of
> arguments, and the type of each argument.
> 
> Above is correct, but it aims to describe the calling convention which should
> be done in a separate BPF psABI doc and not in instruction-set.rst.
> And if we start describing calling convention we should talk about promotion
> rules, sign extensions, expectations for return values, for passing structs by
> value, etc.

The instruction itself requires defining the concept of a helper function, so is the
text in question the part starting with "Conceptually," down to the end of the
quoted text?

Since there is no separate BPF psABI document (and I'm not sure the scope of
that document myself) can we put it here for now and move it when that doc
is created?   If not, what part of the text above would be in a separate document?

[...]

Dave

Re: [Bpf] [PATCH bpf-next v3] bpf, docs: Explain helper functions

[Alexei Starovoitov]

On Wed, Feb 22, 2023 at 2:23 PM Dave Thaler <dthaler@microsoft.com> wrote:
>
> Alexei Starovoitov <alexei.starovoitov@gmail.com> writes:
> [...]
> > > +Helper functions
> > > +~~~~~~~~~~~~~~~~
> > > +
> > > +Helper functions are a concept whereby BPF programs can call into a
> > > +set of function calls exposed by the runtime.  Each helper function
> > > +is identified by an integer used in a ``BPF_CALL`` instruction.
> > > +The available helper functions may differ for each program type.
> > > +
> > > +Conceptually, each helper function is implemented with a commonly
> > > +shared function signature defined as:
> > > +
> > > +  u64 function(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
> > > +
> > > +In actuality, each helper function is defined as taking between 0 and
> > > +5 arguments, with the remaining registers being ignored.  The
> > > +definition of a helper function is responsible for specifying the
> > > +type (e.g., integer, pointer, etc.) of the value returned, the number of
> > arguments, and the type of each argument.
> >
> > Above is correct, but it aims to describe the calling convention which should
> > be done in a separate BPF psABI doc and not in instruction-set.rst.
> > And if we start describing calling convention we should talk about promotion
> > rules, sign extensions, expectations for return values, for passing structs by
> > value, etc.
>
> The instruction itself requires defining the concept of a helper function,

Not really. BPF_CALL instruction doesn't have to define what it's calling
and how the call is being made.
Typical cpu will define CALL insn as:

1. pushes the return address on the stack.
2. changes IP to the call destination

That's it. Mechanics of CALL have no overlap with calling convention.
Different languages and operating systems can do it differently.
BPF ISA doc should describe mechanics of instructions only.

> so is the
> text in question the part starting with "Conceptually," down to the end of the
> quoted text?

Right, that part doesn't belong in BPF ISA doc.
Even
"Each helper function is identified by an integer used in a
``BPF_CALL`` instruction."
arguably belongs in psABI, since it's a bpf's flavor of relocations.
Other cpus do it in ELF relocations while bpf does it inline as part
of instruction encoding. That is not an ISA.
It's a protocol between user and kernel.
We just happen to use bits in instruction to indicate relocation
that kernel has to perform before executing call insn.
At the end JITs will map BPF_CALL insn one to one to CPU call insn
on architectures where calling conventions match.

> Since there is no separate BPF psABI document (and I'm not sure the scope of
> that document myself) can we put it here for now and move it when that doc
> is created?   If not, what part of the text above would be in a separate document?

BPF psABI should look like:
https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf

Ours will be shorter, hopefully.
We haven't defined a bunch of things yet. Like how to pass 6th argument.

Re: [Bpf] [PATCH bpf-next v3] bpf, docs: Explain helper functions

[Jose E. Marchesi]

> BPF psABI should look like:
> https://raw.githubusercontent.com/wiki/hjl-tools/x86-psABI/x86-64-psABI-1.0.pdf

LaTeX sources: https://gitlab.com/x86-psABIs/x86-64-ABI