Re: [PATCH v2 bpf-next 0/4] bpf: Add detection of kfuncs.

[Alexei Starovoitov <alexei.starovoitov@gmail.com>]

On Tue, Jul 25, 2023 at 1:45 PM Matt Bobrowski <mattbobrowski@google.com> wrote:
>
> Hey Alexei/Andrii,
>
> On Fri, Mar 17, 2023 at 01:19:16PM -0700, Alexei Starovoitov wrote:
> > From: Alexei Starovoitov <ast@kernel.org>
> >
> > Allow BPF programs detect at load time whether particular kfunc exists.
>
> So, I'm running a GCC built 6.3.7 Linux kernel and I'm attempting to
> detect whether a specific kfunc i.e. bpf_rcu_read_lock/unlock() exists
> using the bpf_ksym_exists() macro. However, I'm running into several
> BPF verifier constraints that I'm not entirely sure how to work around
> on the aforementioned Linux kernel version, and hence why I'm reaching
> out for some guidance.
>
> The first BPF verifier constraint that I'm running into is that prior
> to commit 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to
> kfunc"), it seems that the ld_imm64 instruction with BPF_PSEUDO_BTF_ID
> can only hold a ksym address for the kind KIND_VAR. However, when
> attempting to use the kfuncs bpf_rcu_read_lock/unlock() from a BPF
> program, the kind associated with the BPF_PSEUDO_BTF_ID is actually
> KIND_FUNC, and therefore trips over this BPF verifier.
>
> The code within the example BPF program is along the lines of the
> following:
> ```
> ...
> void bpf_rcu_read_lock(void) __ksym __weak;
> void bpf_rcu_read_unlock(void) __ksym __weak;
> ...
> if (bpf_ksym_exists(bpf_rcu_read_lock)) {
>    bpf_rcu_read_lock();
> }
> ...
> if (bpf_ksym_exists(bpf_rcu_read_unlock)) {
>    bpf_rcu_read_unlock();
> }
> ...
> ```
>
> The BPF verifier error message that is generated on a 6.3.7 Linux
> kernel when attempting to load a BPF program that makes use of the
> above approach is as follows:
>    * "pseudo btf_id {BTF_ID} in ldimm64 isn't KIND_VAR"
>
> The second BPF verifier constraint comes from attempting to work
> around the first BPF verifier constraint that I've mentioned
> above. This is trivially by dropping the conditionals that contain the
> bpf_ksym_exists() check and unconditionally calling the kfuncs
> bpf_rcu_read_lock/unlock().
>
> The code within the example BPF program is along the lines of the
> following:
> ```
> ...
> void bpf_rcu_read_lock(void) __ksym __weak;
> void bpf_rcu_read_unlock(void) __ksym __weak;
> ...
> bpf_rcu_read_lock();
> ...
> bpf_rcu_read_unlock();
> ...
> ```
>
> However, in this case the BPF verifier error message that is generated
> on a 6.3.7 Linux kernel is as follows:
>    * "no vmlinux btf rcu tag support for kfunc bpf_rcu_read_lock"
>
> This approach would be suboptimal anyway as the BPF program would fail
> to load on older Linux kernels complaining that the kfunc is
> referenced but couldn't be resolved.
>
> Having said this, what's the best way to resolve this on a 6.3.7 Linux
> kernel? The first BPF program I mentioned above making use of the
> bpf_ksym_exists() macro works on a 6.4 Linux kernel with commit
> 58aa2afbb1e6 ("bpf: Allow ld_imm64 instruction to point to kfunc")
> applied. Also, the first BPF program I mentioned above works on a
> 6.1.* Linux kernel...

Backport of that commit to 6.3.x is probably the only way.

And I don't think bpf_ksym_exists() actually works on 6.1