* Headers for whitelisted kernel functions available to BPF programs @ 2021-06-03 18:01 Kenny Ho 2021-06-04 6:13 ` Martin KaFai Lau 0 siblings, 1 reply; 8+ messages in thread From: Kenny Ho @ 2021-06-03 18:01 UTC (permalink / raw) To: Martin KaFai Lau, bpf Hi, I understand that helper functions available to bpf programs are listed in include/uapi/linux/bpf.h and kernel headers can be made available at /sys/kernel/kheaders.tar.xz with CONFIG_IKHEADERS. But with the support of calling kernel functions from bpf programs, how would one know which functions are whitelisted? Are the headers for these whitelisted functions available via something like "bpftool btf dump file /sys/kernel/btf/vmlinux format c"? Regards, Kenny ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-03 18:01 Headers for whitelisted kernel functions available to BPF programs Kenny Ho @ 2021-06-04 6:13 ` Martin KaFai Lau 2021-06-04 16:10 ` Yonghong Song 0 siblings, 1 reply; 8+ messages in thread From: Martin KaFai Lau @ 2021-06-04 6:13 UTC (permalink / raw) To: Kenny Ho; +Cc: bpf, Yonghong Song On Thu, Jun 03, 2021 at 02:01:13PM -0400, Kenny Ho wrote: > Hi, > > I understand that helper functions available to bpf programs are > listed in include/uapi/linux/bpf.h and kernel headers can be made > available at /sys/kernel/kheaders.tar.xz with CONFIG_IKHEADERS. But > with the support of calling kernel functions from bpf programs, how > would one know which functions are whitelisted? Are the headers for > these whitelisted functions available via something like "bpftool btf > dump file /sys/kernel/btf/vmlinux format c"? Like other whitelisted functions in BPF, the list is not in the vmlinux btf now but could be a BTF extension in the future (Cc: Yonghong). Making the kfunc call whitelist more accessible is useful in general. The bpf tcp-cc struct_ops is the only prog type supporting kfunc call. What is your use case to introspect this whitelist? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-04 6:13 ` Martin KaFai Lau @ 2021-06-04 16:10 ` Yonghong Song 2021-06-04 20:47 ` Kenny Ho 0 siblings, 1 reply; 8+ messages in thread From: Yonghong Song @ 2021-06-04 16:10 UTC (permalink / raw) To: Martin KaFai Lau, Kenny Ho; +Cc: bpf On 6/3/21 11:13 PM, Martin KaFai Lau wrote: > On Thu, Jun 03, 2021 at 02:01:13PM -0400, Kenny Ho wrote: >> Hi, >> >> I understand that helper functions available to bpf programs are >> listed in include/uapi/linux/bpf.h and kernel headers can be made >> available at /sys/kernel/kheaders.tar.xz with CONFIG_IKHEADERS. But >> with the support of calling kernel functions from bpf programs, how >> would one know which functions are whitelisted? Are the headers for >> these whitelisted functions available via something like "bpftool btf >> dump file /sys/kernel/btf/vmlinux format c"? > Like other whitelisted functions in BPF, the list is not in the vmlinux btf > now but could be a BTF extension in the future (Cc: Yonghong). Currently I am working to add a custom dwarf attribute in llvm so we can annotate global/static variables, global/static functions, function arguments, struct/unions, struct/union fields, with arbitrary strings and these strings will be preserved in dwarf so we can eventually encode in BTF. The WIP patch is here https://reviews.llvm.org/D103667. > > Making the kfunc call whitelist more accessible is useful in general. > The bpf tcp-cc struct_ops is the only prog type supporting kfunc call. > What is your use case to introspect this whitelist? Agree. It would be good if you can share your use case. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-04 16:10 ` Yonghong Song @ 2021-06-04 20:47 ` Kenny Ho 2021-06-08 19:51 ` Kenny Ho 0 siblings, 1 reply; 8+ messages in thread From: Kenny Ho @ 2021-06-04 20:47 UTC (permalink / raw) To: Yonghong Song Cc: Martin KaFai Lau, bpf, Andrii Nakryiko, Toke Høiland-Jørgensen On Fri, Jun 4, 2021 at 12:11 PM Yonghong Song <yhs@fb.com> wrote: > On 6/3/21 11:13 PM, Martin KaFai Lau wrote: > > > > Making the kfunc call whitelist more accessible is useful in general. > > The bpf tcp-cc struct_ops is the only prog type supporting kfunc call. > > What is your use case to introspect this whitelist? > > Agree. It would be good if you can share your use case. At the high level, I am trying to see if we can use bpf in the drm subsystem and gpu drivers which are kernel modules. My initial motivation was to use bpf for dynamic/run-time reconfiguration of the drm/gpu driver (for experimentation.) But now that I learned more about bpf, I think there are quite a few more things I can do with it. (Debugging during GPU hw bring-ups, profiling driver performance in live system, etc.) I have been looking into bpf with kprobe and struct_ops. In terms of kernel module support for bpf/btf, Andrii told me about it last year and I see that his feature is in (I was able to do a btf dump file for /sys/kernel/btf/amdgpu, /sys/kernel/btf/drm_ttm_helper, for example.) The next thing I thought about was having helper functions from kernel modules and Toke pointed me to Martin's patch around "unstable helpers"/calling whitelisted kernel functions and this is where we are at. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-04 20:47 ` Kenny Ho @ 2021-06-08 19:51 ` Kenny Ho 2021-06-09 6:41 ` Yonghong Song 0 siblings, 1 reply; 8+ messages in thread From: Kenny Ho @ 2021-06-08 19:51 UTC (permalink / raw) To: Yonghong Song Cc: Martin KaFai Lau, bpf, Andrii Nakryiko, Toke Høiland-Jørgensen On Fri, Jun 4, 2021 at 4:47 PM Kenny Ho <y2kenny@gmail.com> wrote: > > On Fri, Jun 4, 2021 at 12:11 PM Yonghong Song <yhs@fb.com> wrote: > > On 6/3/21 11:13 PM, Martin KaFai Lau wrote: > > > > > > Making the kfunc call whitelist more accessible is useful in general. > > > The bpf tcp-cc struct_ops is the only prog type supporting kfunc call. > > > What is your use case to introspect this whitelist? > > > > Agree. It would be good if you can share your use case. > > At the high level, I am trying to see if we can use bpf in the drm > subsystem and gpu drivers which are kernel modules. My initial > motivation was to use bpf for dynamic/run-time reconfiguration of the > drm/gpu driver (for experimentation.) But now that I learned more > about bpf, I think there are quite a few more things I can do with it. > (Debugging during GPU hw bring-ups, profiling driver performance in > live system, etc.) I have been looking into bpf with kprobe and > struct_ops. > > In terms of kernel module support for bpf/btf, Andrii told me about it > last year and I see that his feature is in (I was able to do a btf > dump file for /sys/kernel/btf/amdgpu, /sys/kernel/btf/drm_ttm_helper, > for example.) The next thing I thought about was having helper > functions from kernel modules and Toke pointed me to Martin's patch > around "unstable helpers"/calling whitelisted kernel functions and > this is where we are at. Yonghong and Martin, does the use case make sense? Or am I trying to do something stupid? Kenny ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-08 19:51 ` Kenny Ho @ 2021-06-09 6:41 ` Yonghong Song 2021-06-15 19:34 ` Kenny Ho 0 siblings, 1 reply; 8+ messages in thread From: Yonghong Song @ 2021-06-09 6:41 UTC (permalink / raw) To: Kenny Ho Cc: Martin KaFai Lau, bpf, Andrii Nakryiko, Toke Høiland-Jørgensen On 6/8/21 12:51 PM, Kenny Ho wrote: > On Fri, Jun 4, 2021 at 4:47 PM Kenny Ho <y2kenny@gmail.com> wrote: >> >> On Fri, Jun 4, 2021 at 12:11 PM Yonghong Song <yhs@fb.com> wrote: >>> On 6/3/21 11:13 PM, Martin KaFai Lau wrote: >>>> >>>> Making the kfunc call whitelist more accessible is useful in general. >>>> The bpf tcp-cc struct_ops is the only prog type supporting kfunc call. >>>> What is your use case to introspect this whitelist? >>> >>> Agree. It would be good if you can share your use case. >> >> At the high level, I am trying to see if we can use bpf in the drm >> subsystem and gpu drivers which are kernel modules. My initial >> motivation was to use bpf for dynamic/run-time reconfiguration of the >> drm/gpu driver (for experimentation.) But now that I learned more >> about bpf, I think there are quite a few more things I can do with it. >> (Debugging during GPU hw bring-ups, profiling driver performance in >> live system, etc.) I have been looking into bpf with kprobe and >> struct_ops. >> >> In terms of kernel module support for bpf/btf, Andrii told me about it >> last year and I see that his feature is in (I was able to do a btf >> dump file for /sys/kernel/btf/amdgpu, /sys/kernel/btf/drm_ttm_helper, >> for example.) The next thing I thought about was having helper >> functions from kernel modules and Toke pointed me to Martin's patch >> around "unstable helpers"/calling whitelisted kernel functions and >> this is where we are at. > > Yonghong and Martin, does the use case make sense? Or am I trying to > do something stupid? So your intention is to call functions in drivers/gpu/drm/drm_gem_ttm_helper.c, right? How do you get function parameters? What kinds of programs you intend to call this functions? kprobe probably won't work as kernel does not capture traced function types. fentry program might be a good choice. if you can *craft* a bpf program to show how kernel function is used that will be great. We will then see how verifier can help enforce safety. > > Kenny > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-09 6:41 ` Yonghong Song @ 2021-06-15 19:34 ` Kenny Ho 2021-06-16 16:28 ` Yonghong Song 0 siblings, 1 reply; 8+ messages in thread From: Kenny Ho @ 2021-06-15 19:34 UTC (permalink / raw) To: Yonghong Song Cc: Martin KaFai Lau, bpf, Andrii Nakryiko, Toke Høiland-Jørgensen On Wed, Jun 9, 2021 at 2:42 AM Yonghong Song <yhs@fb.com> wrote: > So your intention is to call functions in > drivers/gpu/drm/drm_gem_ttm_helper.c, right? How do you get function > parameters? What kinds of programs you intend to call > this functions? ok... sounds like my use case was not concret enough. Perhaps I can elaborate further with the following examples: In the GPU scheduler, there's a trace point "trace_drm_run_job(sched_job, entity)": https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/scheduler/sched_main.c#L813 If I want to analyze the jobs being scheduled, I can potentially attach a bpf prog with this tracepoint. Each driver has its own run_job and sched_job implementation so I was thinking the drivers can provide a bpf helper function to resolve this. Alternatively, there could be tracepoint in the driver implementation that one can attach bpf to, but tracepoints are not universally put in place (have trace: https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c#L221; not have trace: https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/etnaviv/etnaviv_sched.c#L72 .) So in cases without tracepoint, I guess I would be using kprobe or fentry? Note that all of these are in kernel modules. My understanding is that BTF will work but having helper functions from the kernel modules are not yet available? So let say I want to whitelist and call " amdgpu_device_gpu_recover" (https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c#L4521) from inside a bpf prog, or whitelist and call "drm_sched_increase_karma" https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/scheduler/sched_main.c#L362, I wouldn't be able to do so? Are there any criteria in terms of what kernel function should or should not be whitelisted (separate from the kernel module support question)? For example, would amdgpu_device_gpu_recover be not "whitelist-able" because of the hw interaction or complexity or likelihood to crash the kernel while drm_sched_increase_karma is ok because it's more or less manipulation of some data structure? A quick side question, does container_of work inside a bpf prog? > kprobe probably won't work as kernel does not capture > traced function types. fentry program might be a good > choice. Thanks for the pointer. I am not familiar with fentry but I will look into it. By function type, what do you mean by that? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Headers for whitelisted kernel functions available to BPF programs 2021-06-15 19:34 ` Kenny Ho @ 2021-06-16 16:28 ` Yonghong Song 0 siblings, 0 replies; 8+ messages in thread From: Yonghong Song @ 2021-06-16 16:28 UTC (permalink / raw) To: Kenny Ho Cc: Martin KaFai Lau, bpf, Andrii Nakryiko, Toke Høiland-Jørgensen On 6/15/21 12:34 PM, Kenny Ho wrote: > On Wed, Jun 9, 2021 at 2:42 AM Yonghong Song <yhs@fb.com> wrote: >> So your intention is to call functions in >> drivers/gpu/drm/drm_gem_ttm_helper.c, right? How do you get function >> parameters? What kinds of programs you intend to call >> this functions? > ok... sounds like my use case was not concret enough. Perhaps I can > elaborate further with the following examples: > > In the GPU scheduler, there's a trace point > "trace_drm_run_job(sched_job, entity)": > > https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/scheduler/sched_main.c#L813 > > If I want to analyze the jobs being scheduled, I can potentially > attach a bpf prog with this tracepoint. Each driver has its own > run_job and sched_job implementation so I was thinking the drivers can > provide a bpf helper function to resolve this. Alternatively, there > could be tracepoint in the driver implementation that one can attach > bpf to, but tracepoints are not universally put in place (have trace: > https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c#L221 ; > not have trace: > https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/etnaviv/etnaviv_sched.c#L72 > .) So in cases without tracepoint, I guess I would be using kprobe or > fentry? Yes, either kprobe or fentry should work. > > Note that all of these are in kernel modules. My understanding is > that BTF will work but having helper functions from the kernel modules > are not yet available? So let say I want to whitelist and call " We could you have some kernel helpers interacts with kernel subsystems. For example, ipv6 could be a module, some kernel helpers actually works fine with ipv6 module, you need have some configure time checking as well as some runtime checking. > amdgpu_device_gpu_recover" > (https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c#L4521 ) > from inside a bpf prog, or whitelist and call > "drm_sched_increase_karma" > https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/scheduler/sched_main.c#L362 , > I wouldn't be able to do so? Are there any criteria in terms of what > kernel function should or should not be whitelisted (separate from the > kernel module support question)? For example, would > amdgpu_device_gpu_recover be not "whitelist-able" because of the hw > interaction or complexity or likelihood to crash the kernel while > drm_sched_increase_karma is ok because it's more or less manipulation > of some data structure? To determine whether a function can be used in what context indeed a very tricky question. The function amdgpu_device_gpu_recover is very complex and has a lot of side effect. Maybe it can still be used, but need to tag the function with some conditions and unless these conditions are verified by verifier, the function cannot be used in bpf program. > A quick side question, does container_of work inside a bpf prog? Yes, the macro is defined in bpf_helpers.h. > >> kprobe probably won't work as kernel does not capture >> traced function types. fentry program might be a good >> choice. > Thanks for the pointer. I am not familiar with fentry but I will look > into it. By function type, what do you mean by that? Sorry, probably typing too fast. "function type" here I actually mean "btf type". To call kernel function inside the bpf program, we need btf type to verify kernel function signature, precondition etc. kprobe uses pt_regs and all types are lost. > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-06-16 16:28 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-03 18:01 Headers for whitelisted kernel functions available to BPF programs Kenny Ho 2021-06-04 6:13 ` Martin KaFai Lau 2021-06-04 16:10 ` Yonghong Song 2021-06-04 20:47 ` Kenny Ho 2021-06-08 19:51 ` Kenny Ho 2021-06-09 6:41 ` Yonghong Song 2021-06-15 19:34 ` Kenny Ho 2021-06-16 16:28 ` Yonghong Song
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).