What happens to a uprobe if it links to a library within a container, and that container gets deleted?

What happens to a uprobe if it links to a library within a container, and that container gets deleted?

[Yadunandan Pillai]

How are uprobes "remembered" in the kernel from a conceptual standpoint? Where is the attach point stored? Is it basically a hashmap with JMP instructions for each function that is being attached to? What exactly does the cleanup process look like if the attach point disappears?

Example of a use case: let's say a uprobe is to "SSL_read" in /proc/[root_pid]/root/.../libssl.so where [root_pid] is the root process of a container. If the container dies, then does that uprobe hang around attaching to empty air or gets deleted as well?

Yadunandan Pillai

Re: What happens to a uprobe if it links to a library within a container, and that container gets deleted?

[Andrii Nakryiko]

On Thu, Jul 7, 2022 at 12:05 PM Yadunandan Pillai <thesw4rm@pm.me> wrote:
>
> How are uprobes "remembered" in the kernel from a conceptual standpoint? Where is the attach point stored? Is it basically a hashmap with JMP instructions for each function that is being attached to? What exactly does the cleanup process look like if the attach point disappears?
>
> Example of a use case: let's say a uprobe is to "SSL_read" in /proc/[root_pid]/root/.../libssl.so where [root_pid] is the root process of a container. If the container dies, then does that uprobe hang around attaching to empty air or gets deleted as well?

In BPF world, uprobe is a combination of two objects, each having
their FD and associated lifetimes:
  - perf_event_open() syscall creates perf_event kernel object that
represents uprobe itself (you specify target binary, which kernel
resolves into inode internally; optionally you also specify PID
filter, so uprobe can be triggered only for specific process or for
all processes that run code from specified binary);
  - uprobe BPF program attached to perf_event object; this attachment
(link) also has associated FD (for older kernels you'll have only
perf_event FD, though);

As long as at least one of those FDs are not closed, your uprobe+BPF
program will be active. They might not be triggered ever because file
was deleted from file system (I think file's inode will be kept around
until perf_event is destroyed, but I haven't checked the code).

So direct answer to your last question depends on what happens with
perf_event that was created during attachment. If its FD survives the
container (because you transferred FD, or the process is outside of
container, or you pinned BPF link representing that attachment), then
no, uprobe is still there. But if the process that attached BPF
program exits and nothing else keeps FD alive, then BPF program and
perf_event will be detached and destroyed.

>
> Yadunandan Pillai

Re: What happens to a uprobe if it links to a library within a container, and that container gets deleted?

[Yadunandan Pillai]

> uprobe BPF program attached to perf_event object; this attachment
(link) also has associated FD (for older kernels you'll have only
perf_event FD, though);

Will there be two FDs in the newer kernel, in that case? One for the perf_event object itself and one for the link between the uprobe to the perf event object.

And how exactly does the uprobe attach to a specific symbol (like a function) within a shared library? Does it basically hook itself into a pre-calculated offset? What happens if the code at that offset is edited while the uprobe is attached?









Yadunandan Pillai


------- Original Message -------
On Monday, July 11th, 2022 at 11:27 PM, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:


> On Thu, Jul 7, 2022 at 12:05 PM
>
> Yadunandan Pillai
>
> thesw4rm@pm.me wrote:
>
> > How are uprobes "remembered" in the kernel from a conceptual standpoint? Where is the attach point stored? Is it basically a hashmap with JMP instructions for each function that is being attached to? What exactly does the cleanup process look like if the attach point disappears?
> >
> > Example of a use case: let's say a uprobe is to "SSL_read" in /proc/[root_pid]/root/.../libssl.so where [root_pid] is the root process of a container. If the container dies, then does that uprobe hang around attaching to empty air or gets deleted as well?
>
>
> In BPF world, uprobe is a combination of two objects, each having
> their FD and associated lifetimes:
> - perf_event_open() syscall creates perf_event kernel object that
> represents uprobe itself (you specify target binary, which kernel
> resolves into inode internally; optionally you also specify PID
> filter, so uprobe can be triggered only for specific process or for
> all processes that run code from specified binary);
> - uprobe BPF program attached to perf_event object; this attachment
> (link) also has associated FD (for older kernels you'll have only
> perf_event FD, though);
>
> As long as at least one of those FDs are not closed, your uprobe+BPF
> program will be active. They might not be triggered ever because file
> was deleted from file system (I think file's inode will be kept around
> until perf_event is destroyed, but I haven't checked the code).
>
> So direct answer to your last question depends on what happens with
> perf_event that was created during attachment. If its FD survives the
> container (because you transferred FD, or the process is outside of
> container, or you pinned BPF link representing that attachment), then
> no, uprobe is still there. But if the process that attached BPF
> program exits and nothing else keeps FD alive, then BPF program and
> perf_event will be detached and destroyed.
>
> > Yadunandan Pillai

Re: What happens to a uprobe if it links to a library within a container, and that container gets deleted?

[Andrii Nakryiko]

On Mon, Jul 18, 2022 at 2:00 PM Yadunandan Pillai <thesw4rm@pm.me> wrote:
>
> > uprobe BPF program attached to perf_event object; this attachment
> (link) also has associated FD (for older kernels you'll have only
> perf_event FD, though);
>
> Will there be two FDs in the newer kernel, in that case? One for the perf_event object itself and one for the link between the uprobe to the perf event object.

yes

>
> And how exactly does the uprobe attach to a specific symbol (like a function) within a shared library? Does it basically hook itself into a pre-calculated offset? What happens if the code at that offset is edited while the uprobe is attached?

yes (about offset), I don't know about the second one, but all the
source code is openly available (plus you can always experiment)


>
>
>
>
>
>
>
>
>
> Yadunandan Pillai
>
>
> ------- Original Message -------
> On Monday, July 11th, 2022 at 11:27 PM, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
>
>
> > On Thu, Jul 7, 2022 at 12:05 PM
> >
> > Yadunandan Pillai
> >
> > thesw4rm@pm.me wrote:
> >
> > > How are uprobes "remembered" in the kernel from a conceptual standpoint? Where is the attach point stored? Is it basically a hashmap with JMP instructions for each function that is being attached to? What exactly does the cleanup process look like if the attach point disappears?
> > >
> > > Example of a use case: let's say a uprobe is to "SSL_read" in /proc/[root_pid]/root/.../libssl.so where [root_pid] is the root process of a container. If the container dies, then does that uprobe hang around attaching to empty air or gets deleted as well?
> >
> >
> > In BPF world, uprobe is a combination of two objects, each having
> > their FD and associated lifetimes:
> > - perf_event_open() syscall creates perf_event kernel object that
> > represents uprobe itself (you specify target binary, which kernel
> > resolves into inode internally; optionally you also specify PID
> > filter, so uprobe can be triggered only for specific process or for
> > all processes that run code from specified binary);
> > - uprobe BPF program attached to perf_event object; this attachment
> > (link) also has associated FD (for older kernels you'll have only
> > perf_event FD, though);
> >
> > As long as at least one of those FDs are not closed, your uprobe+BPF
> > program will be active. They might not be triggered ever because file
> > was deleted from file system (I think file's inode will be kept around
> > until perf_event is destroyed, but I haven't checked the code).
> >
> > So direct answer to your last question depends on what happens with
> > perf_event that was created during attachment. If its FD survives the
> > container (because you transferred FD, or the process is outside of
> > container, or you pinned BPF link representing that attachment), then
> > no, uprobe is still there. But if the process that attached BPF
> > program exits and nothing else keeps FD alive, then BPF program and
> > perf_event will be detached and destroyed.
> >
> > > Yadunandan Pillai