Re: [PATCH bpf-next v1 1/9] bpf: Add mkdir, rmdir, unlink syscalls for prog_bpf_syscall

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Hao Luo <haoluo@google.com>
Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>, KP Singh <kpsingh@kernel.org>,
	Shakeel Butt <shakeelb@google.com>,
	Joe Burton <jevburton.kernel@gmail.com>,
	Tejun Heo <tj@kernel.org>, Josh Don <joshdon@google.com>,
	Stanislav Fomichev <sdf@google.com>, bpf <bpf@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH bpf-next v1 1/9] bpf: Add mkdir, rmdir, unlink syscalls for prog_bpf_syscall
Date: Sat, 5 Mar 2022 15:47:17 -0800	[thread overview]
Message-ID: <CAADnVQ+-9DAuqj3jLvnwPn0PwuRnfSZ4niDOPqOaF+SH-_+P8A@mail.gmail.com> (raw)
In-Reply-To: <CA+khW7hK9JKU3be7gDDJ9DsOeaUS3RxCGJOJAUrZwvyVJiSSSA@mail.gmail.com>

On Fri, Mar 4, 2022 at 10:37 AM Hao Luo <haoluo@google.com> wrote:
>
> I gave this question more thought. We don't need to bind mount the top
> bpffs into the container, instead, we may be able to overlay a bpffs
> directory into the container. Here is the workflow in my mind:

I don't quite follow what you mean by 'overlay' here.
Another bpffs mount or future overlayfs that supports bpffs?

> For each job, let's say A, the container runtime can create a
> directory in bpffs, for example
>
>   /sys/fs/bpf/jobs/A
>
> and then create the cgroup for A. The sleepable tracing prog will
> create the file:
>
>   /sys/fs/bpf/jobs/A/100/stats
>
> 100 is the created cgroup's id. Then the container runtime overlays
> the bpffs directory into container A in the same path:

Why cgroup id ? Wouldn't it be easier to use the same cgroup name
as in cgroupfs ?

>   [A's container path]/sys/fs/bpf/jobs/A.
>
> A can see the stats at the path within its mount ns:
>
>   /sys/fs/bpf/jobs/A/100/stats
>
> When A creates cgroup, it is able to write to the top layer of the
> overlayed directory. So it is
>
>   /sys/fs/bpf/jobs/A/101/stats
>
> Some of my thoughts:
>   1. Compared to bind mount top bpffs into container, overlaying a
> directory avoids exposing other jobs' stats. This gives better
> isolation. I already have a patch for supporting laying bpffs over
> other fs, it's not too hard.

So it's overlayfs combination of bpffs and something like ext4, right?
I thought you found out that overlaryfs has to be upper fs
and lower fs shouldn't be modified underneath.
So if bpffs is a lower fs the writes into it should go
through the upper overlayfs, right?

>   2. Once the container runtime has overlayed directory into the
> container, it has no need to create more cgroups for this job. It
> doesn't need to track the stats of job-created cgroups, which are
> mainly for inspection by the job itself. Even if it needs to collect
> the stats from those cgroups, it can read from the path in the
> container.
>   3. The overlay path in container doesn't have to be exactly the same
> as the path in root mount ns. In the sleepable tracing prog, we may
> select paths based on current process's ns. If we choose to do this,
> we can further avoid exposing cgroup id and job name to the container.

The benefits make sense.