linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Christoph Bartoschek <bartoschek@google.com>
To: Chris Mason <clm@fb.com>, "Paul E. McKenney" <paulmck@kernel.org>,
	Giuseppe Scrivano <gscrivan@redhat.com>
Cc: linux-kernel@vger.kernel.org,
	"riel@surriel.com" <riel@surriel.com>,
	"viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>,
	Christoph Bartoschek <bartoschek@google.com>
Subject: Re: [PATCH RFC fs/namespace] Make kern_unmount() use synchronize_rcu_expedited()
Date: Tue, 26 Apr 2022 08:59:17 +0200	[thread overview]
Message-ID: <20220426065917.3123488-1-bartoschek@google.com> (raw)
In-Reply-To: <8E281DF1-248F-4861-A3C0-2573A5EFEE61@fb.com>

The regression that has been introduced with commit
e1eb26fa62d04ec0955432be1aa8722a97cb52e7 has hit us when building with Bazel
using the linux-sandbox
(https://github.com/bazelbuild/bazel/blob/master/src/main/tools/linux-sandbox.cc).
The sandbox tries to isolate build steps from each other and to ensure that
builds are hermetic and therefore sets up new namespaces for each step. For
large software packages and even with the time spend building we run out of
namespaces on larger machines that allow for enough parallelism. I have reduced
the sandbox to a simple test case:

#define _GNU_SOURCE
#include <errno.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>

int pid1main(void *) {
   return 0;
}

int main(void) {
  int clone_flags = CLONE_NEWUSER | CLONE_NEWIPC | SIGCHLD;
  void * stack = malloc(1024*1024);
  const pid_t child_pid = clone(pid1main, stack + 1024*1024, clone_flags, NULL);

  if (child_pid < 0) {
    perror("clone");
  }
  int ret = waitpid(child_pid, NULL, 0);
  if (ret < 0) {
    perror("waitpid");
    return ret;
  }
  return 0;
}

Run it with
$ gcc clone-test.cc
$ seq 1 10000000 | parallel --halt now,fail=1 -j32 $PWD/a.out
clone: No space left on device
waitpid: No child processes
parallel: This job failed:
/usr/local/google/home/bartoschek/linux-sandbox-test/a.out 53070

I run the test on kernel v5.18-rc4.
Depending on your configured limits you will soon get an ENOSPC even though
never more than 32 additional namespaces should be in use by parallel.
During execution the whole system can become quite unresponsive.
This does not happen without e1eb26fa62d04ec0955432be1aa8722a97cb52e7.

I see that the issue was already reported in 2020:
http://merlin.infradead.org/pipermail/linux-nvme/2020-September/019565.html

Would it be possible to revert e1eb26fa62d04ec0955432be1aa8722a97cb52e7? It
seems to make the kernel less deterministic and hard to reason about active
namespaces.

Christoph

  reply	other threads:[~2022-04-26  7:00 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-14 19:05 [PATCH RFC fs/namespace] Make kern_unmount() use synchronize_rcu_expedited() Paul E. McKenney
2022-02-14 19:26 ` Chris Mason
2022-02-14 19:44   ` Paul E. McKenney
2022-02-14 20:55     ` Rik van Riel
2022-02-14 21:41       ` Paul E. McKenney
2022-02-15 18:28   ` Chris Mason
2022-04-26  6:59     ` Christoph Bartoschek [this message]
2022-04-26 14:09       ` Paul E. McKenney
2022-04-26 22:58         ` Christoph Bartoschek
2022-04-26 23:11           ` Paul E. McKenney
2022-04-27 12:19             ` Chris Mason
2022-04-27 15:01               ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220426065917.3123488-1-bartoschek@google.com \
    --to=bartoschek@google.com \
    --cc=clm@fb.com \
    --cc=gscrivan@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=paulmck@kernel.org \
    --cc=riel@surriel.com \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).