From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: "Theodore Ts'o" <tytso@mit.edu>,
Dmitry Vyukov <dvyukov@google.com>,
syzbot <syzbot+4bfbbf28a2e50ab07368@syzkaller.appspotmail.com>,
Andreas Dilger <adilger.kernel@dilger.ca>,
David Miller <davem@davemloft.net>,
eladr@mellanox.com, Ido Schimmel <idosch@mellanox.com>,
Jiri Pirko <jiri@mellanox.com>,
John Stultz <john.stultz@linaro.org>,
linux-ext4@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
netdev <netdev@vger.kernel.org>,
syzkaller-bugs <syzkaller-bugs@googlegroups.com>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>
Subject: Re: INFO: rcu detected stall in ext4_write_checks
Date: Sun, 14 Jul 2019 12:29:51 -0700 [thread overview]
Message-ID: <20190714192951.GM26519@linux.ibm.com> (raw)
In-Reply-To: <20190714190522.GA24049@mit.edu>
On Sun, Jul 14, 2019 at 03:05:22PM -0400, Theodore Ts'o wrote:
> On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote:
> > But short term I don't see any other solution than stop testing
> > sched_setattr because it does not check arguments enough to prevent
> > system misbehavior. Which is a pity because syzkaller has found some
> > bad misconfigurations that were oversight on checking side.
> > Any other suggestions?
>
> Or maybe syzkaller can put its own limitations on what parameters are
> sent to sched_setattr? In practice, there are any number of ways a
> root user can shoot themselves in the foot when using sched_setattr or
> sched_setaffinity, for that matter. I imagine there must be some such
> constraints already --- or else syzkaller might have set a kernel
> thread to run with priority SCHED_BATCH, with similar catastrophic
> effects --- or do similar configurations to make system threads
> completely unschedulable.
>
> Real time administrators who know what they are doing --- and who know
> that their real-time threads are well behaved --- will always want to
> be able to do things that will be catastrophic if the real-time thread
> is *not* well behaved. I don't it is possible to add safety checks
> which would allow the kernel to automatically detect and reject unsafe
> configurations.
>
> An apt analogy might be civilian versus military aircraft. Most
> airplanes are designed to be "inherently stable"; that way, modulo
> buggy/insane control systems like on the 737 Max, the airplane will
> automatically return to straight and level flight. On the other hand,
> some military planes (for example, the F-16, F-22, F-36, the
> Eurofighter, etc.) are sometimes designed to be unstable, since that
> way they can be more maneuverable.
>
> There are use cases for real-time Linux where this flexibility/power
> vs. stability tradeoff is going to argue for giving root the
> flexibility to crash the system. Some of these systems might
> literally involve using real-time Linux in military applications,
> something for which Paul and I have had some experience. :-)
>
> Speaking of sched_setaffinity, one thing which we can do is have
> syzkaller move all of the system threads to they run on the "system
> CPU's", and then move the syzkaller processes which are testing the
> kernel to be on the "system under test CPU's". Then regardless of
> what priority the syzkaller test programs try to run themselves at,
> they can't crash the system.
>
> Some real-time systems do actually run this way, and it's a
> recommended configuration which is much safer than letting the
> real-time threads take over the whole system:
>
> http://linuxrealtime.org/index.php/Improving_the_Real-Time_Properties#Isolating_the_Application
Good point! We might still have issues with some per-CPU kthreads,
but perhaps use of nohz_full would help at least reduce these sorts
of problems. (There could still be issues on CPUs with more than
one runnable threads.)
Thanx, Paul
next prev parent reply other threads:[~2019-07-14 19:30 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-26 17:27 INFO: rcu detected stall in ext4_write_checks syzbot
2019-06-26 18:42 ` Theodore Ts'o
2019-06-26 21:03 ` Theodore Ts'o
2019-06-26 22:47 ` Theodore Ts'o
2019-07-05 13:24 ` Dmitry Vyukov
2019-07-05 15:16 ` Paul E. McKenney
2019-07-05 15:47 ` Amir Goldstein
2019-07-05 15:48 ` Dmitry Vyukov
2019-07-05 19:10 ` Paul E. McKenney
2019-07-06 4:28 ` Theodore Ts'o
2019-07-06 6:16 ` Paul E. McKenney
2019-07-06 15:02 ` Theodore Ts'o
2019-07-06 18:03 ` Paul E. McKenney
2019-07-07 1:16 ` Paul E. McKenney
2019-07-14 14:48 ` Dmitry Vyukov
2019-07-14 18:49 ` Paul E. McKenney
2019-07-15 13:29 ` Peter Zijlstra
2019-07-15 13:33 ` Dmitry Vyukov
2019-07-15 13:46 ` Peter Zijlstra
2019-07-15 14:02 ` Paul E. McKenney
2019-07-22 10:03 ` Dmitry Vyukov
2019-07-23 8:51 ` Dmitry Vyukov
2019-07-14 19:05 ` Theodore Ts'o
2019-07-14 19:29 ` Paul E. McKenney [this message]
2019-07-15 3:10 ` Paul E. McKenney
2019-07-15 13:01 ` Paul E. McKenney
2019-07-15 13:29 ` Dmitry Vyukov
2019-07-15 13:39 ` Peter Zijlstra
2019-07-15 14:03 ` Paul E. McKenney
2019-07-15 13:22 ` Peter Zijlstra
2019-07-05 13:18 ` Dmitry Vyukov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190714192951.GM26519@linux.ibm.com \
--to=paulmck@linux.ibm.com \
--cc=adilger.kernel@dilger.ca \
--cc=davem@davemloft.net \
--cc=dvyukov@google.com \
--cc=eladr@mellanox.com \
--cc=idosch@mellanox.com \
--cc=jiri@mellanox.com \
--cc=john.stultz@linaro.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=syzbot+4bfbbf28a2e50ab07368@syzkaller.appspotmail.com \
--cc=syzkaller-bugs@googlegroups.com \
--cc=tglx@linutronix.de \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).