Re: [RFC 3/3] Revert "rcutorture: Tweak kvm options"

From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: Joel Fernandes <joel@joelfernandes.org>
Cc: rcu@vger.kernel.org
Subject: Re: [RFC 3/3] Revert "rcutorture: Tweak kvm options"
Date: Mon, 1 Jul 2019 14:50:20 -0700	[thread overview]
Message-ID: <20190701215020.GW26519@linux.ibm.com> (raw)
In-Reply-To: <20190701213107.GA240327@google.com>

On Mon, Jul 01, 2019 at 05:31:07PM -0400, Joel Fernandes wrote:
> On Mon, Jul 01, 2019 at 01:27:27PM -0700, Paul E. McKenney wrote:
> > On Mon, Jul 01, 2019 at 10:59:20AM -0400, Joel Fernandes wrote:
> > > On Mon, Jul 01, 2019 at 04:48:43PM +0200, Dmitry Vyukov wrote:
> > > > On Mon, Jul 1, 2019 at 4:14 PM Joel Fernandes <joel@joelfernandes.org> wrote:
> > > > >
> > > > > On Mon, Jul 01, 2019 at 02:23:58PM +0200, Sebastian Andrzej Siewior wrote:
> > > > > > On 2019-07-01 00:04:15 [-0400], Joel Fernandes (Google) wrote:
> > > > > > > This reverts commit a6fda6dab93c2c06ef4b8cb4b9258df6674d2438 which
> > > > > > > causes kvm.sh to not run on my machines. The qemu-system-x86_64 command
> > > > > > > runs but does nothing.
> > > > > >
> > > > > > Nope. I would like to know *why* you need 'noapic' to work. Is it a
> > > > > > brand new or old qemu-system-x86_64?
> > > > >
> > > > > I did not have time to debug yesterday and I posted this particular revert as
> > > > > an 'RFC' just to make aware of this problem.
> > > > >
> > > > > I spent some more time just now, it looks like this has nothing to do with
> > > > > 'noapic' and appears to be a problem on debian distros with the e1000e NIC.
> > > > > May be this NIC was added to the virtual hardware because of -machine in the
> > > > > patch?
> > > > >
> > > > > Any if I add the following to the qemu command that kvm.sh runs, it works again:
> > > > > -net nic,model=e1000
> > > > >
> > > > > Without it I get:
> > > > >  qemu-system-x86_64: Initialization of device e1000e failed: failed to find romfile "efi-e1000e.rom"
> > > > >
> > > > > Seems to be mentioned here:
> > > > > https://bugs.launchpad.net/ubuntu/+source/ipxe/+bug/1737211
> > > > >
> > > > > And in syzkaller as well:
> > > > > https://github.com/google/syzkaller/blob/master/vm/qemu/qemu.go#L88
> > > > >
> > > > > Adding Dmitry who is syzkaller owner for any thoughts as well.
> > > > 
> > > > I don't have many thoughts on this. That particular error looked like
> > > > a bug in the package in the particular distro/version.
> > > 
> > > Paul, what is your preference here? Can we add the -net nic,model=e1000 to
> > > fix it for the benefit of any other Debian folks running kvm.sh?
> > > 
> > > Or do you prefer if I just built my own custom Qemu? I can't upgrade Qemu on
> > > this machine unfortunately. But may be I can build my own.
> > > 
> > > I prefer the -net option since I can save the time for something else. ;) Let
> > > me know what you prefer and I'll fix it accordingly.
> > 
> > Why not just add the following to your kvm.sh command line?
> > 
> > 	--qemu-args "-net nic,model=e1000"
> > 
> > Easy workaround and no need to wait for changes to hit mainline.
> > 
> > Yes, I am perhaps naively assuming that the qemu bug in Debian will
> > be fixed in about the time that it would take for any change to the
> > rcutorture scripting to make its way into mainline.  ;-)
> 
> Thanks for the pointer to the args! Just what I wanted.
> 
> Only down side is then we run into the risk of other developers running
> kvm.sh on debian as well and wondering why torture is broken. Perhaps the kvm
> tweaks commit hasn't hit that many machines yet but it might. At least we
> have this discussion archive here to guide such poor souls, if we don't want
> to append these arguments by default ;-)

What is the likely timeframe of a qemu fix in Debian?

This does raise a question of rcutorture workarounds in general.
There will be times when some failure or another is expected behavior.
For example, in current -rcu getting an rcutorture_oom_notify() splat or
the occasional RCU CPU stall warning from TREE04 is expected behavior.
This is because rcutorture's forward-progress testing is more aggressive
that the TREE04 configuration can currently handle (but I am working
on it).

For internal-to-RCU issues, I could make rcutorture warn of the likely
problem.  for external-to-RCU issues, such as your fun with Debian qemu,
that won't work all that well.  I could blog about the external-to-RCU
issues, if that would help.  Or we could trust people to either do
the websearch or ask for help on the rcu email list.

Thoughts?

> The other fix, could be I just build and run a new custom qemu instance but I
> am happy with qemu-args as well.

My concern is that applying the workaround to the rcutorture scripts might
break someone else.  Plus it is hard to synchronize with qemu releases,
to say nothing of distro releases containing qemu.  Hence my preference
for use of kvm.sh's --qemu-args for this purpose.

							Thanx, Paul