Re: [PATCH 5/7] x86/fpu: Change fpu->fpregs_active users to fpu->fpstate_active

From: Rik van Riel <riel@redhat.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: linux-kernel@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Andy Lutomirski <luto@amacapital.net>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Fenghua Yu <fenghua.yu@intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Oleg Nesterov <oleg@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Yu-cheng Yu <yu-cheng.yu@intel.com>
Subject: Re: [PATCH 5/7] x86/fpu: Change fpu->fpregs_active users to fpu->fpstate_active
Date: Thu, 26 Jan 2017 10:45:49 -0500	[thread overview]
Message-ID: <1485445549.15964.53.camel@redhat.com> (raw)
In-Reply-To: <20170126151642.GB12274@gmail.com>

On Thu, 2017-01-26 at 16:16 +0100, Ingo Molnar wrote:
> * Rik van Riel <riel@redhat.com> wrote:
> 
> > On Thu, 2017-01-26 at 12:26 +0100, Ingo Molnar wrote:
> > > We want to simplify the FPU state machine by eliminating fpu-
> > > > fpregs_active,
> > > 
> > > and we can do that because the two state flags (::fpregs_active
> > > and
> > > ::fpstate_active) are set essentially together.
> > > 
> > > The old lazy FPU switching code used to make a distinction - but
> > > there's
> > > no lazy switching code anymore, we always switch in an 'eager'
> > > fashion.
> > 
> > I've been working for a while now to fix that for
> > KVM VCPU threads.
> > 
> > Currently when we switch to a VCPU thread, we first
> > load that thread's userspace FPU context, and then
> > soon after we save that, and load the guest side FPU
> > context.
> > 
> > When a VCPU thread goes idle, we also go through
> > two FPU context transitions.
> > 
> > In order to skip the unnecessary FPU context switches
> > for VCPU threads, I have been relying on separate
> > fpstate_active and fpregs_active states.
> > 
> > Do you have any ideas on how I could implement that
> > kind of change without separate fpstate_active and
> > fpregs_active states?
> 
> So the vCPU threads have host side FPU (user-space) state - whatever
> FPU state 
> Qemu has?

Indeed.

> I.e. the vCPU /dev/kvm ioctl() could drop/re-map the FPU state with
> very little 
> overhead (i.e. no full save/restore required in that code path
> either), when it 
> enters/exits vCPU mode.

Remapping might be best.  If we remap, we do not need to call
kernel_fpu_begin/end around actually going into the guest, and
we can hang onto the guest FPU context while doing stuff inside
the host kernel, even while going to sleep in the host kernel.

Let me go totally reimplement this whole project in a different
way...

At least I found some good FPU bugs and cleanups along the way.