From: Andy Lutomirski <luto@amacapital.net>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Vivek Goyal <vgoyal@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Andy Lutomirski <luto@kernel.org>,
Paolo Bonzini <pbonzini@redhat.com>,
LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
kvm list <kvm@vger.kernel.org>, stable <stable@vger.kernel.org>
Subject: Re: [PATCH v2] x86/kvm: Disable KVM_ASYNC_PF_SEND_ALWAYS
Date: Tue, 7 Apr 2020 14:41:02 -0700 [thread overview]
Message-ID: <B85606B0-71B5-4B7D-A892-293CB9C1B434@amacapital.net> (raw)
In-Reply-To: <87eeszjbe6.fsf@nanos.tec.linutronix.de>
> On Apr 7, 2020, at 1:20 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Andy Lutomirski <luto@amacapital.net> writes:
>>>> On Apr 7, 2020, at 10:21 AM, Vivek Goyal <vgoyal@redhat.com> wrote:
>>> Whether interrupts are enabled or not check only happens before we decide
>>> if async pf protocol should be followed or not. Once we decide to
>>> send PAGE_NOT_PRESENT, later notification PAGE_READY does not check
>>> if interrupts are enabled or not. And it kind of makes sense otherwise
>>> guest process will wait infinitely to receive PAGE_READY.
>>>
>>> I modified the code a bit to disable interrupt and wait 10 seconds (after
>>> getting PAGE_NOT_PRESENT message). And I noticed that error async pf
>>> got delivered after 10 seconds after enabling interrupts. So error
>>> async pf was not lost because interrupts were disabled.
>
> Async PF is not the same as a real #PF. It just hijacked the #PF vector
> because someone thought this is a brilliant idea.
>
>>> Havind said that, I thought disabling interrupts does not mask exceptions.
>>> So page fault exception should have been delivered even with interrupts
>>> disabled. Is that correct? May be there was no vm exit/entry during
>>> those 10 seconds and that's why.
>
> No. Async PF is not a real exception. It has interrupt semantics and it
> can only be injected when the guest has interrupts enabled. It's bad
> design.
>
>> My point is that the entire async pf is nonsense. There are two types of events right now:
>>
>> “Page not ready”: normally this isn’t even visible to the guest — the
>> guest just waits. With async pf, the idea is to try to tell the guest
>> that a particular instruction would block and the guest should do
>> something else instead. Sending a normal exception is a poor design,
>> though: the guest may not expect this instruction to cause an
>> exception. I think KVM should try to deliver an *interrupt* and, if it
>> can’t, then just block the guest.
>
> That's pretty much what it does, just that it runs this through #PF and
> has the checks for interrupts disabled - i.e can't right now' around
> that. If it can't then KVM schedules the guest out until the situation
> has been resolved.
>
>> “Page ready”: this is a regular asynchronous notification just like,
>> say, a virtio completion. It should be an ordinary interrupt. Some in
>> memory data structure should indicate which pages are ready.
>>
>> “Page is malfunctioning” is tricky because you *must* deliver the
>> event. x86’s #MC is not exactly a masterpiece, but it does kind of
>> work.
>
> Nooooo. This does not need #MC at all. Don't even think about it.
Yessssssssssss. Please do think about it. :)
>
> The point is that the access to such a page is either happening in user
> space or in kernel space with a proper exception table fixup.
>
> That means a real #PF is perfectly fine. That can be injected any time
> and does not have the interrupt semantics of async PF.
The hypervisor has no way to distinguish between MOV-and-has-valid-stack-and-extable-entry and MOV-definitely-can’t-fault-here. Or, for that matter, MOV-in-do_page_fault()-will-recurve-if-it-faults.
>
> So now lets assume we distangled async PF from #PF and made it a regular
> interrupt, then the following situation still needs to be dealt with:
>
> guest -> access faults
>
> host -> injects async fault
>
> guest -> handles and blocks the task
>
> host figures out that the page does not exist anymore and now needs to
> fixup the situation.
>
> host -> injects async wakeup
>
> guest -> returns from aysnc PF interrupt and retries the instruction
> which faults again.
>
> host -> knows by now that this is a real fault and injects a proper #PF
>
> guest -> #PF runs and either sends signal to user space or runs
> the exception table fixup for a kernel fault.
Or guest blows up because the fault could not be recovered using #PF.
I can see two somewhat sane ways to make this work.
1. Access to bad memory results in an async-page-not-present, except that, it’s not deliverable, the guest is killed. Either that async-page-not-present has a special flag saying “memory failure” or the eventual wakeup says “memory failure”.
2. Access to bad memory results in #MC. Sure, #MC is a turd, but it’s an *architectural* turd. By all means, have a nice simple PV mechanism to tell the #MC code exactly what went wrong, but keep the overall flow the same as in the native case.
I think I like #2 much better. It has another nice effect: a good implementation will serve as a way to exercise the #MC code without needing to muck with EINJ or with whatever magic Tony uses. The average kernel developer does not have access to a box with testable memory failure reporting.
>
> Thanks,
>
> tglx
>
>
>
>
next prev parent reply other threads:[~2020-04-07 21:41 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-07 2:26 [PATCH v2] x86/kvm: Disable KVM_ASYNC_PF_SEND_ALWAYS Andy Lutomirski
2020-03-07 15:03 ` Andy Lutomirski
2020-03-07 15:47 ` Thomas Gleixner
2020-03-07 15:59 ` Andy Lutomirski
2020-03-07 19:01 ` Thomas Gleixner
2020-03-07 19:34 ` Andy Lutomirski
2020-03-08 7:23 ` Thomas Gleixner
2020-03-09 6:57 ` Thomas Gleixner
2020-03-09 8:40 ` Paolo Bonzini
2020-03-09 9:09 ` Thomas Gleixner
2020-03-09 18:14 ` Andy Lutomirski
2020-03-09 19:05 ` Thomas Gleixner
2020-03-09 20:22 ` Peter Zijlstra
2020-04-06 19:09 ` Vivek Goyal
2020-04-06 20:25 ` Peter Zijlstra
2020-04-06 20:32 ` Andy Lutomirski
2020-04-06 20:42 ` Andy Lutomirski
2020-04-07 17:21 ` Vivek Goyal
2020-04-07 17:38 ` Andy Lutomirski
2020-04-07 20:20 ` Thomas Gleixner
2020-04-07 21:41 ` Andy Lutomirski [this message]
2020-04-07 22:07 ` Paolo Bonzini
2020-04-07 22:29 ` Andy Lutomirski
2020-04-08 0:30 ` Paolo Bonzini
2020-05-21 15:55 ` Vivek Goyal
2020-04-07 22:48 ` Thomas Gleixner
2020-04-08 4:48 ` Andy Lutomirski
2020-04-08 9:32 ` Borislav Petkov
2020-04-08 10:12 ` Thomas Gleixner
2020-04-08 18:23 ` Vivek Goyal
2020-04-07 22:49 ` Vivek Goyal
2020-04-08 10:01 ` Borislav Petkov
2020-04-07 22:04 ` Paolo Bonzini
2020-04-07 23:21 ` Thomas Gleixner
2020-04-08 8:23 ` Paolo Bonzini
2020-04-08 13:01 ` Thomas Gleixner
2020-04-08 15:38 ` Peter Zijlstra
2020-04-08 16:41 ` Thomas Gleixner
2020-04-09 9:03 ` Paolo Bonzini
2020-04-08 15:34 ` Sean Christopherson
2020-04-08 16:50 ` Paolo Bonzini
2020-04-08 18:01 ` Thomas Gleixner
2020-04-08 20:34 ` Vivek Goyal
2020-04-08 23:06 ` Thomas Gleixner
2020-04-08 23:14 ` Thomas Gleixner
2020-04-09 4:50 ` Andy Lutomirski
2020-04-09 9:43 ` Paolo Bonzini
2020-04-09 11:36 ` Andrew Cooper
2020-04-09 12:47 ` Paolo Bonzini
2020-04-09 14:13 ` Andrew Cooper
2020-04-09 14:32 ` Paolo Bonzini
2020-04-09 15:03 ` Andy Lutomirski
2020-04-09 15:17 ` Paolo Bonzini
2020-04-09 17:32 ` Andy Lutomirski
2020-04-06 21:32 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=B85606B0-71B5-4B7D-A892-293CB9C1B434@amacapital.net \
--to=luto@amacapital.net \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=vgoyal@redhat.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).