From: Andy Lutomirski <luto@amacapital.net>
To: Dave Hansen <dave.hansen@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Keno Fischer <keno@juliacomputing.com>,
linux-kernel@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>,
x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
Andi Kleen <andi@firstfloor.org>, Kyle Huey <khuey@kylehuey.com>,
Robert O'Callahan <robert@ocallahan.org>
Subject: Re: [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET_XCR0 to set XCR0 per-thread
Date: Tue, 7 Apr 2020 07:16:34 -0700 [thread overview]
Message-ID: <BEA3CCB8-5127-4E6A-9696-E293C00BFA82@amacapital.net> (raw)
In-Reply-To: <a5b07aa9-96ea-a9b5-13db-e5dcbd7760e6@intel.com>
> On Apr 7, 2020, at 7:07 AM, Dave Hansen <dave.hansen@intel.com> wrote:
>
> On 4/7/20 5:21 AM, Peter Zijlstra wrote:
>> You had a fairly long changelog detailing what the patchd does; but I've
>> failed to find a single word on _WHY_ we want to do any of that.
>
> The goal in these record/replay systems is to be able to recreate thee
> exact same program state on two systems at two different times. To make
> it reasonably fast, they try to minimize the number of snapshots they
> have to take and avoid things like single stepping.
>
> So, there are some windows where they just let the CPU run and don't
> bother with taking any snapshots of register state, for instance. Let's
> say you read a word from shared memory, multiply it and shift it around
> some registers, then stick it back in shared memory. Most of these
> things will just a record the snapshot at the memory read and assume
> that all the instructions in the middle execute deterministically. That
> eliminates a ton of snapshots.
>
> But, what if an instruction in the middle isn't deterministic between
> two machines. Let's say you record a trace on a a Broadwell system,
> then try to replay it on a Skylake, and one of the non-snapshotted
> instructions is xgetbv. Skylake added MPX, so xgetbv will return
> different values. Your replay diverges from what was "recorded", and
> life sucks.
>
> Same problem exists for CPUID, but that was hacked around in another set.
>
> I'm also trying to think of what kinds of things CPU companies add to
> their architectures that would break this stuff. I can't recall ever
> having a discussion with folks at Intel where we're designing a CPU
> feature and we say, "Can't do that, it would break record/replay". I
> suspect there are more of these landmines around and I bet that we're
> building more of them into CPUs every day.
TSX!
I think rr should give the raw KVM API at least a try. It should be possible to fire up a vCPU in CPL3 in the correct state. No guest kernel required. I don’t know if there will be issues with the perf API, though.
If we actually do merge this XCR0 hack, I think the rule should be that it has no effect on kernel behavior. Signals, ptrace, etc reflect the normal XCR0, not the overridden value.
next prev parent reply other threads:[~2020-04-07 14:16 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-07 1:12 [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET_XCR0 to set XCR0 per-thread Keno Fischer
2020-04-07 3:57 ` Andy Lutomirski
2020-04-07 4:44 ` Keno Fischer
2020-04-07 4:53 ` Kyle Huey
2020-04-07 12:33 ` Peter Zijlstra
2020-04-07 13:52 ` Keno Fischer
2020-04-07 12:21 ` Peter Zijlstra
2020-04-07 14:06 ` Dave Hansen
2020-04-07 14:16 ` Andy Lutomirski [this message]
2020-04-07 18:30 ` Keno Fischer
2020-04-14 23:20 ` Andy Lutomirski
2020-04-15 0:09 ` Keno Fischer
2020-04-16 1:07 ` Andy Lutomirski
2020-04-16 1:14 ` Keno Fischer
2020-04-16 1:16 ` Keno Fischer
2020-04-16 1:22 ` Andy Lutomirski
2020-04-07 16:29 ` Kyle Huey
2020-04-07 13:14 ` Dave Hansen
[not found] ` <CABV8kRw1TQsqs+z43bSfZ5isctuFGMB4g_ztDYihiiXHcy4nVA@mail.gmail.com>
2020-04-07 16:27 ` Dave Hansen
2020-04-07 17:55 ` Keno Fischer
2020-04-07 20:21 ` Dave Hansen
2020-04-07 21:42 ` Andy Lutomirski
2020-04-07 22:15 ` Keno Fischer
2020-04-14 19:55 ` Keno Fischer
2020-04-07 14:20 ` Andi Kleen
2020-04-07 18:06 ` Keno Fischer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BEA3CCB8-5127-4E6A-9696-E293C00BFA82@amacapital.net \
--to=luto@amacapital.net \
--cc=andi@firstfloor.org \
--cc=bp@alien8.de \
--cc=dave.hansen@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=keno@juliacomputing.com \
--cc=khuey@kylehuey.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=robert@ocallahan.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).