linux-coco.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: Joerg Roedel <joro@8bytes.org>
To: x86@kernel.org
Cc: Eric Biederman <ebiederm@xmission.com>,
	kexec@lists.infradead.org, Joerg Roedel <jroedel@suse.de>,
	hpa@zytor.com, Andy Lutomirski <luto@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Jiri Slaby <jslaby@suse.cz>,
	Dan Williams <dan.j.williams@intel.com>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	Juergen Gross <jgross@suse.com>,
	Kees Cook <keescook@chromium.org>,
	David Rientjes <rientjes@google.com>,
	Cfir Cohen <cfir@google.com>, Erdem Aktas <erdemaktas@google.com>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Mike Stunes <mstunes@vmware.com>,
	Sean Christopherson <seanjc@google.com>,
	Martin Radev <martin.b.radev@gmail.com>,
	Arvind Sankar <nivedita@alum.mit.edu>,
	Joerg Roedel <joro@8bytes.org>,
	linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org,
	kvm@vger.kernel.org, virtualization@lists.linux-foundation.org
Subject: [PATCH v2 00/12] x86/sev: KEXEC/KDUMP support for SEV-ES guests
Date: Mon, 13 Sep 2021 17:55:51 +0200	[thread overview]
Message-ID: <20210913155603.28383-1-joro@8bytes.org> (raw)

From: Joerg Roedel <jroedel@suse.de>

Hi,

here are changes to enable kexec/kdump in SEV-ES guests. The biggest
problem for supporting kexec/kdump under SEV-ES is to find a way to
hand the non-boot CPUs (APs) from one kernel to another.

Without SEV-ES the first kernel parks the CPUs in a HLT loop until
they get reset by the kexec'ed kernel via an INIT-SIPI-SIPI sequence.
For virtual machines the CPU reset is emulated by the hypervisor,
which sets the vCPU registers back to reset state.

This does not work under SEV-ES, because the hypervisor has no access
to the vCPU registers and can't make modifications to them. So an
SEV-ES guest needs to reset the vCPU itself and park it using the
AP-reset-hold protocol. Upon wakeup the guest needs to jump to
real-mode and to the reset-vector configured in the AP-Jump-Table.

The code to do this is the main part of this patch-set. It works by
placing code on the AP Jump-Table page itself to park the vCPU and for
jumping to the reset vector upon wakeup. The code on the AP Jump Table
runs in 16-bit protected mode with segment base set to the beginning
of the page. The AP Jump-Table is usually not within the first 1MB of
memory, so the code can't run in real-mode.

The AP Jump-Table is the best place to put the parking code, because
the memory is owned, but read-only by the firmware and writeable by
the OS. Only the first 4 bytes are used for the reset-vector, leaving
the rest of the page for code/data/stack to park a vCPU. The code
can't be in kernel memory because by the time the vCPU wakes up the
memory will be owned by the new kernel, which might have overwritten it
already.

The other patches add initial GHCB Version 2 protocol support, because
kexec/kdump need the MSR-based (without a GHCB) AP-reset-hold VMGEXIT,
which is a GHCB protocol version 2 feature.

The kexec'ed kernel is also entered via the decompressor and needs
MMIO support there, so this patch-set also adds MMIO #VC support to
the decompressor and support for handling CLFLUSH instructions.

Finally there is also code to disable kexec/kdump support at runtime
when the environment does not support it (e.g. no GHCB protocol
version 2 support or AP Jump Table over 4GB).

The diffstat looks big, but most of it is moving code for MMIO #VC
support around to make it available to the decompressor.

These patches need a fix I sent out earlier today to work reliably:

	https://lore.kernel.org/lkml/20210913095236.24937-1-joro@8bytes.org/

Please review.

Thanks,

	Joerg

Changes v1->v2:

	- Rebased to v5.15-rc1

	- Fixed occasional triple-faults when parking APs, see
	  separate fix

Joerg Roedel (12):
  kexec: Allow architecture code to opt-out at runtime
  x86/kexec/64: Forbid kexec when running as an SEV-ES guest
  x86/sev: Save and print negotiated GHCB protocol version
  x86/sev: Do not hardcode GHCB protocol version
  x86/sev: Use GHCB protocol version 2 if supported
  x86/sev: Cache AP Jump Table Address
  x86/sev: Setup code to park APs in the AP Jump Table
  x86/sev: Park APs on AP Jump Table with GHCB protocol version 2
  x86/sev: Use AP Jump Table blob to stop CPU
  x86/sev: Add MMIO handling support to boot/compressed/ code
  x86/sev: Handle CLFLUSH MMIO events
  x86/sev: Support kexec under SEV-ES with AP Jump Table blob

 arch/x86/boot/compressed/sev.c          |  56 +-
 arch/x86/include/asm/realmode.h         |   5 +
 arch/x86/include/asm/sev-ap-jumptable.h |  25 +
 arch/x86/include/asm/sev.h              |  13 +-
 arch/x86/kernel/machine_kexec_64.c      |  12 +
 arch/x86/kernel/process.c               |   8 +
 arch/x86/kernel/sev-shared.c            | 333 +++++++++-
 arch/x86/kernel/sev.c                   | 494 ++++++---------
 arch/x86/lib/insn-eval-shared.c         | 805 ++++++++++++++++++++++++
 arch/x86/lib/insn-eval.c                | 802 +----------------------
 arch/x86/realmode/Makefile              |   9 +-
 arch/x86/realmode/rm/Makefile           |  11 +-
 arch/x86/realmode/rm/header.S           |   3 +
 arch/x86/realmode/rm/sev_ap_park.S      |  89 +++
 arch/x86/realmode/rmpiggy.S             |   6 +
 arch/x86/realmode/sev/Makefile          |  41 ++
 arch/x86/realmode/sev/ap_jump_table.S   | 130 ++++
 arch/x86/realmode/sev/ap_jump_table.lds |  24 +
 include/linux/kexec.h                   |   1 +
 kernel/kexec.c                          |  14 +
 kernel/kexec_file.c                     |   9 +
 21 files changed, 1764 insertions(+), 1126 deletions(-)
 create mode 100644 arch/x86/include/asm/sev-ap-jumptable.h
 create mode 100644 arch/x86/lib/insn-eval-shared.c
 create mode 100644 arch/x86/realmode/rm/sev_ap_park.S
 create mode 100644 arch/x86/realmode/sev/Makefile
 create mode 100644 arch/x86/realmode/sev/ap_jump_table.S
 create mode 100644 arch/x86/realmode/sev/ap_jump_table.lds


base-commit: 6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f
-- 
2.33.0


             reply	other threads:[~2021-09-13 15:56 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-13 15:55 Joerg Roedel [this message]
2021-09-13 15:55 ` [PATCH v2 01/12] kexec: Allow architecture code to opt-out at runtime Joerg Roedel
2021-11-01 16:10   ` Borislav Petkov
2021-11-01 21:11     ` Eric W. Biederman
2021-11-02 16:37       ` Joerg Roedel
2021-11-02 17:00       ` Joerg Roedel
2021-11-02 18:17         ` Eric W. Biederman
2021-11-02 17:17       ` Borislav Petkov
2021-09-13 15:55 ` [PATCH v2 02/12] x86/kexec/64: Forbid kexec when running as an SEV-ES guest Joerg Roedel
2021-09-13 15:55 ` [PATCH v2 03/12] x86/sev: Save and print negotiated GHCB protocol version Joerg Roedel
2021-11-03 14:27   ` Borislav Petkov
2021-09-13 15:55 ` [PATCH v2 04/12] x86/sev: Do not hardcode " Joerg Roedel
2021-09-13 15:55 ` [PATCH v2 05/12] x86/sev: Use GHCB protocol version 2 if supported Joerg Roedel
2021-11-03 16:05   ` Borislav Petkov
2021-09-13 15:55 ` [PATCH v2 06/12] x86/sev: Cache AP Jump Table Address Joerg Roedel
2021-11-08 18:14   ` Borislav Petkov
2021-09-13 15:55 ` [PATCH v2 07/12] x86/sev: Setup code to park APs in the AP Jump Table Joerg Roedel
2021-11-10 16:37   ` Borislav Petkov
2021-09-13 15:55 ` [PATCH v2 08/12] x86/sev: Park APs on AP Jump Table with GHCB protocol version 2 Joerg Roedel
2021-11-12 16:33   ` Borislav Petkov
2021-09-13 15:56 ` [PATCH v2 09/12] x86/sev: Use AP Jump Table blob to stop CPU Joerg Roedel
2021-11-15 18:44   ` Borislav Petkov
2021-09-13 15:56 ` [PATCH v2 10/12] x86/sev: Add MMIO handling support to boot/compressed/ code Joerg Roedel
2021-09-13 15:56 ` [PATCH v2 11/12] x86/sev: Handle CLFLUSH MMIO events Joerg Roedel
2021-09-13 15:56 ` [PATCH v2 12/12] x86/sev: Support kexec under SEV-ES with AP Jump Table blob Joerg Roedel
2021-09-13 16:02 ` [PATCH v2 00/12] x86/sev: KEXEC/KDUMP support for SEV-ES guests Dave Hansen
2021-09-13 16:14   ` Joerg Roedel
2021-09-13 16:21     ` Dave Hansen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210913155603.28383-1-joro@8bytes.org \
    --to=joro@8bytes.org \
    --cc=cfir@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=ebiederm@xmission.com \
    --cc=erdemaktas@google.com \
    --cc=hpa@zytor.com \
    --cc=jgross@suse.com \
    --cc=jroedel@suse.de \
    --cc=jslaby@suse.cz \
    --cc=keescook@chromium.org \
    --cc=kexec@lists.infradead.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=martin.b.radev@gmail.com \
    --cc=mhiramat@kernel.org \
    --cc=mstunes@vmware.com \
    --cc=nivedita@alum.mit.edu \
    --cc=peterz@infradead.org \
    --cc=rientjes@google.com \
    --cc=seanjc@google.com \
    --cc=thomas.lendacky@amd.com \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=x86@kernel.org \
    --subject='Re: [PATCH v2 00/12] x86/sev: KEXEC/KDUMP support for SEV-ES guests' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).