linux-sgx.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Kai Huang <kai.huang@intel.com>
To: linux-sgx@vger.kernel.org, kvm@vger.kernel.org, x86@kernel.org
Cc: seanjc@google.com, jarkko@kernel.org, luto@kernel.org,
	dave.hansen@intel.com, haitao.huang@intel.com,
	pbonzini@redhat.com, bp@alien8.de, tglx@linutronix.de,
	mingo@redhat.com, hpa@zytor.com, Kai Huang <kai.huang@intel.com>
Subject: [RFC PATCH v2 05/26] x86/sgx: Introduce virtual EPC for use by KVM guests
Date: Mon, 18 Jan 2021 16:26:53 +1300	[thread overview]
Message-ID: <4597db567351468c360fc810fff5a8232cb96c4c.1610935432.git.kai.huang@intel.com> (raw)
In-Reply-To: <cover.1610935432.git.kai.huang@intel.com>

From: Sean Christopherson <sean.j.christopherson@intel.com>

Add a misc device /dev/sgx_virt_epc to allow userspace to allocate "raw"
EPC without an associated enclave.  The intended and only known use case
for raw EPC allocation is to expose EPC to a KVM guest, hence the
virt_epc moniker, virt.{c,h} files and X86_SGX_VIRTUALIZATION Kconfig.

More specifically, to allocate a virtual EPC instance with particular
size, the userspace hypervisor opens the device node, and uses mmap()
with the intended size to get an address range of virtual EPC.  Then
it may use the address range to create one KVM memory slot as virtual
EPC for guest.

Implement the "raw" EPC allocation in the x86 core-SGX subsystem via
/dev/sgx_virt_epc rather than in KVM. Doing so has two major advantages:

  - Does not require changes to KVM's uAPI, e.g. EPC gets handled as
    just another memory backend for guests.

  - EPC management is wholly contained in the SGX subsystem, e.g. SGX
    does not have to export any symbols, changes to reclaim flows don't
    need to be routed through KVM, SGX's dirty laundry doesn't have to
    get aired out for the world to see, and so on and so forth.

The virtual EPC pages allocated to guests are currently not reclaimable.
Reclaiming EPC page used by enclave requires a special reclaim mechanism
separate from normal page reclaim, and that mechanism is not supported
for virutal EPC pages.  Due to the complications of handling reclaim
conflicts between guest and host, reclaiming virtual EPC pages is
significantly more complex than basic support for SGX virtualization.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Co-developed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Kai Huang <kai.huang@intel.com>
---
v1->v2:

 - Added one paragraph to explain fops of virtual EPC, per Jarkko's suggestion.
 - Moved change to sgx_init() out of this patch to a separate patch, as stated
   in cover letter.
 - In sgx_virt_epc_init(), return error if VMX is not supported, or
   CONFIG_KVM_INTEL is not enabled, because there's no point to create
   /dev/sgx_virt_epc if KVM is not supported.
 - Removed 'struct mm_struct *mm' in 'struct sgx_virt_epc', and related logic in
   sgx_virt_epc_open/release/mmap(), per Dave's comment.
 - Renamed 'virtual_epc_zombie_pages' and 'virt_epc_lock' to 'zombie_secs_pages'
   'zombie_secs_pages_lock', per Dave's suggestion.
 - Changed __sgx_free_epc_page() to sgx_free_epc_page() due to Jarkko's patch
   removes EREMOVE in sgx_free_epc_page().
 - Changed all struct sgx_virt_epc *epc to struct sgx_virt_epc *vepc.
 - In __sgx_virt_epc_fault(), changed comment to use WARN_ON() to make sure
   vepc->lock has already been hold, per Dave's suggestion.
 - In sgx_virt_epc_free_page(), added comments to explain SGX_ENCLAVE_ACT is not
   expected; and changed to use WARN_ONCE() to dump actual error code, per
   Dave's comment.
 - Removed NULL page check in sgx_virt_epc_free_page(), per Dave's comment.

---
 arch/x86/Kconfig                 |  12 ++
 arch/x86/kernel/cpu/sgx/Makefile |   1 +
 arch/x86/kernel/cpu/sgx/virt.c   | 255 +++++++++++++++++++++++++++++++
 arch/x86/kernel/cpu/sgx/virt.h   |  14 ++
 4 files changed, 282 insertions(+)
 create mode 100644 arch/x86/kernel/cpu/sgx/virt.c
 create mode 100644 arch/x86/kernel/cpu/sgx/virt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 21f851179ff0..d089ddf01846 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1951,6 +1951,18 @@ config X86_SGX
 
 	  If unsure, say N.
 
+config X86_SGX_VIRTUALIZATION
+	bool "Software Guard eXtensions (SGX) Virtualization"
+	depends on X86_SGX && KVM_INTEL
+	help
+
+	  Enables KVM guests to create SGX enclaves.
+
+	  This includes support to expose "raw" unreclaimable enclave memory to
+	  guests via a device node, e.g. /dev/sgx_virt_epc.
+
+	  If unsure, say N.
+
 config EFI
 	bool "EFI runtime service support"
 	depends on ACPI
diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile
index 91d3dc784a29..7a25bf63adfb 100644
--- a/arch/x86/kernel/cpu/sgx/Makefile
+++ b/arch/x86/kernel/cpu/sgx/Makefile
@@ -3,3 +3,4 @@ obj-y += \
 	encl.o \
 	ioctl.o \
 	main.o
+obj-$(CONFIG_X86_SGX_VIRTUALIZATION)	+= virt.o
diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c
new file mode 100644
index 000000000000..1e8620f20651
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/virt.c
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0
+/*  Copyright(c) 2016-20 Intel Corporation. */
+
+#define pr_fmt(fmt)	"SGX virtual EPC: " fmt
+
+#include <linux/miscdevice.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/sched/mm.h>
+#include <linux/sched/signal.h>
+#include <linux/slab.h>
+#include <linux/xarray.h>
+#include <asm/sgx.h>
+#include <uapi/asm/sgx.h>
+
+#include "encls.h"
+#include "sgx.h"
+#include "virt.h"
+
+struct sgx_virt_epc {
+	struct xarray page_array;
+	struct mutex lock;
+};
+
+static struct mutex zombie_secs_pages_lock;
+static struct list_head zombie_secs_pages;
+
+static int __sgx_virt_epc_fault(struct sgx_virt_epc *vepc,
+				struct vm_area_struct *vma, unsigned long addr)
+{
+	struct sgx_epc_page *epc_page;
+	unsigned long index, pfn;
+	int ret;
+
+	WARN_ON(!mutex_is_locked(&vepc->lock));
+
+	/* Calculate index of EPC page in virtual EPC's page_array */
+	index = vma->vm_pgoff + PFN_DOWN(addr - vma->vm_start);
+
+	epc_page = xa_load(&vepc->page_array, index);
+	if (epc_page)
+		return 0;
+
+	epc_page = sgx_alloc_epc_page(vepc, false);
+	if (IS_ERR(epc_page))
+		return PTR_ERR(epc_page);
+
+	ret = xa_err(xa_store(&vepc->page_array, index, epc_page, GFP_KERNEL));
+	if (ret)
+		goto err_free;
+
+	pfn = PFN_DOWN(sgx_get_epc_phys_addr(epc_page));
+
+	ret = vmf_insert_pfn(vma, addr, pfn);
+	if (ret != VM_FAULT_NOPAGE) {
+		ret = -EFAULT;
+		goto err_delete;
+	}
+
+	return 0;
+
+err_delete:
+	xa_erase(&vepc->page_array, index);
+err_free:
+	sgx_free_epc_page(epc_page);
+	return ret;
+}
+
+static vm_fault_t sgx_virt_epc_fault(struct vm_fault *vmf)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct sgx_virt_epc *vepc = vma->vm_private_data;
+	int ret;
+
+	mutex_lock(&vepc->lock);
+	ret = __sgx_virt_epc_fault(vepc, vma, vmf->address);
+	mutex_unlock(&vepc->lock);
+
+	if (!ret)
+		return VM_FAULT_NOPAGE;
+
+	if (ret == -EBUSY && (vmf->flags & FAULT_FLAG_ALLOW_RETRY)) {
+		mmap_read_unlock(vma->vm_mm);
+		return VM_FAULT_RETRY;
+	}
+
+	return VM_FAULT_SIGBUS;
+}
+
+const struct vm_operations_struct sgx_virt_epc_vm_ops = {
+	.fault = sgx_virt_epc_fault,
+};
+
+static int sgx_virt_epc_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct sgx_virt_epc *vepc = file->private_data;
+
+	if (!(vma->vm_flags & VM_SHARED))
+		return -EINVAL;
+
+	vma->vm_ops = &sgx_virt_epc_vm_ops;
+	/* Don't copy VMA in fork() */
+	vma->vm_flags |= VM_PFNMAP | VM_IO | VM_DONTDUMP | VM_DONTCOPY;
+	vma->vm_private_data = vepc;
+
+	return 0;
+}
+
+static int sgx_virt_epc_free_page(struct sgx_epc_page *epc_page)
+{
+	int ret;
+
+	/*
+	 * Take a previously guest-owned EPC page and return it to the
+	 * general EPC page pool.
+	 *
+	 * Guests can not be trusted to have left this page in a good
+	 * state, so run EREMOVE on the page unconditionally.  In the
+	 * case that a guest properly EREMOVE'd this page, a superfluous
+	 * EREMOVE is harmless.
+	 */
+	ret = __eremove(sgx_get_epc_virt_addr(epc_page));
+	if (ret) {
+		/*
+		 * Only SGX_CHILD_PRESENT is expected, which is because of
+		 * EREMOVE'ing an SECS still with child, in which case it can
+		 * be handled by EREMOVE'ing the SECS again after all pages in
+		 * virtual EPC have been EREMOVE'd. See comments in below in
+		 * sgx_virt_epc_release().
+		 *
+		 * The user of virtual EPC (KVM) needs to guarantee there's no
+		 * logical processor is still running in the enclave in guest,
+		 * otherwise EREMOVE will get SGX_ENCLAVE_ACT which cannot be
+		 * handled here.
+		 */
+		WARN_ONCE(ret != SGX_CHILD_PRESENT,
+			  "EREMOVE (EPC page 0x%lx): unexpected error: %d\n",
+			  sgx_get_epc_phys_addr(epc_page), ret);
+		return ret;
+	}
+
+	sgx_free_epc_page(epc_page);
+	return 0;
+}
+
+static int sgx_virt_epc_release(struct inode *inode, struct file *file)
+{
+	struct sgx_virt_epc *vepc = file->private_data;
+	struct sgx_epc_page *epc_page, *tmp, *entry;
+	unsigned long index;
+
+	LIST_HEAD(secs_pages);
+
+	xa_for_each(&vepc->page_array, index, entry) {
+		/*
+		 * Remove all normal, child pages.  sgx_virt_epc_free_page()
+		 * will fail if EREMOVE fails, but this is OK and expected on
+		 * SECS pages.  Those can only be EREMOVE'd *after* all their
+		 * child pages. Retries below will clean them up.
+		 */
+		if (sgx_virt_epc_free_page(entry))
+			continue;
+
+		xa_erase(&vepc->page_array, index);
+	}
+
+	/*
+	 * Retry EREMOVE'ing pages.  This will clean up any SECS pages that
+	 * only had children in this 'epc' area.
+	 */
+	xa_for_each(&vepc->page_array, index, entry) {
+		epc_page = entry;
+		/*
+		 * An EREMOVE failure here means that the SECS page
+		 * still has children.  But, since all children in this
+		 * 'sgx_virt_epc' have been removed, the SECS page must
+		 * have a child on another instance.
+		 */
+		if (sgx_virt_epc_free_page(epc_page))
+			list_add_tail(&epc_page->list, &secs_pages);
+
+		xa_erase(&vepc->page_array, index);
+	}
+
+	/*
+	 * SECS pages are "pinned" by child pages, an unpinned once all
+	 * children have been EREMOVE'd.  A child page in this instance
+	 * may have pinned an SECS page encountered in an earlier
+	 * release(), creating a zombie.  Since some children were
+	 * EREMOVE'd above, try to EREMOVE all zombies in the hopes that
+	 * one was unpinned.
+	 */
+	mutex_lock(&zombie_secs_pages_lock);
+	list_for_each_entry_safe(epc_page, tmp, &zombie_secs_pages, list) {
+		/*
+		 * Speculatively remove the page from the list of zombies, if
+		 * the page is successfully EREMOVE it will be added to the
+		 * list of free pages.  If EREMOVE fails, throw the page on the
+		 * local list, which will be spliced on at the end.
+		 */
+		list_del(&epc_page->list);
+
+		if (sgx_virt_epc_free_page(epc_page))
+			list_add_tail(&epc_page->list, &secs_pages);
+	}
+
+	if (!list_empty(&secs_pages))
+		list_splice_tail(&secs_pages, &zombie_secs_pages);
+	mutex_unlock(&zombie_secs_pages_lock);
+
+	kfree(vepc);
+
+	return 0;
+}
+
+static int sgx_virt_epc_open(struct inode *inode, struct file *file)
+{
+	struct sgx_virt_epc *vepc;
+
+	vepc = kzalloc(sizeof(struct sgx_virt_epc), GFP_KERNEL);
+	if (!vepc)
+		return -ENOMEM;
+	mutex_init(&vepc->lock);
+	xa_init(&vepc->page_array);
+
+	file->private_data = vepc;
+
+	return 0;
+}
+
+static const struct file_operations sgx_virt_epc_fops = {
+	.owner			= THIS_MODULE,
+	.open			= sgx_virt_epc_open,
+	.release		= sgx_virt_epc_release,
+	.mmap			= sgx_virt_epc_mmap,
+};
+
+static struct miscdevice sgx_virt_epc_dev = {
+	.minor = MISC_DYNAMIC_MINOR,
+	.name = "sgx_virt_epc",
+	.nodename = "sgx_virt_epc",
+	.fops = &sgx_virt_epc_fops,
+};
+
+int __init sgx_virt_epc_init(void)
+{
+	/* SGX virtualization requires KVM to work */
+	if (!boot_cpu_has(X86_FEATURE_VMX) || !IS_ENABLED(CONFIG_KVM_INTEL))
+		return -ENODEV;
+
+	INIT_LIST_HEAD(&zombie_secs_pages);
+	mutex_init(&zombie_secs_pages_lock);
+
+	return misc_register(&sgx_virt_epc_dev);
+}
diff --git a/arch/x86/kernel/cpu/sgx/virt.h b/arch/x86/kernel/cpu/sgx/virt.h
new file mode 100644
index 000000000000..e5434541a122
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/virt.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+#ifndef _ASM_X86_SGX_VIRT_H
+#define _ASM_X86_SGX_VIRT_H
+
+#ifdef CONFIG_X86_SGX_VIRTUALIZATION
+int __init sgx_virt_epc_init(void);
+#else
+static inline int __init sgx_virt_epc_init(void)
+{
+	return -ENODEV;
+}
+#endif
+
+#endif /* _ASM_X86_SGX_VIRT_H */
-- 
2.29.2


  parent reply	other threads:[~2021-01-18  3:28 UTC|newest]

Thread overview: 91+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-18  3:26 [RFC PATCH v2 00/26] KVM SGX virtualization support Kai Huang
2021-01-18  3:26 ` [RFC PATCH v2 01/26] x86/cpufeatures: Add SGX1 and SGX2 sub-features Kai Huang
2021-01-19 16:19   ` Borislav Petkov
2021-01-19 18:03     ` Sean Christopherson
2021-01-19 22:54       ` Kai Huang
2021-01-20 10:28       ` Borislav Petkov
2021-01-20 11:50   ` Jarkko Sakkinen
2021-01-20 23:23     ` Kai Huang
2021-01-21  1:01       ` Jarkko Sakkinen
2021-01-18  3:26 ` [RFC PATCH v2 02/26] x86/sgx: Remove a warn from sgx_free_epc_page() Kai Huang
2021-01-19  8:32   ` Jarkko Sakkinen
2021-01-20  0:42     ` Kai Huang
2021-01-18  3:26 ` [RFC PATCH v2 03/26] x86/sgx: Wipe out EREMOVE " Kai Huang
2021-01-18  3:26 ` [RFC PATCH v2 04/26] x86/sgx: Add SGX_CHILD_PRESENT hardware error code Kai Huang
2021-01-20 11:51   ` Jarkko Sakkinen
2021-01-18  3:26 ` Kai Huang [this message]
2021-01-20 11:54   ` [RFC PATCH v2 05/26] x86/sgx: Introduce virtual EPC for use by KVM guests Jarkko Sakkinen
2021-01-20 17:40     ` Sean Christopherson
2021-01-21  0:54       ` Jarkko Sakkinen
2021-01-21  0:55         ` Jarkko Sakkinen
2021-01-18  3:27 ` [RFC PATCH v2 06/26] x86/cpu/intel: Allow SGX virtualization without Launch Control support Kai Huang
2021-01-20 21:02   ` Dave Hansen
2021-01-20 22:36     ` Sean Christopherson
2021-01-20 23:27       ` Dave Hansen
2021-01-20 23:48         ` Kai Huang
2021-01-20 23:51           ` Dave Hansen
2021-01-21  1:53             ` Kai Huang
2021-01-21 14:35               ` Jarkko Sakkinen
2021-01-21  1:12         ` Jarkko Sakkinen
2021-01-20 23:50     ` Kai Huang
2021-01-21  1:11     ` Jarkko Sakkinen
2021-01-18  3:27 ` [RFC PATCH v2 07/26] x86/sgx: Initialize virtual EPC driver even when SGX driver is disabled Kai Huang
2021-01-18  3:27 ` [RFC PATCH v2 08/26] x86/sgx: Expose SGX architectural definitions to the kernel Kai Huang
2021-01-20 11:58   ` Jarkko Sakkinen
2021-01-20 23:53     ` Kai Huang
2021-01-18  3:27 ` [RFC PATCH v2 09/26] x86/sgx: Move ENCLS leaf definitions to sgx_arch.h Kai Huang
2021-01-20 11:59   ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 10/26] x86/sgx: Add SGX2 ENCLS leaf definitions (EAUG, EMODPR and EMODT) Kai Huang
2021-01-20 11:59   ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 11/26] x86/sgx: Add encls_faulted() helper Kai Huang
2021-01-20 12:03   ` Jarkko Sakkinen
2021-01-20 23:43     ` Kai Huang
2021-01-21  1:08       ` Jarkko Sakkinen
2021-01-21  1:12         ` Kai Huang
2021-01-21 14:38           ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 12/26] x86/sgx: Add helper to update SGX_LEPUBKEYHASHn MSRs Kai Huang
2021-01-20 12:03   ` Jarkko Sakkinen
2021-01-20 18:36     ` Dave Hansen
2021-01-20 23:36       ` Kai Huang
2021-01-20 23:50         ` Dave Hansen
2021-01-21  1:06           ` Kai Huang
2021-01-21  1:15             ` Dave Hansen
2021-01-21  1:44               ` Kai Huang
2021-01-21 14:36                 ` Jarkko Sakkinen
2021-01-21  1:18             ` Kai Huang
2021-01-21  1:09         ` Jarkko Sakkinen
2021-01-21  1:08       ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 13/26] x86/sgx: Add helpers to expose ECREATE and EINIT to KVM Kai Huang
2021-01-20 12:04   ` Jarkko Sakkinen
2021-01-20 23:29     ` Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 14/26] x86/sgx: Move provisioning device creation out of SGX driver Kai Huang
2021-01-20 14:09   ` Jarkko Sakkinen
2021-01-20 23:24     ` Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 15/26] KVM: VMX: Convert vcpu_vmx.exit_reason to a union Kai Huang
2021-01-20 14:18   ` Jarkko Sakkinen
2021-01-20 16:39     ` Sean Christopherson
2021-01-21  0:47       ` Jarkko Sakkinen
2021-01-21 16:33         ` Sean Christopherson
2021-01-22 17:29           ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 16/26] KVM: x86: Export kvm_mmu_gva_to_gpa_{read,write}() for SGX (VMX) Kai Huang
2021-01-20 14:19   ` Jarkko Sakkinen
2021-01-18  3:28 ` [RFC PATCH v2 17/26] KVM: x86: Define new #PF SGX error code bit Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 18/26] KVM: x86: Add support for reverse CPUID lookup of scattered features Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 19/26] KVM: x86: Add reverse-CPUID lookup support for scattered SGX features Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 20/26] KVM: VMX: Add basic handling of VM-Exit from SGX enclave Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 21/26] KVM: VMX: Frame in ENCLS handler for SGX virtualization Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 22/26] KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 23/26] KVM: VMX: Add emulation of SGX Launch Control LE hash MSRs Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 24/26] KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC) Kai Huang
2021-01-18  3:28 ` [RFC PATCH v2 25/26] KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC Kai Huang
2021-01-18  3:29 ` [RFC PATCH v2 26/26] KVM: x86: Add capability to grant VM access to privileged SGX attribute Kai Huang
2021-01-19  8:23 ` [RFC PATCH v2 00/26] KVM SGX virtualization support Jarkko Sakkinen
2021-01-20  0:52   ` Kai Huang
2021-01-20 16:35     ` Sean Christopherson
2021-01-20 16:39       ` Paolo Bonzini
2021-01-21  1:28         ` Kai Huang
2021-01-20 23:43     ` Jarkko Sakkinen
2021-01-20 23:52       ` Kai Huang
2021-01-21  1:16         ` Jarkko Sakkinen
2021-01-21  1:27           ` Kai Huang
2021-01-21 14:34             ` Jarkko Sakkinen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4597db567351468c360fc810fff5a8232cb96c4c.1610935432.git.kai.huang@intel.com \
    --to=kai.huang@intel.com \
    --cc=bp@alien8.de \
    --cc=dave.hansen@intel.com \
    --cc=haitao.huang@intel.com \
    --cc=hpa@zytor.com \
    --cc=jarkko@kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=mingo@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).