* [PATCH v6 00/18] HSM driver for ACRN hypervisor
@ 2020-12-01 9:38 shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 01/18] docs: acrn: Introduce ACRN shuo.a.liu
` (17 more replies)
0 siblings, 18 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu
From: Shuo Liu <shuo.a.liu@intel.com>
ACRN is a Type 1 reference hypervisor stack, running directly on the bare-metal
hardware, and is suitable for a variety of IoT and embedded device solutions.
ACRN implements a hybrid VMM architecture, using a privileged Service VM. The
Service VM manages the system resources (CPU, memory, etc.) and I/O devices of
User VMs. Multiple User VMs are supported, with each of them running Linux,
Android OS or Windows. Both Service VM and User VMs are guest VM.
Below figure shows the architecture.
Service VM User VM
+----------------------------+ | +------------------+
| +--------------+ | | | |
| |ACRN userspace| | | | |
| +--------------+ | | | |
|-----------------ioctl------| | | | ...
|kernel space +----------+ | | | |
| | HSM | | | | Drivers |
| +----------+ | | | |
+--------------------|-------+ | +------------------+
+---------------------hypercall----------------------------------------+
| ACRN Hypervisor |
+----------------------------------------------------------------------+
| Hardware |
+----------------------------------------------------------------------+
There is only one Service VM which could run Linux as OS.
In a typical case, the Service VM will be auto started when ACRN Hypervisor is
booted. Then the ACRN userspace (an application running in Service VM) could be
used to start/stop User VMs by communicating with ACRN Hypervisor Service
Module (HSM).
ACRN Hypervisor Service Module (HSM) is a middle layer that allows the ACRN
userspace and Service VM OS kernel to communicate with ACRN Hypervisor
and manage different User VMs. This middle layer provides the following
functionalities,
- Issues hypercalls to the hypervisor to manage User VMs:
* VM/vCPU management
* Memory management
* Device passthrough
* Interrupts injection
- I/O requests handling from User VMs.
- Exports ioctl through HSM char device.
- Exports function calls for other kernel modules
ACRN is focused on embedded system. So it doesn't support some features.
E.g.,
- ACRN doesn't support VM migration.
- ACRN doesn't support vCPU migration.
This patch set adds the HSM to the Linux kernel.
I also added a simple example to launch a small guest (with several instructions
as payload) on ACRN with demonstration ioctl usage.
The basic ARCN support was merged to upstream already.
https://lore.kernel.org/lkml/1559108037-18813-3-git-send-email-yakui.zhao@intel.com/
ChangeLog:
v6:
- Added the cpuid.rst documentation (Boris)
- Deleted exported acrn_is_privileged_vm(), user detects feature bits by cpuid_eax() directly (Boris)
- Used 'g' as the hcall_id constrain in hypercall definitions (Boris, Segher)
- Removed unnecessary reserved fields from structures. Sorted some fields for alignment (Greg)
- Used built-in kernel guid_t types (Greg)
- Specified the endian of some fields in user/kernel interface structures (Greg)
- Removed the alignment attribute from user/kernel interface structures (Greg)
- Set reserved fields to zero (Greg)
- Added a ioctl interface usage sample in the last patch (Greg)
- Used pin_user_pages*() instead of get_user_pages*().
v5:
- Corrected typo in documentation.
- Removed unused pr_fmt().
- Used supported constraint with a explicit MOV to R8 at beginning of ASM for hypercall interface.
- Used dev_dbg() to replace dev_err() in places which might cause a DoS.
- Introduced acrn_vm_list_lock as a mutex for friendly review.
- Changed to use default attribute group list to add attribute files.
v4:
- Used acrn_dev.this_device directly for dev_*() (Reinette)
- Removed the odd usage of {get|put}_device() on &acrn_dev->this_device (Greg)
- Removed unused log code. (Greg)
- Corrected the return error values. (Greg)
- Mentioned that HSM relies hypervisor for sanity check in acrn_dev_ioctl() comments (Greg)
v3:
- Used {get|put}_device() helpers on &acrn_dev->this_device
- Moved unused code from front patches to later ones.
- Removed self-defined pr_fmt() and dev_fmt()
- Provided comments for acrn_vm_list_lock.
v2:
- Removed API version related code. (Dave)
- Replaced pr_*() by dev_*(). (Greg)
- Used -ENOTTY as the error code of unsupported ioctl. (Greg)
Shuo Liu (17):
docs: acrn: Introduce ACRN
x86/acrn: Introduce acrn_{setup, remove}_intr_handler()
x86/acrn: Introduce hypercall interfaces
virt: acrn: Introduce ACRN HSM basic driver
virt: acrn: Introduce VM management interfaces
virt: acrn: Introduce an ioctl to set vCPU registers state
virt: acrn: Introduce EPT mapping management
virt: acrn: Introduce I/O request management
virt: acrn: Introduce PCI configuration space PIO accesses combiner
virt: acrn: Introduce interfaces for PCI device passthrough
virt: acrn: Introduce interrupt injection interfaces
virt: acrn: Introduce interfaces to query C-states and P-states
allowed by hypervisor
virt: acrn: Introduce I/O ranges operation interfaces
virt: acrn: Introduce ioeventfd
virt: acrn: Introduce irqfd
virt: acrn: Introduce an interface for Service VM to control vCPU
sample/acrn: Introduce a sample of HSM ioctl interface usage
Yin Fengwei (1):
x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits
.../userspace-api/ioctl/ioctl-number.rst | 1 +
Documentation/virt/acrn/cpuid.rst | 46 ++
Documentation/virt/acrn/index.rst | 12 +
Documentation/virt/acrn/introduction.rst | 43 ++
Documentation/virt/acrn/io-request.rst | 97 +++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 9 +
arch/x86/include/asm/acrn.h | 78 +++
arch/x86/kernel/cpu/acrn.c | 16 +-
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/acrn/Kconfig | 15 +
drivers/virt/acrn/Makefile | 3 +
drivers/virt/acrn/acrn_drv.h | 227 ++++++
drivers/virt/acrn/hsm.c | 441 ++++++++++++
drivers/virt/acrn/hypercall.h | 254 +++++++
drivers/virt/acrn/ioeventfd.c | 273 ++++++++
drivers/virt/acrn/ioreq.c | 645 ++++++++++++++++++
drivers/virt/acrn/irqfd.c | 235 +++++++
drivers/virt/acrn/mm.c | 306 +++++++++
drivers/virt/acrn/vm.c | 126 ++++
include/uapi/linux/acrn.h | 479 +++++++++++++
samples/acrn/Makefile | 12 +
samples/acrn/guest.ld | 9 +
samples/acrn/payload.ld | 9 +
samples/acrn/vm-sample.c | 136 ++++
26 files changed, 3475 insertions(+), 1 deletion(-)
create mode 100644 Documentation/virt/acrn/cpuid.rst
create mode 100644 Documentation/virt/acrn/index.rst
create mode 100644 Documentation/virt/acrn/introduction.rst
create mode 100644 Documentation/virt/acrn/io-request.rst
create mode 100644 arch/x86/include/asm/acrn.h
create mode 100644 drivers/virt/acrn/Kconfig
create mode 100644 drivers/virt/acrn/Makefile
create mode 100644 drivers/virt/acrn/acrn_drv.h
create mode 100644 drivers/virt/acrn/hsm.c
create mode 100644 drivers/virt/acrn/hypercall.h
create mode 100644 drivers/virt/acrn/ioeventfd.c
create mode 100644 drivers/virt/acrn/ioreq.c
create mode 100644 drivers/virt/acrn/irqfd.c
create mode 100644 drivers/virt/acrn/mm.c
create mode 100644 drivers/virt/acrn/vm.c
create mode 100644 include/uapi/linux/acrn.h
create mode 100644 samples/acrn/Makefile
create mode 100644 samples/acrn/guest.ld
create mode 100644 samples/acrn/payload.ld
create mode 100644 samples/acrn/vm-sample.c
base-commit: 127c501a03d5db8b833e953728d3bcf53c8832a9
--
2.28.0
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v6 01/18] docs: acrn: Introduce ACRN
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-08 11:52 ` Borislav Petkov
2020-12-01 9:38 ` [PATCH v6 02/18] x86/acrn: Introduce acrn_{setup, remove}_intr_handler() shuo.a.liu
` (16 subsequent siblings)
17 siblings, 1 reply; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Dave Hansen, Dan Williams,
Fengwei Yin, Zhenyu Wang, Randy Dunlap
From: Shuo Liu <shuo.a.liu@intel.com>
Add documentation on the following aspects of ACRN:
1) A brief introduction on the architecture of ACRN.
2) I/O request handling in ACRN.
3) CPUID functions of ACRN.
To learn more about ACRN, please go to ACRN project website
https://projectacrn.org, or the documentation page
https://projectacrn.github.io/.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Sen Christopherson <sean.j.christopherson@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Fengwei Yin <fengwei.yin@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
---
Documentation/virt/acrn/cpuid.rst | 46 +++++++++++
Documentation/virt/acrn/index.rst | 12 +++
Documentation/virt/acrn/introduction.rst | 43 +++++++++++
Documentation/virt/acrn/io-request.rst | 97 ++++++++++++++++++++++++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 7 ++
6 files changed, 206 insertions(+)
create mode 100644 Documentation/virt/acrn/cpuid.rst
create mode 100644 Documentation/virt/acrn/index.rst
create mode 100644 Documentation/virt/acrn/introduction.rst
create mode 100644 Documentation/virt/acrn/io-request.rst
diff --git a/Documentation/virt/acrn/cpuid.rst b/Documentation/virt/acrn/cpuid.rst
new file mode 100644
index 000000000000..972d6803bf63
--- /dev/null
+++ b/Documentation/virt/acrn/cpuid.rst
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+ACRN CPUID bits
+===============
+
+A guest VM running on an ACRN hypervisor, can check some of its features using
+cpuid.
+
+ACRN cpuid functions are:
+
+function: 0x40000000
+
+returns::
+
+ eax = 0x40000010
+ ebx = 0x4e524341
+ ecx = 0x4e524341
+ edx = 0x4e524341
+
+Note that this value in ebx, ecx and edx corresponds to the string
+"ACRNACRNACRN". The value in eax corresponds to the maximum cpuid function
+present in this leaf, and will be updated if more functions are added in the
+future.
+
+function: define ACRN_CPUID_FEATURES (0x40000001)
+
+returns::
+
+ ebx, ecx, edx
+ eax = an OR'ed group of (1 << flag)
+
+where ``flag`` is defined as below:
+
+================================= =========== ================================
+flag value meaning
+================================= =========== ================================
+ACRN_FEATURE_PRIVILEGED_VM 0 guest VM is a privileged VM
+================================= =========== ================================
+
+function: 0x40000010
+
+returns::
+
+ ebx, ecx, edx
+ eax = (Virtual) TSC frequency in kHz.
diff --git a/Documentation/virt/acrn/index.rst b/Documentation/virt/acrn/index.rst
new file mode 100644
index 000000000000..b5f793e73df5
--- /dev/null
+++ b/Documentation/virt/acrn/index.rst
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+ACRN Hypervisor
+===============
+
+.. toctree::
+ :maxdepth: 1
+
+ introduction
+ io-request
+ cpuid
diff --git a/Documentation/virt/acrn/introduction.rst b/Documentation/virt/acrn/introduction.rst
new file mode 100644
index 000000000000..87154d48bbe6
--- /dev/null
+++ b/Documentation/virt/acrn/introduction.rst
@@ -0,0 +1,43 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+ACRN Hypervisor Introduction
+============================
+
+The ACRN Hypervisor is a Type 1 hypervisor, running directly on the bare-metal
+hardware. It has a privileged management VM, called Service VM, to manage User
+VMs and do I/O emulation.
+
+ACRN userspace is an application running in the Service VM that emulates
+devices for a User VM based on command line configurations. ACRN Hypervisor
+Service Module (HSM) is a kernel module in the Service VM which provides
+hypervisor services to the ACRN userspace.
+
+Below figure shows the architecture.
+
+::
+
+ Service VM User VM
+ +----------------------------+ | +------------------+
+ | +--------------+ | | | |
+ | |ACRN userspace| | | | |
+ | +--------------+ | | | |
+ |-----------------ioctl------| | | | ...
+ |kernel space +----------+ | | | |
+ | | HSM | | | | Drivers |
+ | +----------+ | | | |
+ +--------------------|-------+ | +------------------+
+ +---------------------hypercall----------------------------------------+
+ | ACRN Hypervisor |
+ +----------------------------------------------------------------------+
+ | Hardware |
+ +----------------------------------------------------------------------+
+
+ACRN userspace allocates memory for the User VM, configures and initializes the
+devices used by the User VM, loads the virtual bootloader, initializes the
+virtual CPU state and handles I/O request accesses from the User VM. It uses
+ioctls to communicate with the HSM. HSM implements hypervisor services by
+interacting with the ACRN Hypervisor via hypercalls. HSM exports a char device
+interface (/dev/acrn_hsm) to userspace.
+
+The ACRN hypervisor is open for contribution from anyone. The source repo is
+available at https://github.com/projectacrn/acrn-hypervisor.
diff --git a/Documentation/virt/acrn/io-request.rst b/Documentation/virt/acrn/io-request.rst
new file mode 100644
index 000000000000..79022a671ea7
--- /dev/null
+++ b/Documentation/virt/acrn/io-request.rst
@@ -0,0 +1,97 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+I/O request handling
+====================
+
+An I/O request of a User VM, which is constructed by the hypervisor, is
+distributed by the ACRN Hypervisor Service Module to an I/O client
+corresponding to the address range of the I/O request. Details of I/O request
+handling are described in the following sections.
+
+1. I/O request
+--------------
+
+For each User VM, there is a shared 4-KByte memory region used for I/O requests
+communication between the hypervisor and Service VM. An I/O request is a
+256-byte structure buffer, which is 'struct acrn_io_request', that is filled by
+an I/O handler of the hypervisor when a trapped I/O access happens in a User
+VM. ACRN userspace in the Service VM first allocates a 4-KByte page and passes
+the GPA (Guest Physical Address) of the buffer to the hypervisor. The buffer is
+used as an array of 16 I/O request slots with each I/O request slot being 256
+bytes. This array is indexed by vCPU ID.
+
+2. I/O clients
+--------------
+
+An I/O client is responsible for handling User VM I/O requests whose accessed
+GPA falls in a certain range. Multiple I/O clients can be associated with each
+User VM. There is a special client associated with each User VM, called the
+default client, that handles all I/O requests that do not fit into the range of
+any other clients. The ACRN userspace acts as the default client for each User
+VM.
+
+Below illustration shows the relationship between I/O requests shared buffer,
+I/O requests and I/O clients.
+
+::
+
+ +------------------------------------------------------+
+ | Service VM |
+ |+--------------------------------------------------+ |
+ || +----------------------------------------+ | |
+ || | shared page ACRN userspace | | |
+ || | +-----------------+ +------------+ | | |
+ || +----+->| acrn_io_request |<-+ default | | | |
+ || | | | +-----------------+ | I/O client | | | |
+ || | | | | ... | +------------+ | | |
+ || | | | +-----------------+ | | |
+ || | +-|--------------------------------------+ | |
+ ||---|----|-----------------------------------------| |
+ || | | kernel | |
+ || | | +----------------------+ | |
+ || | | | +-------------+ HSM | | |
+ || | +--------------+ | | | |
+ || | | | I/O clients | | | |
+ || | | | | | | |
+ || | | +-------------+ | | |
+ || | +----------------------+ | |
+ |+---|----------------------------------------------+ |
+ +----|-------------------------------------------------+
+ |
+ +----|-------------------------------------------------+
+ | +-+-----------+ |
+ | | I/O handler | ACRN Hypervisor |
+ | +-------------+ |
+ +------------------------------------------------------+
+
+3. I/O request state transition
+-------------------------------
+
+The state transitions of an ACRN I/O request are as follows.
+
+::
+
+ FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...
+
+- FREE: this I/O request slot is empty
+- PENDING: a valid I/O request is pending in this slot
+- PROCESSING: the I/O request is being processed
+- COMPLETE: the I/O request has been processed
+
+An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and
+ACRN userspace are in charge of processing the others.
+
+4. Processing flow of I/O requests
+-------------------------------
+
+a. The I/O handler of the hypervisor will fill an I/O request with PENDING
+ state when a trapped I/O access happens in a User VM.
+b. The hypervisor makes an upcall, which is a notification interrupt, to
+ the Service VM.
+c. The upcall handler schedules a tasklet to dispatch I/O requests.
+d. The tasklet looks for the PENDING I/O requests, assigns them to different
+ registered clients based on the address of the I/O accesses, updates
+ their state to PROCESSING, and notifies the corresponding client to handle.
+e. The notified client handles the assigned I/O requests.
+f. The HSM updates I/O requests states to COMPLETE and notifies the hypervisor
+ of the completion via hypercalls.
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index 350f5c869b56..edea7fea95a8 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -12,6 +12,7 @@ Linux Virtualization Support
paravirt_ops
guest-halt-polling
ne_overview
+ acrn/index
.. only:: html and subproject
diff --git a/MAINTAINERS b/MAINTAINERS
index a008b70f3c16..e6e5bd9ce117 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -436,6 +436,13 @@ S: Orphan
F: drivers/platform/x86/wmi.c
F: include/uapi/linux/wmi.h
+ACRN HYPERVISOR SERVICE MODULE
+M: Shuo Liu <shuo.a.liu@intel.com>
+L: acrn-dev@lists.projectacrn.org
+S: Supported
+W: https://projectacrn.org
+F: Documentation/virt/acrn/
+
AD1889 ALSA SOUND DRIVER
L: linux-parisc@vger.kernel.org
S: Maintained
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 02/18] x86/acrn: Introduce acrn_{setup, remove}_intr_handler()
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 01/18] docs: acrn: Introduce ACRN shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 03/18] x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits shuo.a.liu
` (15 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Yakui Zhao, Zhi Wang, Dave Hansen,
Dan Williams, Fengwei Yin, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
The ACRN Hypervisor builds an I/O request when a trapped I/O access
happens in User VM. Then, ACRN Hypervisor issues an upcall by sending
a notification interrupt to the Service VM. HSM in the Service VM needs
to hook the notification interrupt to handle I/O requests.
Notification interrupts from ACRN Hypervisor are already supported and
a, currently uninitialized, callback called.
Export two APIs for HSM to setup/remove its callback.
Originally-by: Yakui Zhao <yakui.zhao@intel.com>
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Fengwei Yin <fengwei.yin@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
arch/x86/include/asm/acrn.h | 8 ++++++++
arch/x86/kernel/cpu/acrn.c | 14 ++++++++++++++
2 files changed, 22 insertions(+)
create mode 100644 arch/x86/include/asm/acrn.h
diff --git a/arch/x86/include/asm/acrn.h b/arch/x86/include/asm/acrn.h
new file mode 100644
index 000000000000..ff259b69cde7
--- /dev/null
+++ b/arch/x86/include/asm/acrn.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_ACRN_H
+#define _ASM_X86_ACRN_H
+
+void acrn_setup_intr_handler(void (*handler)(void));
+void acrn_remove_intr_handler(void);
+
+#endif /* _ASM_X86_ACRN_H */
diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c
index 0b2c03943ac6..e0c181781905 100644
--- a/arch/x86/kernel/cpu/acrn.c
+++ b/arch/x86/kernel/cpu/acrn.c
@@ -10,6 +10,8 @@
*/
#include <linux/interrupt.h>
+
+#include <asm/acrn.h>
#include <asm/apic.h>
#include <asm/cpufeatures.h>
#include <asm/desc.h>
@@ -55,6 +57,18 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_acrn_hv_callback)
set_irq_regs(old_regs);
}
+void acrn_setup_intr_handler(void (*handler)(void))
+{
+ acrn_intr_handler = handler;
+}
+EXPORT_SYMBOL_GPL(acrn_setup_intr_handler);
+
+void acrn_remove_intr_handler(void)
+{
+ acrn_intr_handler = NULL;
+}
+EXPORT_SYMBOL_GPL(acrn_remove_intr_handler);
+
const __initconst struct hypervisor_x86 x86_hyper_acrn = {
.name = "ACRN",
.detect = acrn_detect,
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 03/18] x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 01/18] docs: acrn: Introduce ACRN shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 02/18] x86/acrn: Introduce acrn_{setup, remove}_intr_handler() shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces shuo.a.liu
` (14 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Yin Fengwei, Shuo Liu, Dave Hansen,
Dan Williams, Zhi Wang, Zhenyu Wang
From: Yin Fengwei <fengwei.yin@intel.com>
ACRN Hypervisor reports hypervisor features via CPUID leaf 0x40000001
which is similar to KVM. A VM can check if it's the privileged VM using
the feature bits. The Service VM is the only privileged VM by design.
Signed-off-by: Yin Fengwei <fengwei.yin@intel.com>
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Fengwei Yin <fengwei.yin@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
arch/x86/include/asm/acrn.h | 16 ++++++++++++++++
arch/x86/kernel/cpu/acrn.c | 2 +-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/acrn.h b/arch/x86/include/asm/acrn.h
index ff259b69cde7..127f20672c5d 100644
--- a/arch/x86/include/asm/acrn.h
+++ b/arch/x86/include/asm/acrn.h
@@ -2,7 +2,23 @@
#ifndef _ASM_X86_ACRN_H
#define _ASM_X86_ACRN_H
+/*
+ * This CPUID returns feature bitmaps in EAX.
+ * Guest VM uses this to detect the appropriate feature bit.
+ */
+#define ACRN_CPUID_FEATURES 0x40000001
+/* Bit 0 indicates whether guest VM is privileged */
+#define ACRN_FEATURE_PRIVILEGED_VM BIT(0)
+
void acrn_setup_intr_handler(void (*handler)(void));
void acrn_remove_intr_handler(void);
+static inline u32 acrn_cpuid_base(void)
+{
+ if (boot_cpu_has(X86_FEATURE_HYPERVISOR))
+ return hypervisor_cpuid_base("ACRNACRNACRN", 0);
+
+ return 0;
+}
+
#endif /* _ASM_X86_ACRN_H */
diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c
index e0c181781905..23f5f27b5a02 100644
--- a/arch/x86/kernel/cpu/acrn.c
+++ b/arch/x86/kernel/cpu/acrn.c
@@ -21,7 +21,7 @@
static u32 __init acrn_detect(void)
{
- return hypervisor_cpuid_base("ACRNACRNACRN", 0);
+ return acrn_cpuid_base();
}
static void __init acrn_init_platform(void)
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (2 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 03/18] x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-30 17:33 ` Borislav Petkov
2020-12-01 9:38 ` [PATCH v6 05/18] virt: acrn: Introduce ACRN HSM basic driver shuo.a.liu
` (13 subsequent siblings)
17 siblings, 1 reply; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Yakui Zhao, Nick Desaulniers,
Dave Hansen, Dan Williams, Fengwei Yin, Zhi Wang, Zhenyu Wang,
Arvind Sankar, Peter Zijlstra, Segher Boessenkool
From: Shuo Liu <shuo.a.liu@intel.com>
The Service VM communicates with the hypervisor via conventional
hypercalls. VMCALL instruction is used to make the hypercalls.
ACRN hypercall ABI:
* Hypercall number is in R8 register.
* Up to 2 parameters are in RDI and RSI registers.
* Return value is in RAX register.
Introduce the ACRN hypercall interfaces. Because GCC doesn't support R8
register as direct register constraints, use supported constraint as
input with a explicit MOV to R8 in beginning of asm.
Originally-by: Yakui Zhao <yakui.zhao@intel.com>
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Fengwei Yin <fengwei.yin@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Arvind Sankar <nivedita@alum.mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Segher Boessenkool <segher@kernel.crashing.org>
---
arch/x86/include/asm/acrn.h | 54 +++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/arch/x86/include/asm/acrn.h b/arch/x86/include/asm/acrn.h
index 127f20672c5d..e003a01b7c67 100644
--- a/arch/x86/include/asm/acrn.h
+++ b/arch/x86/include/asm/acrn.h
@@ -21,4 +21,58 @@ static inline u32 acrn_cpuid_base(void)
return 0;
}
+/*
+ * Hypercalls for ACRN
+ *
+ * - VMCALL instruction is used to implement ACRN hypercalls.
+ * - ACRN hypercall ABI:
+ * - Hypercall number is passed in R8 register.
+ * - Up to 2 arguments are passed in RDI, RSI.
+ * - Return value will be placed in RAX.
+ *
+ * Because GCC doesn't support R8 register as direct register constraints, use
+ * supported constraint as input with a explicit MOV to R8 in beginning of asm.
+ */
+static inline long acrn_hypercall0(unsigned long hcall_id)
+{
+ long result;
+
+ asm volatile("movl %1, %%r8d\n\t"
+ "vmcall\n\t"
+ : "=a" (result)
+ : "g" (hcall_id)
+ : "r8", "memory");
+
+ return result;
+}
+
+static inline long acrn_hypercall1(unsigned long hcall_id,
+ unsigned long param1)
+{
+ long result;
+
+ asm volatile("movl %1, %%r8d\n\t"
+ "vmcall\n\t"
+ : "=a" (result)
+ : "g" (hcall_id), "D" (param1)
+ : "r8", "memory");
+
+ return result;
+}
+
+static inline long acrn_hypercall2(unsigned long hcall_id,
+ unsigned long param1,
+ unsigned long param2)
+{
+ long result;
+
+ asm volatile("movl %1, %%r8d\n\t"
+ "vmcall\n\t"
+ : "=a" (result)
+ : "g" (hcall_id), "D" (param1), "S" (param2)
+ : "r8", "memory");
+
+ return result;
+}
+
#endif /* _ASM_X86_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 05/18] virt: acrn: Introduce ACRN HSM basic driver
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (3 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces shuo.a.liu
` (12 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Dave Hansen, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
ACRN Hypervisor Service Module (HSM) is a kernel module in Service VM
which communicates with ACRN userspace through ioctls and talks to ACRN
Hypervisor through hypercalls.
Add a basic HSM driver which allows Service VM userspace to communicate
with ACRN. The following patches will add more ioctls, guest VM memory
mapping caching, I/O request processing, ioeventfd and irqfd into this
module. HSM exports a char device interface (/dev/acrn_hsm) to userspace.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
MAINTAINERS | 1 +
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/acrn/Kconfig | 14 ++++++
drivers/virt/acrn/Makefile | 3 ++
drivers/virt/acrn/acrn_drv.h | 18 ++++++++
drivers/virt/acrn/hsm.c | 87 ++++++++++++++++++++++++++++++++++++
7 files changed, 126 insertions(+)
create mode 100644 drivers/virt/acrn/Kconfig
create mode 100644 drivers/virt/acrn/Makefile
create mode 100644 drivers/virt/acrn/acrn_drv.h
create mode 100644 drivers/virt/acrn/hsm.c
diff --git a/MAINTAINERS b/MAINTAINERS
index e6e5bd9ce117..46eb29ed9c4f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -442,6 +442,7 @@ L: acrn-dev@lists.projectacrn.org
S: Supported
W: https://projectacrn.org
F: Documentation/virt/acrn/
+F: drivers/virt/acrn/
AD1889 ALSA SOUND DRIVER
L: linux-parisc@vger.kernel.org
diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig
index 80c5f9c16ec1..8061e8ef449f 100644
--- a/drivers/virt/Kconfig
+++ b/drivers/virt/Kconfig
@@ -34,4 +34,6 @@ config FSL_HV_MANAGER
source "drivers/virt/vboxguest/Kconfig"
source "drivers/virt/nitro_enclaves/Kconfig"
+
+source "drivers/virt/acrn/Kconfig"
endif
diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile
index f28425ce4b39..3e272ea60cd9 100644
--- a/drivers/virt/Makefile
+++ b/drivers/virt/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o
obj-y += vboxguest/
obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/
+obj-$(CONFIG_ACRN_HSM) += acrn/
diff --git a/drivers/virt/acrn/Kconfig b/drivers/virt/acrn/Kconfig
new file mode 100644
index 000000000000..36c80378c30c
--- /dev/null
+++ b/drivers/virt/acrn/Kconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+config ACRN_HSM
+ tristate "ACRN Hypervisor Service Module"
+ depends on ACRN_GUEST
+ help
+ ACRN Hypervisor Service Module (HSM) is a kernel module which
+ communicates with ACRN userspace through ioctls and talks to
+ the ACRN Hypervisor through hypercalls. HSM will only run in
+ a privileged management VM, called Service VM, to manage User
+ VMs and do I/O emulation. Not required for simply running
+ under ACRN as a User VM.
+
+ To compile as a module, choose M, the module will be called
+ acrn. If unsure, say N.
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
new file mode 100644
index 000000000000..6920ed798aaf
--- /dev/null
+++ b/drivers/virt/acrn/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_ACRN_HSM) := acrn.o
+acrn-y := hsm.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
new file mode 100644
index 000000000000..29eedd696327
--- /dev/null
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ACRN_HSM_DRV_H
+#define __ACRN_HSM_DRV_H
+
+#include <linux/types.h>
+
+#define ACRN_INVALID_VMID (0xffffU)
+
+/**
+ * struct acrn_vm - Properties of ACRN User VM.
+ * @vmid: User VM ID
+ */
+struct acrn_vm {
+ u16 vmid;
+};
+
+#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
new file mode 100644
index 000000000000..a8dcb250649d
--- /dev/null
+++ b/drivers/virt/acrn/hsm.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN Hypervisor Service Module (HSM)
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Fengwei Yin <fengwei.yin@intel.com>
+ * Yakui Zhao <yakui.zhao@intel.com>
+ */
+
+#include <linux/miscdevice.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+
+#include <asm/acrn.h>
+#include <asm/hypervisor.h>
+
+#include "acrn_drv.h"
+
+/*
+ * When /dev/acrn_hsm is opened, a 'struct acrn_vm' object is created to
+ * represent a VM instance and continues to be associated with the opened file
+ * descriptor. All ioctl operations on this file descriptor will be targeted to
+ * the VM instance. Release of this file descriptor will destroy the object.
+ */
+static int acrn_dev_open(struct inode *inode, struct file *filp)
+{
+ struct acrn_vm *vm;
+
+ vm = kzalloc(sizeof(*vm), GFP_KERNEL);
+ if (!vm)
+ return -ENOMEM;
+
+ vm->vmid = ACRN_INVALID_VMID;
+ filp->private_data = vm;
+ return 0;
+}
+
+static int acrn_dev_release(struct inode *inode, struct file *filp)
+{
+ struct acrn_vm *vm = filp->private_data;
+
+ kfree(vm);
+ return 0;
+}
+
+static const struct file_operations acrn_fops = {
+ .owner = THIS_MODULE,
+ .open = acrn_dev_open,
+ .release = acrn_dev_release,
+};
+
+static struct miscdevice acrn_dev = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "acrn_hsm",
+ .fops = &acrn_fops,
+};
+
+static int __init hsm_init(void)
+{
+ int ret;
+
+ if (x86_hyper_type != X86_HYPER_ACRN)
+ return -ENODEV;
+
+ if (!(cpuid_eax(ACRN_CPUID_FEATURES) & ACRN_FEATURE_PRIVILEGED_VM))
+ return -EPERM;
+
+ ret = misc_register(&acrn_dev);
+ if (ret)
+ pr_err("Create misc dev failed!\n");
+
+ return ret;
+}
+
+static void __exit hsm_exit(void)
+{
+ misc_deregister(&acrn_dev);
+}
+module_init(hsm_init);
+module_exit(hsm_exit);
+
+MODULE_AUTHOR("Intel Corporation");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("ACRN Hypervisor Service Module (HSM)");
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (4 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 05/18] virt: acrn: Introduce ACRN HSM basic driver shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 10:09 ` Greg Kroah-Hartman
2020-12-01 9:38 ` [PATCH v6 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state shuo.a.liu
` (11 subsequent siblings)
17 siblings, 1 reply; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
The VM management interfaces expose several VM operations to ACRN
userspace via ioctls. For example, creating VM, starting VM, destroying
VM and so on.
The ACRN Hypervisor needs to exchange data with the ACRN userspace
during the VM operations. HSM provides VM operation ioctls to the ACRN
userspace and communicates with the ACRN Hypervisor for VM operations
via hypercalls.
HSM maintains a list of User VM. Each User VM will be bound to an
existing file descriptor of /dev/acrn_hsm. The User VM will be
destroyed when the file descriptor is closed.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
.../userspace-api/ioctl/ioctl-number.rst | 1 +
MAINTAINERS | 1 +
drivers/virt/acrn/Makefile | 2 +-
drivers/virt/acrn/acrn_drv.h | 21 ++++-
drivers/virt/acrn/hsm.c | 75 +++++++++++++++++-
drivers/virt/acrn/hypercall.h | 78 +++++++++++++++++++
drivers/virt/acrn/vm.c | 68 ++++++++++++++++
include/uapi/linux/acrn.h | 55 +++++++++++++
8 files changed, 297 insertions(+), 4 deletions(-)
create mode 100644 drivers/virt/acrn/hypercall.h
create mode 100644 drivers/virt/acrn/vm.c
create mode 100644 include/uapi/linux/acrn.h
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 55a2d9b2ce33..822df5281d72 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -319,6 +319,7 @@ Code Seq# Include File Comments
0xA0 all linux/sdp/sdp.h Industrial Device Project
<mailto:kenji@bitgate.com>
0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver
+0xA2 all uapi/linux/acrn.h ACRN hypervisor
0xA3 80-8F Port ACL in development:
<mailto:tlewis@mindspring.com>
0xA3 90-9F linux/dtlk.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 46eb29ed9c4f..5863e381115f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -443,6 +443,7 @@ S: Supported
W: https://projectacrn.org
F: Documentation/virt/acrn/
F: drivers/virt/acrn/
+F: include/uapi/linux/acrn.h
AD1889 ALSA SOUND DRIVER
L: linux-parisc@vger.kernel.org
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index 6920ed798aaf..cf8b4ed5e74e 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o
+acrn-y := hsm.o vm.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 29eedd696327..e5aba86cad8c 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -3,16 +3,35 @@
#ifndef __ACRN_HSM_DRV_H
#define __ACRN_HSM_DRV_H
+#include <linux/acrn.h>
+#include <linux/dev_printk.h>
+#include <linux/miscdevice.h>
#include <linux/types.h>
+#include "hypercall.h"
+
+extern struct miscdevice acrn_dev;
+
#define ACRN_INVALID_VMID (0xffffU)
+#define ACRN_VM_FLAG_DESTROYED 0U
/**
* struct acrn_vm - Properties of ACRN User VM.
+ * @list: Entry within global list of all VMs
* @vmid: User VM ID
+ * @vcpu_num: Number of virtual CPUs in the VM
+ * @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM flag management
+ * in HSM which is different from the &acrn_vm_creation.vm_flag.
*/
struct acrn_vm {
- u16 vmid;
+ struct list_head list;
+ u16 vmid;
+ int vcpu_num;
+ unsigned long flags;
};
+struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
+ struct acrn_vm_creation *vm_param);
+int acrn_vm_destroy(struct acrn_vm *vm);
+
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index a8dcb250649d..e618ac6aadaf 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -9,7 +9,6 @@
* Yakui Zhao <yakui.zhao@intel.com>
*/
-#include <linux/miscdevice.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -38,10 +37,81 @@ static int acrn_dev_open(struct inode *inode, struct file *filp)
return 0;
}
+/*
+ * HSM relies on hypercall layer of the ACRN hypervisor to do the
+ * sanity check against the input parameters.
+ */
+static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
+ unsigned long ioctl_param)
+{
+ struct acrn_vm *vm = filp->private_data;
+ struct acrn_vm_creation *vm_param;
+ int ret = 0;
+
+ if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
+ dev_dbg(acrn_dev.this_device,
+ "ioctl 0x%x: Invalid VM state!\n", cmd);
+ return -EINVAL;
+ }
+
+ switch (cmd) {
+ case ACRN_IOCTL_CREATE_VM:
+ vm_param = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_vm_creation));
+ if (IS_ERR(vm_param))
+ return PTR_ERR(vm_param);
+
+ vm_param->reserved0 = 0;
+ vm_param->reserved1 = 0;
+ vm = acrn_vm_create(vm, vm_param);
+ if (!vm) {
+ ret = -EINVAL;
+ kfree(vm_param);
+ break;
+ }
+
+ if (copy_to_user((void __user *)ioctl_param, vm_param,
+ sizeof(struct acrn_vm_creation))) {
+ acrn_vm_destroy(vm);
+ ret = -EFAULT;
+ }
+
+ kfree(vm_param);
+ break;
+ case ACRN_IOCTL_START_VM:
+ ret = hcall_start_vm(vm->vmid);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to start VM %u!\n", vm->vmid);
+ break;
+ case ACRN_IOCTL_PAUSE_VM:
+ ret = hcall_pause_vm(vm->vmid);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to pause VM %u!\n", vm->vmid);
+ break;
+ case ACRN_IOCTL_RESET_VM:
+ ret = hcall_reset_vm(vm->vmid);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to restart VM %u!\n", vm->vmid);
+ break;
+ case ACRN_IOCTL_DESTROY_VM:
+ ret = acrn_vm_destroy(vm);
+ break;
+ default:
+ dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
+ ret = -ENOTTY;
+ }
+
+ return ret;
+}
+
static int acrn_dev_release(struct inode *inode, struct file *filp)
{
struct acrn_vm *vm = filp->private_data;
+ acrn_vm_destroy(vm);
kfree(vm);
return 0;
}
@@ -50,9 +120,10 @@ static const struct file_operations acrn_fops = {
.owner = THIS_MODULE,
.open = acrn_dev_open,
.release = acrn_dev_release,
+ .unlocked_ioctl = acrn_dev_ioctl,
};
-static struct miscdevice acrn_dev = {
+struct miscdevice acrn_dev = {
.minor = MISC_DYNAMIC_MINOR,
.name = "acrn_hsm",
.fops = &acrn_fops,
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
new file mode 100644
index 000000000000..426b66cadb1f
--- /dev/null
+++ b/drivers/virt/acrn/hypercall.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ACRN HSM: hypercalls of ACRN Hypervisor
+ */
+#ifndef __ACRN_HSM_HYPERCALL_H
+#define __ACRN_HSM_HYPERCALL_H
+#include <asm/acrn.h>
+
+/*
+ * Hypercall IDs of the ACRN Hypervisor
+ */
+#define _HC_ID(x, y) (((x) << 24) | (y))
+
+#define HC_ID 0x80UL
+
+#define HC_ID_VM_BASE 0x10UL
+#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00)
+#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01)
+#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02)
+#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03)
+#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)
+
+/**
+ * hcall_create_vm() - Create a User VM
+ * @vminfo: Service VM GPA of info of User VM creation
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_create_vm(u64 vminfo)
+{
+ return acrn_hypercall1(HC_CREATE_VM, vminfo);
+}
+
+/**
+ * hcall_start_vm() - Start a User VM
+ * @vmid: User VM ID
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_start_vm(u64 vmid)
+{
+ return acrn_hypercall1(HC_START_VM, vmid);
+}
+
+/**
+ * hcall_pause_vm() - Pause a User VM
+ * @vmid: User VM ID
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_pause_vm(u64 vmid)
+{
+ return acrn_hypercall1(HC_PAUSE_VM, vmid);
+}
+
+/**
+ * hcall_destroy_vm() - Destroy a User VM
+ * @vmid: User VM ID
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_destroy_vm(u64 vmid)
+{
+ return acrn_hypercall1(HC_DESTROY_VM, vmid);
+}
+
+/**
+ * hcall_reset_vm() - Reset a User VM
+ * @vmid: User VM ID
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_reset_vm(u64 vmid)
+{
+ return acrn_hypercall1(HC_RESET_VM, vmid);
+}
+
+#endif /* __ACRN_HSM_HYPERCALL_H */
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
new file mode 100644
index 000000000000..3f667ac8ac1e
--- /dev/null
+++ b/drivers/virt/acrn/vm.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN_HSM: Virtual Machine management
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Jason Chen CJ <jason.cj.chen@intel.com>
+ * Yakui Zhao <yakui.zhao@intel.com>
+ */
+#include <linux/io.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+
+#include "acrn_drv.h"
+
+/* List of VMs */
+static LIST_HEAD(acrn_vm_list);
+/* To protect acrn_vm_list */
+static DEFINE_MUTEX(acrn_vm_list_lock);
+
+struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
+ struct acrn_vm_creation *vm_param)
+{
+ int ret;
+
+ ret = hcall_create_vm(virt_to_phys(vm_param));
+ if (ret < 0 || vm_param->vmid == ACRN_INVALID_VMID) {
+ dev_err(acrn_dev.this_device,
+ "Failed to create VM! Error: %d\n", ret);
+ return NULL;
+ }
+
+ vm->vmid = vm_param->vmid;
+ vm->vcpu_num = vm_param->vcpu_num;
+
+ mutex_lock(&acrn_vm_list_lock);
+ list_add(&vm->list, &acrn_vm_list);
+ mutex_unlock(&acrn_vm_list_lock);
+
+ dev_dbg(acrn_dev.this_device, "VM %u created.\n", vm->vmid);
+ return vm;
+}
+
+int acrn_vm_destroy(struct acrn_vm *vm)
+{
+ int ret;
+
+ if (vm->vmid == ACRN_INVALID_VMID ||
+ test_and_set_bit(ACRN_VM_FLAG_DESTROYED, &vm->flags))
+ return 0;
+
+ /* Remove from global VM list */
+ mutex_lock(&acrn_vm_list_lock);
+ list_del_init(&vm->list);
+ mutex_unlock(&acrn_vm_list_lock);
+
+ ret = hcall_destroy_vm(vm->vmid);
+ if (ret < 0) {
+ dev_err(acrn_dev.this_device,
+ "Failed to destroy VM %u\n", vm->vmid);
+ clear_bit(ACRN_VM_FLAG_DESTROYED, &vm->flags);
+ return ret;
+ }
+ dev_dbg(acrn_dev.this_device, "VM %u destroyed.\n", vm->vmid);
+ vm->vmid = ACRN_INVALID_VMID;
+ return 0;
+}
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
new file mode 100644
index 000000000000..e1608b8a50a2
--- /dev/null
+++ b/include/uapi/linux/acrn.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Userspace interface for /dev/acrn_hsm - ACRN Hypervisor Service Module
+ *
+ * This file can be used by applications that need to communicate with the HSM
+ * via the ioctl interface.
+ */
+
+#ifndef _UAPI_ACRN_H
+#define _UAPI_ACRN_H
+
+#include <linux/types.h>
+#include <linux/uuid.h>
+
+/**
+ * struct acrn_vm_creation - Info to create a User VM
+ * @vmid: User VM ID returned from the hypervisor
+ * @reserved0: Reserved and must be 0
+ * @vcpu_num: Number of vCPU in the VM. Return from hypervisor.
+ * @reserved1: Reserved and must be 0
+ * @uuid: UUID of the VM. Pass to hypervisor directly.
+ * @vm_flag: Flag of the VM creating. Pass to hypervisor directly.
+ * @ioreq_buf: Service VM GPA of I/O request buffer. Pass to
+ * hypervisor directly.
+ * @cpu_affinity: CPU affinity of the VM. Pass to hypervisor directly.
+ */
+struct acrn_vm_creation {
+ __u16 vmid;
+ __u16 reserved0;
+ __u16 vcpu_num;
+ __u16 reserved1;
+ guid_t uuid;
+ __u64 vm_flag;
+ __u64 ioreq_buf;
+ __u64 cpu_affinity;
+};
+
+/* The ioctl type, documented in ioctl-number.rst */
+#define ACRN_IOCTL_TYPE 0xA2
+
+/*
+ * Common IOCTL IDs definition for ACRN userspace
+ */
+#define ACRN_IOCTL_CREATE_VM \
+ _IOWR(ACRN_IOCTL_TYPE, 0x10, struct acrn_vm_creation)
+#define ACRN_IOCTL_DESTROY_VM \
+ _IO(ACRN_IOCTL_TYPE, 0x11)
+#define ACRN_IOCTL_START_VM \
+ _IO(ACRN_IOCTL_TYPE, 0x12)
+#define ACRN_IOCTL_PAUSE_VM \
+ _IO(ACRN_IOCTL_TYPE, 0x13)
+#define ACRN_IOCTL_RESET_VM \
+ _IO(ACRN_IOCTL_TYPE, 0x15)
+
+#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (5 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 08/18] virt: acrn: Introduce EPT mapping management shuo.a.liu
` (10 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
A virtual CPU of User VM has different context due to the different
registers state. ACRN userspace needs to set the virtual CPU
registers state (e.g. giving a initial registers state to a virtual
BSP of a User VM).
HSM provides an ioctl ACRN_IOCTL_SET_VCPU_REGS to do the virtual CPU
registers state setting. The ioctl passes the registers state from ACRN
userspace to the hypervisor directly.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/hsm.c | 16 ++++++++
drivers/virt/acrn/hypercall.h | 13 +++++++
include/uapi/linux/acrn.h | 69 +++++++++++++++++++++++++++++++++++
3 files changed, 98 insertions(+)
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index e618ac6aadaf..77c749fe1fc2 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -9,6 +9,7 @@
* Yakui Zhao <yakui.zhao@intel.com>
*/
+#include <linux/io.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -46,6 +47,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
{
struct acrn_vm *vm = filp->private_data;
struct acrn_vm_creation *vm_param;
+ struct acrn_vcpu_regs *cpu_regs;
int ret = 0;
if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
@@ -99,6 +101,20 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
case ACRN_IOCTL_DESTROY_VM:
ret = acrn_vm_destroy(vm);
break;
+ case ACRN_IOCTL_SET_VCPU_REGS:
+ cpu_regs = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_vcpu_regs));
+ if (IS_ERR(cpu_regs))
+ return PTR_ERR(cpu_regs);
+
+ memset(cpu_regs->reserved, 0, sizeof(cpu_regs->reserved));
+ ret = hcall_set_vcpu_regs(vm->vmid, virt_to_phys(cpu_regs));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to set regs state of VM%u!\n",
+ vm->vmid);
+ kfree(cpu_regs);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index 426b66cadb1f..f29cfae08862 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -19,6 +19,7 @@
#define HC_START_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x02)
#define HC_PAUSE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x03)
#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)
+#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)
/**
* hcall_create_vm() - Create a User VM
@@ -75,4 +76,16 @@ static inline long hcall_reset_vm(u64 vmid)
return acrn_hypercall1(HC_RESET_VM, vmid);
}
+/**
+ * hcall_set_vcpu_regs() - Set up registers of virtual CPU of a User VM
+ * @vmid: User VM ID
+ * @regs_state: Service VM GPA of registers state
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state)
+{
+ return acrn_hypercall2(HC_SET_VCPU_REGS, vmid, regs_state);
+}
+
#endif /* __ACRN_HSM_HYPERCALL_H */
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index e1608b8a50a2..d5d66b93586e 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -35,6 +35,73 @@ struct acrn_vm_creation {
__u64 cpu_affinity;
};
+struct acrn_gp_regs {
+ __le64 rax;
+ __le64 rcx;
+ __le64 rdx;
+ __le64 rbx;
+ __le64 rsp;
+ __le64 rbp;
+ __le64 rsi;
+ __le64 rdi;
+ __le64 r8;
+ __le64 r9;
+ __le64 r10;
+ __le64 r11;
+ __le64 r12;
+ __le64 r13;
+ __le64 r14;
+ __le64 r15;
+};
+
+struct acrn_descriptor_ptr {
+ __le16 limit;
+ __le64 base;
+ __le16 reserved[3];
+} __attribute__ ((__packed__));
+
+struct acrn_regs {
+ struct acrn_gp_regs gprs;
+ struct acrn_descriptor_ptr gdt;
+ struct acrn_descriptor_ptr idt;
+
+ __le64 rip;
+ __le64 cs_base;
+ __le64 cr0;
+ __le64 cr4;
+ __le64 cr3;
+ __le64 ia32_efer;
+ __le64 rflags;
+ __le64 reserved_64[4];
+
+ __le32 cs_ar;
+ __le32 cs_limit;
+ __le32 reserved_32[3];
+
+ __le16 cs_sel;
+ __le16 ss_sel;
+ __le16 ds_sel;
+ __le16 es_sel;
+ __le16 fs_sel;
+ __le16 gs_sel;
+ __le16 ldt_sel;
+ __le16 tr_sel;
+};
+
+/**
+ * struct acrn_vcpu_regs - Info of vCPU registers state
+ * @vcpu_id: vCPU ID
+ * @reserved: Reserved and must be 0
+ * @vcpu_regs: vCPU registers state
+ *
+ * This structure will be passed to hypervisor directly.
+ */
+struct acrn_vcpu_regs {
+ __u16 vcpu_id;
+ __u16 reserved[3];
+ struct acrn_regs vcpu_regs;
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -51,5 +118,7 @@ struct acrn_vm_creation {
_IO(ACRN_IOCTL_TYPE, 0x13)
#define ACRN_IOCTL_RESET_VM \
_IO(ACRN_IOCTL_TYPE, 0x15)
+#define ACRN_IOCTL_SET_VCPU_REGS \
+ _IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs)
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 08/18] virt: acrn: Introduce EPT mapping management
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (6 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 09/18] virt: acrn: Introduce I/O request management shuo.a.liu
` (9 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
The HSM provides hypervisor services to the ACRN userspace. While
launching a User VM, ACRN userspace needs to allocate memory and request
the ACRN Hypervisor to set up the EPT mapping for the VM.
A mapping cache is introduced for accelerating the translation between
the Service VM kernel virtual address and User VM physical address.
From the perspective of the hypervisor, the types of GPA of User VM can be
listed as following:
1) RAM region, which is used by User VM as system ram.
2) MMIO region, which is recognized by User VM as MMIO. MMIO region is
used to be utilized for devices emulation.
Generally, User VM RAM regions mapping is set up before VM started and
is released in the User VM destruction. MMIO regions mapping may be set
and unset dynamically during User VM running.
To achieve this, ioctls ACRN_IOCTL_SET_MEMSEG and ACRN_IOCTL_UNSET_MEMSEG
are introduced in HSM.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/Makefile | 2 +-
drivers/virt/acrn/acrn_drv.h | 96 ++++++++++-
drivers/virt/acrn/hsm.c | 15 ++
drivers/virt/acrn/hypercall.h | 14 ++
drivers/virt/acrn/mm.c | 306 ++++++++++++++++++++++++++++++++++
drivers/virt/acrn/vm.c | 4 +
include/uapi/linux/acrn.h | 49 ++++++
7 files changed, 476 insertions(+), 10 deletions(-)
create mode 100644 drivers/virt/acrn/mm.c
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index cf8b4ed5e74e..38bc44b6edcd 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o vm.o
+acrn-y := hsm.o vm.o mm.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index e5aba86cad8c..e47a45280eea 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -12,26 +12,104 @@
extern struct miscdevice acrn_dev;
+#define ACRN_MEM_MAPPING_MAX 256
+
+#define ACRN_MEM_REGION_ADD 0
+#define ACRN_MEM_REGION_DEL 2
+/**
+ * struct vm_memory_region_op - Hypervisor memory operation
+ * @type: Operation type (ACRN_MEM_REGION_*)
+ * @attr: Memory attribute (ACRN_MEM_TYPE_* | ACRN_MEM_ACCESS_*)
+ * @user_vm_pa: Physical address of User VM to be mapped.
+ * @service_vm_pa: Physical address of Service VM to be mapped.
+ * @size: Size of this region.
+ *
+ * Structure containing needed information that is provided to ACRN Hypervisor
+ * to manage the EPT mappings of a single memory region of the User VM. Several
+ * &struct vm_memory_region_op can be batched to ACRN Hypervisor, see &struct
+ * vm_memory_region_batch.
+ */
+struct vm_memory_region_op {
+ u32 type;
+ u32 attr;
+ u64 user_vm_pa;
+ u64 service_vm_pa;
+ u64 size;
+};
+
+/**
+ * struct vm_memory_region_batch - A batch of vm_memory_region_op.
+ * @vmid: A User VM ID.
+ * @reserved: Reserved.
+ * @regions_num: The number of vm_memory_region_op.
+ * @regions_gpa: Physical address of a vm_memory_region_op array.
+ *
+ * HC_VM_SET_MEMORY_REGIONS uses this structure to manage EPT mappings of
+ * multiple memory regions of a User VM. A &struct vm_memory_region_batch
+ * contains multiple &struct vm_memory_region_op for batch processing in the
+ * ACRN Hypervisor.
+ */
+struct vm_memory_region_batch {
+ u16 vmid;
+ u16 reserved[3];
+ u32 regions_num;
+ u64 regions_gpa;
+};
+
+/**
+ * struct vm_memory_mapping - Memory map between a User VM and the Service VM
+ * @pages: Pages in Service VM kernel.
+ * @npages: Number of pages.
+ * @service_vm_va: Virtual address in Service VM kernel.
+ * @user_vm_pa: Physical address in User VM.
+ * @size: Size of this memory region.
+ *
+ * HSM maintains memory mappings between a User VM GPA and the Service VM
+ * kernel VA for accelerating the User VM GPA translation.
+ */
+struct vm_memory_mapping {
+ struct page **pages;
+ int npages;
+ void *service_vm_va;
+ u64 user_vm_pa;
+ size_t size;
+};
+
#define ACRN_INVALID_VMID (0xffffU)
#define ACRN_VM_FLAG_DESTROYED 0U
/**
* struct acrn_vm - Properties of ACRN User VM.
- * @list: Entry within global list of all VMs
- * @vmid: User VM ID
- * @vcpu_num: Number of virtual CPUs in the VM
- * @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM flag management
- * in HSM which is different from the &acrn_vm_creation.vm_flag.
+ * @list: Entry within global list of all VMs.
+ * @vmid: User VM ID.
+ * @vcpu_num: Number of virtual CPUs in the VM.
+ * @flags: Flags (ACRN_VM_FLAG_*) of the VM. This is VM
+ * flag management in HSM which is different
+ * from the &acrn_vm_creation.vm_flag.
+ * @regions_mapping_lock: Lock to protect &acrn_vm.regions_mapping and
+ * &acrn_vm.regions_mapping_count.
+ * @regions_mapping: Memory mappings of this VM.
+ * @regions_mapping_count: Number of memory mapping of this VM.
*/
struct acrn_vm {
- struct list_head list;
- u16 vmid;
- int vcpu_num;
- unsigned long flags;
+ struct list_head list;
+ u16 vmid;
+ int vcpu_num;
+ unsigned long flags;
+ struct mutex regions_mapping_lock;
+ struct vm_memory_mapping regions_mapping[ACRN_MEM_MAPPING_MAX];
+ int regions_mapping_count;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
struct acrn_vm_creation *vm_param);
int acrn_vm_destroy(struct acrn_vm *vm);
+int acrn_mm_region_add(struct acrn_vm *vm, u64 user_gpa, u64 service_gpa,
+ u64 size, u32 mem_type, u32 mem_access_right);
+int acrn_mm_region_del(struct acrn_vm *vm, u64 user_gpa, u64 size);
+int acrn_vm_memseg_map(struct acrn_vm *vm, struct acrn_vm_memmap *memmap);
+int acrn_vm_memseg_unmap(struct acrn_vm *vm, struct acrn_vm_memmap *memmap);
+int acrn_vm_ram_map(struct acrn_vm *vm, struct acrn_vm_memmap *memmap);
+void acrn_vm_all_ram_unmap(struct acrn_vm *vm);
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index 77c749fe1fc2..3d76ab27cf89 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -48,6 +48,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_vm *vm = filp->private_data;
struct acrn_vm_creation *vm_param;
struct acrn_vcpu_regs *cpu_regs;
+ struct acrn_vm_memmap memmap;
int ret = 0;
if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
@@ -115,6 +116,20 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
vm->vmid);
kfree(cpu_regs);
break;
+ case ACRN_IOCTL_SET_MEMSEG:
+ if (copy_from_user(&memmap, (void __user *)ioctl_param,
+ sizeof(memmap)))
+ return -EFAULT;
+
+ ret = acrn_vm_memseg_map(vm, &memmap);
+ break;
+ case ACRN_IOCTL_UNSET_MEMSEG:
+ if (copy_from_user(&memmap, (void __user *)ioctl_param,
+ sizeof(memmap)))
+ return -EFAULT;
+
+ ret = acrn_vm_memseg_unmap(vm, &memmap);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index f29cfae08862..a1a70a071713 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -21,6 +21,9 @@
#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)
#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)
+#define HC_ID_MEM_BASE 0x40UL
+#define HC_VM_SET_MEMORY_REGIONS _HC_ID(HC_ID, HC_ID_MEM_BASE + 0x02)
+
/**
* hcall_create_vm() - Create a User VM
* @vminfo: Service VM GPA of info of User VM creation
@@ -88,4 +91,15 @@ static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state)
return acrn_hypercall2(HC_SET_VCPU_REGS, vmid, regs_state);
}
+/**
+ * hcall_set_memory_regions() - Inform the hypervisor to set up EPT mappings
+ * @regions_pa: Service VM GPA of &struct vm_memory_region_batch
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_set_memory_regions(u64 regions_pa)
+{
+ return acrn_hypercall1(HC_VM_SET_MEMORY_REGIONS, regions_pa);
+}
+
#endif /* __ACRN_HSM_HYPERCALL_H */
diff --git a/drivers/virt/acrn/mm.c b/drivers/virt/acrn/mm.c
new file mode 100644
index 000000000000..c4f2e15c8a2b
--- /dev/null
+++ b/drivers/virt/acrn/mm.c
@@ -0,0 +1,306 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN: Memory mapping management
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Fei Li <lei1.li@intel.com>
+ * Shuo Liu <shuo.a.liu@intel.com>
+ */
+
+#include <linux/io.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+
+#include "acrn_drv.h"
+
+static int modify_region(struct acrn_vm *vm, struct vm_memory_region_op *region)
+{
+ struct vm_memory_region_batch *regions;
+ int ret;
+
+ regions = kzalloc(sizeof(*regions), GFP_KERNEL);
+ if (!regions)
+ return -ENOMEM;
+
+ regions->vmid = vm->vmid;
+ regions->regions_num = 1;
+ regions->regions_gpa = virt_to_phys(region);
+
+ ret = hcall_set_memory_regions(virt_to_phys(regions));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to set memory region for VM[%u]!\n", vm->vmid);
+
+ kfree(regions);
+ return ret;
+}
+
+/**
+ * acrn_mm_region_add() - Set up the EPT mapping of a memory region.
+ * @vm: User VM.
+ * @user_gpa: A GPA of User VM.
+ * @service_gpa: A GPA of Service VM.
+ * @size: Size of the region.
+ * @mem_type: Combination of ACRN_MEM_TYPE_*.
+ * @mem_access_right: Combination of ACRN_MEM_ACCESS_*.
+ *
+ * Return: 0 on success, <0 on error.
+ */
+int acrn_mm_region_add(struct acrn_vm *vm, u64 user_gpa, u64 service_gpa,
+ u64 size, u32 mem_type, u32 mem_access_right)
+{
+ struct vm_memory_region_op *region;
+ int ret = 0;
+
+ region = kzalloc(sizeof(*region), GFP_KERNEL);
+ if (!region)
+ return -ENOMEM;
+
+ region->type = ACRN_MEM_REGION_ADD;
+ region->user_vm_pa = user_gpa;
+ region->service_vm_pa = service_gpa;
+ region->size = size;
+ region->attr = ((mem_type & ACRN_MEM_TYPE_MASK) |
+ (mem_access_right & ACRN_MEM_ACCESS_RIGHT_MASK));
+ ret = modify_region(vm, region);
+
+ dev_dbg(acrn_dev.this_device,
+ "%s: user-GPA[%pK] service-GPA[%pK] size[0x%llx].\n",
+ __func__, (void *)user_gpa, (void *)service_gpa, size);
+ kfree(region);
+ return ret;
+}
+
+/**
+ * acrn_mm_region_del() - Del the EPT mapping of a memory region.
+ * @vm: User VM.
+ * @user_gpa: A GPA of the User VM.
+ * @size: Size of the region.
+ *
+ * Return: 0 on success, <0 for error.
+ */
+int acrn_mm_region_del(struct acrn_vm *vm, u64 user_gpa, u64 size)
+{
+ struct vm_memory_region_op *region;
+ int ret = 0;
+
+ region = kzalloc(sizeof(*region), GFP_KERNEL);
+ if (!region)
+ return -ENOMEM;
+
+ region->type = ACRN_MEM_REGION_DEL;
+ region->user_vm_pa = user_gpa;
+ region->service_vm_pa = 0UL;
+ region->size = size;
+ region->attr = 0U;
+
+ ret = modify_region(vm, region);
+
+ dev_dbg(acrn_dev.this_device, "%s: user-GPA[%pK] size[0x%llx].\n",
+ __func__, (void *)user_gpa, size);
+ kfree(region);
+ return ret;
+}
+
+int acrn_vm_memseg_map(struct acrn_vm *vm, struct acrn_vm_memmap *memmap)
+{
+ int ret;
+
+ if (memmap->type == ACRN_MEMMAP_RAM)
+ return acrn_vm_ram_map(vm, memmap);
+
+ if (memmap->type != ACRN_MEMMAP_MMIO) {
+ dev_dbg(acrn_dev.this_device,
+ "Invalid memmap type: %u\n", memmap->type);
+ return -EINVAL;
+ }
+
+ ret = acrn_mm_region_add(vm, memmap->user_vm_pa,
+ memmap->service_vm_pa, memmap->len,
+ ACRN_MEM_TYPE_UC, memmap->attr);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Add memory region failed, VM[%u]!\n", vm->vmid);
+
+ return ret;
+}
+
+int acrn_vm_memseg_unmap(struct acrn_vm *vm, struct acrn_vm_memmap *memmap)
+{
+ int ret;
+
+ if (memmap->type != ACRN_MEMMAP_MMIO) {
+ dev_dbg(acrn_dev.this_device,
+ "Invalid memmap type: %u\n", memmap->type);
+ return -EINVAL;
+ }
+
+ ret = acrn_mm_region_del(vm, memmap->user_vm_pa, memmap->len);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Del memory region failed, VM[%u]!\n", vm->vmid);
+
+ return ret;
+}
+
+/**
+ * acrn_vm_ram_map() - Create a RAM EPT mapping of User VM.
+ * @vm: The User VM pointer
+ * @memmap: Info of the EPT mapping
+ *
+ * Return: 0 on success, <0 for error.
+ */
+int acrn_vm_ram_map(struct acrn_vm *vm, struct acrn_vm_memmap *memmap)
+{
+ struct vm_memory_region_batch *regions_info;
+ int nr_pages, i = 0, order, nr_regions = 0;
+ struct vm_memory_mapping *region_mapping;
+ struct vm_memory_region_op *vm_region;
+ struct page **pages = NULL, *page;
+ void *remap_vaddr;
+ int ret, pinned;
+ u64 user_vm_pa;
+
+ if (!vm || !memmap)
+ return -EINVAL;
+
+ /* Get the page number of the map region */
+ nr_pages = memmap->len >> PAGE_SHIFT;
+ pages = vzalloc(nr_pages * sizeof(struct page *));
+ if (!pages)
+ return -ENOMEM;
+
+ /* Lock the pages of user memory map region */
+ pinned = pin_user_pages_fast(memmap->vma_base,
+ nr_pages, FOLL_WRITE | FOLL_LONGTERM,
+ pages);
+ if (pinned < 0) {
+ ret = pinned;
+ goto free_pages;
+ } else if (pinned != nr_pages) {
+ ret = -EFAULT;
+ goto put_pages;
+ }
+
+ /* Create a kernel map for the map region */
+ remap_vaddr = vmap(pages, nr_pages, VM_MAP, PAGE_KERNEL);
+ if (!remap_vaddr) {
+ ret = -ENOMEM;
+ goto put_pages;
+ }
+
+ /* Record Service VM va <-> User VM pa mapping */
+ mutex_lock(&vm->regions_mapping_lock);
+ region_mapping = &vm->regions_mapping[vm->regions_mapping_count];
+ if (vm->regions_mapping_count < ACRN_MEM_MAPPING_MAX) {
+ region_mapping->pages = pages;
+ region_mapping->npages = nr_pages;
+ region_mapping->size = memmap->len;
+ region_mapping->service_vm_va = remap_vaddr;
+ region_mapping->user_vm_pa = memmap->user_vm_pa;
+ vm->regions_mapping_count++;
+ } else {
+ dev_warn(acrn_dev.this_device,
+ "Run out of memory mapping slots!\n");
+ ret = -ENOMEM;
+ mutex_unlock(&vm->regions_mapping_lock);
+ goto unmap_no_count;
+ }
+ mutex_unlock(&vm->regions_mapping_lock);
+
+ /* Calculate count of vm_memory_region_op */
+ while (i < nr_pages) {
+ page = pages[i];
+ VM_BUG_ON_PAGE(PageTail(page), page);
+ order = compound_order(page);
+ nr_regions++;
+ i += 1 << order;
+ }
+
+ /* Prepare the vm_memory_region_batch */
+ regions_info = kzalloc(sizeof(*regions_info) +
+ sizeof(*vm_region) * nr_regions,
+ GFP_KERNEL);
+ if (!regions_info) {
+ ret = -ENOMEM;
+ goto unmap_kernel_map;
+ }
+
+ /* Fill each vm_memory_region_op */
+ vm_region = (struct vm_memory_region_op *)(regions_info + 1);
+ regions_info->vmid = vm->vmid;
+ regions_info->regions_num = nr_regions;
+ regions_info->regions_gpa = virt_to_phys(vm_region);
+ user_vm_pa = memmap->user_vm_pa;
+ i = 0;
+ while (i < nr_pages) {
+ u32 region_size;
+
+ page = pages[i];
+ VM_BUG_ON_PAGE(PageTail(page), page);
+ order = compound_order(page);
+ region_size = PAGE_SIZE << order;
+ vm_region->type = ACRN_MEM_REGION_ADD;
+ vm_region->user_vm_pa = user_vm_pa;
+ vm_region->service_vm_pa = page_to_phys(page);
+ vm_region->size = region_size;
+ vm_region->attr = (ACRN_MEM_TYPE_WB & ACRN_MEM_TYPE_MASK) |
+ (memmap->attr & ACRN_MEM_ACCESS_RIGHT_MASK);
+
+ vm_region++;
+ user_vm_pa += region_size;
+ i += 1 << order;
+ }
+
+ /* Inform the ACRN Hypervisor to set up EPT mappings */
+ ret = hcall_set_memory_regions(virt_to_phys(regions_info));
+ if (ret < 0) {
+ dev_dbg(acrn_dev.this_device,
+ "Failed to set regions, VM[%u]!\n", vm->vmid);
+ goto unset_region;
+ }
+ kfree(regions_info);
+
+ dev_dbg(acrn_dev.this_device,
+ "%s: VM[%u] service-GVA[%pK] user-GPA[%pK] size[0x%llx]\n",
+ __func__, vm->vmid,
+ remap_vaddr, (void *)memmap->user_vm_pa, memmap->len);
+ return ret;
+
+unset_region:
+ kfree(regions_info);
+unmap_kernel_map:
+ mutex_lock(&vm->regions_mapping_lock);
+ vm->regions_mapping_count--;
+ mutex_unlock(&vm->regions_mapping_lock);
+unmap_no_count:
+ vunmap(remap_vaddr);
+put_pages:
+ for (i = 0; i < pinned; i++)
+ unpin_user_page(pages[i]);
+free_pages:
+ vfree(pages);
+ return ret;
+}
+
+/**
+ * acrn_vm_all_ram_unmap() - Destroy a RAM EPT mapping of User VM.
+ * @vm: The User VM
+ */
+void acrn_vm_all_ram_unmap(struct acrn_vm *vm)
+{
+ struct vm_memory_mapping *region_mapping;
+ int i, j;
+
+ mutex_lock(&vm->regions_mapping_lock);
+ for (i = 0; i < vm->regions_mapping_count; i++) {
+ region_mapping = &vm->regions_mapping[i];
+ vunmap(region_mapping->service_vm_va);
+ for (j = 0; j < region_mapping->npages; j++)
+ unpin_user_page(region_mapping->pages[j]);
+ vfree(region_mapping->pages);
+ }
+ mutex_unlock(&vm->regions_mapping_lock);
+}
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
index 3f667ac8ac1e..ff5df7acb551 100644
--- a/drivers/virt/acrn/vm.c
+++ b/drivers/virt/acrn/vm.c
@@ -31,6 +31,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
return NULL;
}
+ mutex_init(&vm->regions_mapping_lock);
vm->vmid = vm_param->vmid;
vm->vcpu_num = vm_param->vcpu_num;
@@ -62,6 +63,9 @@ int acrn_vm_destroy(struct acrn_vm *vm)
clear_bit(ACRN_VM_FLAG_DESTROYED, &vm->flags);
return ret;
}
+
+ acrn_vm_all_ram_unmap(vm);
+
dev_dbg(acrn_dev.this_device, "VM %u destroyed.\n", vm->vmid);
vm->vmid = ACRN_INVALID_VMID;
return 0;
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index d5d66b93586e..c260d0e9caea 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -102,6 +102,50 @@ struct acrn_vcpu_regs {
struct acrn_regs vcpu_regs;
};
+#define ACRN_MEM_ACCESS_RIGHT_MASK 0x00000007U
+#define ACRN_MEM_ACCESS_READ 0x00000001U
+#define ACRN_MEM_ACCESS_WRITE 0x00000002U
+#define ACRN_MEM_ACCESS_EXEC 0x00000004U
+#define ACRN_MEM_ACCESS_RWX (ACRN_MEM_ACCESS_READ | \
+ ACRN_MEM_ACCESS_WRITE | \
+ ACRN_MEM_ACCESS_EXEC)
+
+#define ACRN_MEM_TYPE_MASK 0x000007C0U
+#define ACRN_MEM_TYPE_WB 0x00000040U
+#define ACRN_MEM_TYPE_WT 0x00000080U
+#define ACRN_MEM_TYPE_UC 0x00000100U
+#define ACRN_MEM_TYPE_WC 0x00000200U
+#define ACRN_MEM_TYPE_WP 0x00000400U
+
+/* Memory mapping types */
+#define ACRN_MEMMAP_RAM 0
+#define ACRN_MEMMAP_MMIO 1
+
+/**
+ * struct acrn_vm_memmap - A EPT memory mapping info for a User VM.
+ * @type: Type of the memory mapping (ACRM_MEMMAP_*).
+ * Pass to hypervisor directly.
+ * @attr: Attribute of the memory mapping.
+ * Pass to hypervisor directly.
+ * @user_vm_pa: Physical address of User VM.
+ * Pass to hypervisor directly.
+ * @service_vm_pa: Physical address of Service VM.
+ * Pass to hypervisor directly.
+ * @vma_base: VMA address of Service VM. Pass to hypervisor directly.
+ * @len: Length of the memory mapping.
+ * Pass to hypervisor directly.
+ */
+struct acrn_vm_memmap {
+ __u32 type;
+ __u32 attr;
+ __u64 user_vm_pa;
+ union {
+ __u64 service_vm_pa;
+ __u64 vma_base;
+ };
+ __u64 len;
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -121,4 +165,9 @@ struct acrn_vcpu_regs {
#define ACRN_IOCTL_SET_VCPU_REGS \
_IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs)
+#define ACRN_IOCTL_SET_MEMSEG \
+ _IOW(ACRN_IOCTL_TYPE, 0x41, struct acrn_vm_memmap)
+#define ACRN_IOCTL_UNSET_MEMSEG \
+ _IOW(ACRN_IOCTL_TYPE, 0x42, struct acrn_vm_memmap)
+
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 09/18] virt: acrn: Introduce I/O request management
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (7 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 08/18] virt: acrn: Introduce EPT mapping management shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 10/18] virt: acrn: Introduce PCI configuration space PIO accesses combiner shuo.a.liu
` (8 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
An I/O request of a User VM, which is constructed by the hypervisor, is
distributed by the ACRN Hypervisor Service Module to an I/O client
corresponding to the address range of the I/O request.
For each User VM, there is a shared 4-KByte memory region used for I/O
requests communication between the hypervisor and Service VM. An I/O
request is a 256-byte structure buffer, which is 'struct
acrn_io_request', that is filled by an I/O handler of the hypervisor
when a trapped I/O access happens in a User VM. ACRN userspace in the
Service VM first allocates a 4-KByte page and passes the GPA (Guest
Physical Address) of the buffer to the hypervisor. The buffer is used as
an array of 16 I/O request slots with each I/O request slot being 256
bytes. This array is indexed by vCPU ID.
An I/O client, which is 'struct acrn_ioreq_client', is responsible for
handling User VM I/O requests whose accessed GPA falls in a certain
range. Multiple I/O clients can be associated with each User VM. There
is a special client associated with each User VM, called the default
client, that handles all I/O requests that do not fit into the range of
any other I/O clients. The ACRN userspace acts as the default client for
each User VM.
The state transitions of a ACRN I/O request are as follows.
FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...
FREE: this I/O request slot is empty
PENDING: a valid I/O request is pending in this slot
PROCESSING: the I/O request is being processed
COMPLETE: the I/O request has been processed
An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM
and ACRN userspace are in charge of processing the others.
The processing flow of I/O requests are listed as following:
a) The I/O handler of the hypervisor will fill an I/O request with
PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to
the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to
different registered clients based on the address of the I/O accesses,
updates their state to PROCESSING, and notifies the corresponding
client to handle.
e) The notified client handles the assigned I/O requests.
f) The HSM updates I/O requests states to COMPLETE and notifies the
hypervisor of the completion via hypercalls.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/Makefile | 2 +-
drivers/virt/acrn/acrn_drv.h | 82 ++++++
drivers/virt/acrn/hsm.c | 34 ++-
drivers/virt/acrn/hypercall.h | 28 ++
drivers/virt/acrn/ioreq.c | 509 ++++++++++++++++++++++++++++++++++
drivers/virt/acrn/vm.c | 27 +-
include/uapi/linux/acrn.h | 134 +++++++++
7 files changed, 806 insertions(+), 10 deletions(-)
create mode 100644 drivers/virt/acrn/ioreq.c
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index 38bc44b6edcd..21721cbf6a80 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o vm.o mm.o
+acrn-y := hsm.o vm.o mm.o ioreq.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index e47a45280eea..7e89e4944d0f 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -12,10 +12,15 @@
extern struct miscdevice acrn_dev;
+#define ACRN_NAME_LEN 16
#define ACRN_MEM_MAPPING_MAX 256
#define ACRN_MEM_REGION_ADD 0
#define ACRN_MEM_REGION_DEL 2
+
+struct acrn_vm;
+struct acrn_ioreq_client;
+
/**
* struct vm_memory_region_op - Hypervisor memory operation
* @type: Operation type (ACRN_MEM_REGION_*)
@@ -75,9 +80,63 @@ struct vm_memory_mapping {
size_t size;
};
+/**
+ * struct acrn_ioreq_buffer - Data for setting the ioreq buffer of User VM
+ * @ioreq_buf: The GPA of the IO request shared buffer of a VM
+ *
+ * The parameter for the HC_SET_IOREQ_BUFFER hypercall used to set up
+ * the shared I/O request buffer between Service VM and ACRN hypervisor.
+ */
+struct acrn_ioreq_buffer {
+ u64 ioreq_buf;
+};
+
+struct acrn_ioreq_range {
+ struct list_head list;
+ u32 type;
+ u64 start;
+ u64 end;
+};
+
+#define ACRN_IOREQ_CLIENT_DESTROYING 0U
+typedef int (*ioreq_handler_t)(struct acrn_ioreq_client *client,
+ struct acrn_io_request *req);
+/**
+ * struct acrn_ioreq_client - Structure of I/O client.
+ * @name: Client name
+ * @vm: The VM that the client belongs to
+ * @list: List node for this acrn_ioreq_client
+ * @is_default: If this client is the default one
+ * @flags: Flags (ACRN_IOREQ_CLIENT_*)
+ * @range_list: I/O ranges
+ * @range_lock: Lock to protect range_list
+ * @ioreqs_map: The pending I/O requests bitmap.
+ * @handler: I/O requests handler of this client
+ * @thread: The thread which executes the handler
+ * @wq: The wait queue for the handler thread parking
+ * @priv: Data for the thread
+ */
+struct acrn_ioreq_client {
+ char name[ACRN_NAME_LEN];
+ struct acrn_vm *vm;
+ struct list_head list;
+ bool is_default;
+ unsigned long flags;
+ struct list_head range_list;
+ rwlock_t range_lock;
+ DECLARE_BITMAP(ioreqs_map, ACRN_IO_REQUEST_MAX);
+ ioreq_handler_t handler;
+ struct task_struct *thread;
+ wait_queue_head_t wq;
+ void *priv;
+};
+
#define ACRN_INVALID_VMID (0xffffU)
#define ACRN_VM_FLAG_DESTROYED 0U
+#define ACRN_VM_FLAG_CLEARING_IOREQ 1U
+extern struct list_head acrn_vm_list;
+extern rwlock_t acrn_vm_list_lock;
/**
* struct acrn_vm - Properties of ACRN User VM.
* @list: Entry within global list of all VMs.
@@ -90,6 +149,11 @@ struct vm_memory_mapping {
* &acrn_vm.regions_mapping_count.
* @regions_mapping: Memory mappings of this VM.
* @regions_mapping_count: Number of memory mapping of this VM.
+ * @ioreq_clients_lock: Lock to protect ioreq_clients and default_client
+ * @ioreq_clients: The I/O request clients list of this VM
+ * @default_client: The default I/O request client
+ * @ioreq_buf: I/O request shared buffer
+ * @ioreq_page: The page of the I/O request shared buffer
*/
struct acrn_vm {
struct list_head list;
@@ -99,6 +163,11 @@ struct acrn_vm {
struct mutex regions_mapping_lock;
struct vm_memory_mapping regions_mapping[ACRN_MEM_MAPPING_MAX];
int regions_mapping_count;
+ spinlock_t ioreq_clients_lock;
+ struct list_head ioreq_clients;
+ struct acrn_ioreq_client *default_client;
+ struct acrn_io_request_buffer *ioreq_buf;
+ struct page *ioreq_page;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -112,4 +181,17 @@ int acrn_vm_memseg_unmap(struct acrn_vm *vm, struct acrn_vm_memmap *memmap);
int acrn_vm_ram_map(struct acrn_vm *vm, struct acrn_vm_memmap *memmap);
void acrn_vm_all_ram_unmap(struct acrn_vm *vm);
+int acrn_ioreq_init(struct acrn_vm *vm, u64 buf_vma);
+void acrn_ioreq_deinit(struct acrn_vm *vm);
+void acrn_ioreq_intr_setup(void);
+void acrn_ioreq_intr_remove(void);
+void acrn_ioreq_request_clear(struct acrn_vm *vm);
+int acrn_ioreq_client_wait(struct acrn_ioreq_client *client);
+int acrn_ioreq_request_default_complete(struct acrn_vm *vm, u16 vcpu);
+struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm,
+ ioreq_handler_t handler,
+ void *data, bool is_default,
+ const char *name);
+void acrn_ioreq_client_destroy(struct acrn_ioreq_client *client);
+
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index 3d76ab27cf89..ea86be0ab5e4 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -48,6 +48,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_vm *vm = filp->private_data;
struct acrn_vm_creation *vm_param;
struct acrn_vcpu_regs *cpu_regs;
+ struct acrn_ioreq_notify notify;
struct acrn_vm_memmap memmap;
int ret = 0;
@@ -130,6 +131,31 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
ret = acrn_vm_memseg_unmap(vm, &memmap);
break;
+ case ACRN_IOCTL_CREATE_IOREQ_CLIENT:
+ if (vm->default_client)
+ return -EEXIST;
+ if (!acrn_ioreq_client_create(vm, NULL, NULL, true, "acrndm"))
+ ret = -EINVAL;
+ break;
+ case ACRN_IOCTL_DESTROY_IOREQ_CLIENT:
+ if (vm->default_client)
+ acrn_ioreq_client_destroy(vm->default_client);
+ break;
+ case ACRN_IOCTL_ATTACH_IOREQ_CLIENT:
+ if (vm->default_client)
+ ret = acrn_ioreq_client_wait(vm->default_client);
+ else
+ ret = -ENODEV;
+ break;
+ case ACRN_IOCTL_NOTIFY_REQUEST_FINISH:
+ if (copy_from_user(¬ify, (void __user *)ioctl_param,
+ sizeof(struct acrn_ioreq_notify)))
+ return -EFAULT;
+ ret = acrn_ioreq_request_default_complete(vm, notify.vcpu);
+ break;
+ case ACRN_IOCTL_CLEAR_VM_IOREQ:
+ acrn_ioreq_request_clear(vm);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
@@ -171,14 +197,18 @@ static int __init hsm_init(void)
return -EPERM;
ret = misc_register(&acrn_dev);
- if (ret)
+ if (ret) {
pr_err("Create misc dev failed!\n");
+ return ret;
+ }
- return ret;
+ acrn_ioreq_intr_setup();
+ return 0;
}
static void __exit hsm_exit(void)
{
+ acrn_ioreq_intr_remove();
misc_deregister(&acrn_dev);
}
module_init(hsm_init);
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index a1a70a071713..5eba29e3ed38 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -21,6 +21,10 @@
#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)
#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)
+#define HC_ID_IOREQ_BASE 0x30UL
+#define HC_SET_IOREQ_BUFFER _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x00)
+#define HC_NOTIFY_REQUEST_FINISH _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x01)
+
#define HC_ID_MEM_BASE 0x40UL
#define HC_VM_SET_MEMORY_REGIONS _HC_ID(HC_ID, HC_ID_MEM_BASE + 0x02)
@@ -91,6 +95,30 @@ static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state)
return acrn_hypercall2(HC_SET_VCPU_REGS, vmid, regs_state);
}
+/**
+ * hcall_set_ioreq_buffer() - Set up the shared buffer for I/O Requests.
+ * @vmid: User VM ID
+ * @buffer: Service VM GPA of the shared buffer
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_set_ioreq_buffer(u64 vmid, u64 buffer)
+{
+ return acrn_hypercall2(HC_SET_IOREQ_BUFFER, vmid, buffer);
+}
+
+/**
+ * hcall_notify_req_finish() - Notify ACRN Hypervisor of I/O request completion.
+ * @vmid: User VM ID
+ * @vcpu: The vCPU which initiated the I/O request
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_notify_req_finish(u64 vmid, u64 vcpu)
+{
+ return acrn_hypercall2(HC_NOTIFY_REQUEST_FINISH, vmid, vcpu);
+}
+
/**
* hcall_set_memory_regions() - Inform the hypervisor to set up EPT mappings
* @regions_pa: Service VM GPA of &struct vm_memory_region_batch
diff --git a/drivers/virt/acrn/ioreq.c b/drivers/virt/acrn/ioreq.c
new file mode 100644
index 000000000000..ed22b1c4775d
--- /dev/null
+++ b/drivers/virt/acrn/ioreq.c
@@ -0,0 +1,509 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN_HSM: Handle I/O requests
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Jason Chen CJ <jason.cj.chen@intel.com>
+ * Fengwei Yin <fengwei.yin@intel.com>
+ */
+
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/kthread.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+
+#include <asm/acrn.h>
+
+#include "acrn_drv.h"
+
+static void ioreq_pause(void);
+static void ioreq_resume(void);
+
+static struct tasklet_struct ioreq_tasklet;
+
+static inline bool has_pending_request(struct acrn_ioreq_client *client)
+{
+ return !bitmap_empty(client->ioreqs_map, ACRN_IO_REQUEST_MAX);
+}
+
+static inline bool is_destroying(struct acrn_ioreq_client *client)
+{
+ return test_bit(ACRN_IOREQ_CLIENT_DESTROYING, &client->flags);
+}
+
+static int ioreq_complete_request(struct acrn_vm *vm, u16 vcpu,
+ struct acrn_io_request *acrn_req)
+{
+ bool polling_mode;
+ int ret = 0;
+
+ polling_mode = acrn_req->completion_polling;
+ /* Add barrier() to make sure the writes are done before completion */
+ smp_store_release(&acrn_req->processed, ACRN_IOREQ_STATE_COMPLETE);
+
+ /*
+ * To fulfill the requirement of real-time in several industry
+ * scenarios, like automotive, ACRN can run under the partition mode,
+ * in which User VMs and Service VM are bound to dedicated CPU cores.
+ * Polling mode of handling the I/O request is introduced to achieve a
+ * faster I/O request handling. In polling mode, the hypervisor polls
+ * I/O request's completion. Once an I/O request is marked as
+ * ACRN_IOREQ_STATE_COMPLETE, hypervisor resumes from the polling point
+ * to continue the I/O request flow. Thus, the completion notification
+ * from HSM of I/O request is not needed. Please note,
+ * completion_polling needs to be read before the I/O request being
+ * marked as ACRN_IOREQ_STATE_COMPLETE to avoid racing with the
+ * hypervisor.
+ */
+ if (!polling_mode) {
+ ret = hcall_notify_req_finish(vm->vmid, vcpu);
+ if (ret < 0)
+ dev_err(acrn_dev.this_device,
+ "Notify I/O request finished failed!\n");
+ }
+
+ return ret;
+}
+
+static int acrn_ioreq_complete_request(struct acrn_ioreq_client *client,
+ u16 vcpu,
+ struct acrn_io_request *acrn_req)
+{
+ int ret;
+
+ if (vcpu >= client->vm->vcpu_num)
+ return -EINVAL;
+
+ clear_bit(vcpu, client->ioreqs_map);
+ if (!acrn_req) {
+ acrn_req = (struct acrn_io_request *)client->vm->ioreq_buf;
+ acrn_req += vcpu;
+ }
+
+ ret = ioreq_complete_request(client->vm, vcpu, acrn_req);
+
+ return ret;
+}
+
+int acrn_ioreq_request_default_complete(struct acrn_vm *vm, u16 vcpu)
+{
+ int ret = 0;
+
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ if (vm->default_client)
+ ret = acrn_ioreq_complete_request(vm->default_client,
+ vcpu, NULL);
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+
+ return ret;
+}
+
+/*
+ * ioreq_task() is the execution entity of handler thread of an I/O client.
+ * The handler callback of the I/O client is called within the handler thread.
+ */
+static int ioreq_task(void *data)
+{
+ struct acrn_ioreq_client *client = data;
+ struct acrn_io_request *req;
+ unsigned long *ioreqs_map;
+ int vcpu, ret;
+
+ /*
+ * Lockless access to ioreqs_map is safe, because
+ * 1) set_bit() and clear_bit() are atomic operations.
+ * 2) I/O requests arrives serialized. The access flow of ioreqs_map is:
+ * set_bit() - in tasklet
+ * Handler callback handles corresponding I/O request
+ * clear_bit() - in handler thread (include ACRN userspace)
+ * Mark corresponding I/O request completed
+ * Loop again if a new I/O request occurs
+ */
+ ioreqs_map = client->ioreqs_map;
+ while (!kthread_should_stop()) {
+ acrn_ioreq_client_wait(client);
+ while (has_pending_request(client)) {
+ vcpu = find_first_bit(ioreqs_map, client->vm->vcpu_num);
+ req = client->vm->ioreq_buf->req_slot + vcpu;
+ ret = client->handler(client, req);
+ if (ret < 0) {
+ dev_err(acrn_dev.this_device,
+ "IO handle failure: %d\n", ret);
+ break;
+ }
+ acrn_ioreq_complete_request(client, vcpu, req);
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * For the non-default I/O clients, give them chance to complete the current
+ * I/O requests if there are any. For the default I/O client, it is safe to
+ * clear all pending I/O requests because the clearing request is from ACRN
+ * userspace.
+ */
+void acrn_ioreq_request_clear(struct acrn_vm *vm)
+{
+ struct acrn_ioreq_client *client;
+ bool has_pending = false;
+ unsigned long vcpu;
+ int retry = 10;
+
+ /*
+ * IO requests of this VM will be completed directly in
+ * acrn_ioreq_dispatch if ACRN_VM_FLAG_CLEARING_IOREQ flag is set.
+ */
+ set_bit(ACRN_VM_FLAG_CLEARING_IOREQ, &vm->flags);
+
+ /*
+ * acrn_ioreq_request_clear is only called in VM reset case. Simply
+ * wait 100ms in total for the IO requests' completion.
+ */
+ do {
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ list_for_each_entry(client, &vm->ioreq_clients, list) {
+ has_pending = has_pending_request(client);
+ if (has_pending)
+ break;
+ }
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+
+ if (has_pending)
+ schedule_timeout_interruptible(HZ / 100);
+ } while (has_pending && --retry > 0);
+ if (retry == 0)
+ dev_warn(acrn_dev.this_device,
+ "%s cannot flush pending request!\n", client->name);
+
+ /* Clear all ioreqs belonging to the default client */
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ client = vm->default_client;
+ if (client) {
+ vcpu = find_next_bit(client->ioreqs_map,
+ ACRN_IO_REQUEST_MAX, 0);
+ while (vcpu < ACRN_IO_REQUEST_MAX) {
+ acrn_ioreq_complete_request(client, vcpu, NULL);
+ vcpu = find_next_bit(client->ioreqs_map,
+ ACRN_IO_REQUEST_MAX, vcpu + 1);
+ }
+ }
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+
+ /* Clear ACRN_VM_FLAG_CLEARING_IOREQ flag after the clearing */
+ clear_bit(ACRN_VM_FLAG_CLEARING_IOREQ, &vm->flags);
+}
+
+int acrn_ioreq_client_wait(struct acrn_ioreq_client *client)
+{
+ if (client->is_default) {
+ /*
+ * In the default client, a user space thread waits on the
+ * waitqueue. The is_destroying() check is used to notify user
+ * space the client is going to be destroyed.
+ */
+ wait_event_interruptible(client->wq,
+ has_pending_request(client) ||
+ is_destroying(client));
+ if (is_destroying(client))
+ return -ENODEV;
+ } else {
+ wait_event_interruptible(client->wq,
+ has_pending_request(client) ||
+ kthread_should_stop());
+ }
+
+ return 0;
+}
+
+static bool in_range(struct acrn_ioreq_range *range,
+ struct acrn_io_request *req)
+{
+ bool ret = false;
+
+ if (range->type == req->type) {
+ switch (req->type) {
+ case ACRN_IOREQ_TYPE_MMIO:
+ if (req->reqs.mmio_request.address >= range->start &&
+ (req->reqs.mmio_request.address +
+ req->reqs.mmio_request.size - 1) <= range->end)
+ ret = true;
+ break;
+ case ACRN_IOREQ_TYPE_PORTIO:
+ if (req->reqs.pio_request.address >= range->start &&
+ (req->reqs.pio_request.address +
+ req->reqs.pio_request.size - 1) <= range->end)
+ ret = true;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return ret;
+}
+
+static struct acrn_ioreq_client *find_ioreq_client(struct acrn_vm *vm,
+ struct acrn_io_request *req)
+{
+ struct acrn_ioreq_client *client, *found = NULL;
+ struct acrn_ioreq_range *range;
+
+ lockdep_assert_held(&vm->ioreq_clients_lock);
+
+ list_for_each_entry(client, &vm->ioreq_clients, list) {
+ read_lock_bh(&client->range_lock);
+ list_for_each_entry(range, &client->range_list, list) {
+ if (in_range(range, req)) {
+ found = client;
+ break;
+ }
+ }
+ read_unlock_bh(&client->range_lock);
+ if (found)
+ break;
+ }
+ return found ? found : vm->default_client;
+}
+
+/**
+ * acrn_ioreq_client_create() - Create an ioreq client
+ * @vm: The VM that this client belongs to
+ * @handler: The ioreq_handler of ioreq client acrn_hsm will create a kernel
+ * thread and call the handler to handle I/O requests.
+ * @priv: Private data for the handler
+ * @is_default: If it is the default client
+ * @name: The name of ioreq client
+ *
+ * Return: acrn_ioreq_client pointer on success, NULL on error
+ */
+struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm,
+ ioreq_handler_t handler,
+ void *priv, bool is_default,
+ const char *name)
+{
+ struct acrn_ioreq_client *client;
+
+ if (!handler && !is_default) {
+ dev_dbg(acrn_dev.this_device,
+ "Cannot create non-default client w/o handler!\n");
+ return NULL;
+ }
+ client = kzalloc(sizeof(*client), GFP_KERNEL);
+ if (!client)
+ return NULL;
+
+ client->handler = handler;
+ client->vm = vm;
+ client->priv = priv;
+ client->is_default = is_default;
+ if (name)
+ strncpy(client->name, name, sizeof(client->name) - 1);
+ rwlock_init(&client->range_lock);
+ INIT_LIST_HEAD(&client->range_list);
+ init_waitqueue_head(&client->wq);
+
+ if (client->handler) {
+ client->thread = kthread_run(ioreq_task, client, "VM%u-%s",
+ client->vm->vmid, client->name);
+ if (IS_ERR(client->thread)) {
+ kfree(client);
+ return NULL;
+ }
+ }
+
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ if (is_default)
+ vm->default_client = client;
+ else
+ list_add(&client->list, &vm->ioreq_clients);
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+
+ dev_dbg(acrn_dev.this_device, "Created ioreq client %s.\n", name);
+ return client;
+}
+
+/**
+ * acrn_ioreq_client_destroy() - Destroy an ioreq client
+ * @client: The ioreq client
+ */
+void acrn_ioreq_client_destroy(struct acrn_ioreq_client *client)
+{
+ struct acrn_ioreq_range *range, *next;
+ struct acrn_vm *vm = client->vm;
+
+ dev_dbg(acrn_dev.this_device,
+ "Destroy ioreq client %s.\n", client->name);
+ ioreq_pause();
+ set_bit(ACRN_IOREQ_CLIENT_DESTROYING, &client->flags);
+ if (client->is_default)
+ wake_up_interruptible(&client->wq);
+ else
+ kthread_stop(client->thread);
+
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ if (client->is_default)
+ vm->default_client = NULL;
+ else
+ list_del(&client->list);
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+
+ write_lock_bh(&client->range_lock);
+ list_for_each_entry_safe(range, next, &client->range_list, list) {
+ list_del(&range->list);
+ kfree(range);
+ }
+ write_unlock_bh(&client->range_lock);
+ kfree(client);
+
+ ioreq_resume();
+}
+
+static int acrn_ioreq_dispatch(struct acrn_vm *vm)
+{
+ struct acrn_ioreq_client *client;
+ struct acrn_io_request *req;
+ int i;
+
+ for (i = 0; i < vm->vcpu_num; i++) {
+ req = vm->ioreq_buf->req_slot + i;
+
+ /* barrier the read of processed of acrn_io_request */
+ if (smp_load_acquire(&req->processed) ==
+ ACRN_IOREQ_STATE_PENDING) {
+ /* Complete the IO request directly in clearing stage */
+ if (test_bit(ACRN_VM_FLAG_CLEARING_IOREQ, &vm->flags)) {
+ ioreq_complete_request(vm, i, req);
+ continue;
+ }
+
+ spin_lock_bh(&vm->ioreq_clients_lock);
+ client = find_ioreq_client(vm, req);
+ if (!client) {
+ dev_err(acrn_dev.this_device,
+ "Failed to find ioreq client!\n");
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+ return -EINVAL;
+ }
+ if (!client->is_default)
+ req->kernel_handled = 1;
+ else
+ req->kernel_handled = 0;
+ /*
+ * Add barrier() to make sure the writes are done
+ * before setting ACRN_IOREQ_STATE_PROCESSING
+ */
+ smp_store_release(&req->processed,
+ ACRN_IOREQ_STATE_PROCESSING);
+ set_bit(i, client->ioreqs_map);
+ wake_up_interruptible(&client->wq);
+ spin_unlock_bh(&vm->ioreq_clients_lock);
+ }
+ }
+
+ return 0;
+}
+
+static void ioreq_tasklet_handler(unsigned long data)
+{
+ struct acrn_vm *vm;
+
+ read_lock(&acrn_vm_list_lock);
+ list_for_each_entry(vm, &acrn_vm_list, list) {
+ if (!vm->ioreq_buf)
+ break;
+ acrn_ioreq_dispatch(vm);
+ }
+ read_unlock(&acrn_vm_list_lock);
+}
+
+static void ioreq_pause(void)
+{
+ /* Flush and disable the tasklet to ensure no I/O requests pending */
+ tasklet_disable(&ioreq_tasklet);
+}
+
+static void ioreq_resume(void)
+{
+ /* Schedule once after enabling in case other clients miss a tasklet */
+ tasklet_enable(&ioreq_tasklet);
+ tasklet_schedule(&ioreq_tasklet);
+}
+
+static void ioreq_intr_handler(void)
+{
+ tasklet_schedule(&ioreq_tasklet);
+}
+
+void acrn_ioreq_intr_setup(void)
+{
+ acrn_setup_intr_handler(ioreq_intr_handler);
+ tasklet_init(&ioreq_tasklet, ioreq_tasklet_handler, 0);
+}
+
+void acrn_ioreq_intr_remove(void)
+{
+ acrn_remove_intr_handler();
+}
+
+int acrn_ioreq_init(struct acrn_vm *vm, u64 buf_vma)
+{
+ struct acrn_ioreq_buffer *set_buffer;
+ struct page *page;
+ int ret;
+
+ if (vm->ioreq_buf)
+ return -EEXIST;
+
+ set_buffer = kzalloc(sizeof(*set_buffer), GFP_KERNEL);
+ if (!set_buffer)
+ return -ENOMEM;
+
+ ret = pin_user_pages_fast(buf_vma, 1,
+ FOLL_WRITE | FOLL_LONGTERM, &page);
+ if (unlikely(ret != 1) || !page) {
+ dev_err(acrn_dev.this_device, "Failed to pin ioreq page!\n");
+ ret = -EFAULT;
+ goto free_buf;
+ }
+
+ vm->ioreq_buf = page_address(page);
+ vm->ioreq_page = page;
+ set_buffer->ioreq_buf = page_to_phys(page);
+ ret = hcall_set_ioreq_buffer(vm->vmid, virt_to_phys(set_buffer));
+ if (ret < 0) {
+ dev_err(acrn_dev.this_device, "Failed to init ioreq buffer!\n");
+ unpin_user_page(page);
+ vm->ioreq_buf = NULL;
+ goto free_buf;
+ }
+
+ dev_dbg(acrn_dev.this_device,
+ "Init ioreq buffer %pK!\n", vm->ioreq_buf);
+ ret = 0;
+free_buf:
+ kfree(set_buffer);
+ return ret;
+}
+
+void acrn_ioreq_deinit(struct acrn_vm *vm)
+{
+ struct acrn_ioreq_client *client, *next;
+
+ dev_dbg(acrn_dev.this_device,
+ "Deinit ioreq buffer %pK!\n", vm->ioreq_buf);
+ /* Destroy all clients belonging to this VM */
+ list_for_each_entry_safe(client, next, &vm->ioreq_clients, list)
+ acrn_ioreq_client_destroy(client);
+ if (vm->default_client)
+ acrn_ioreq_client_destroy(vm->default_client);
+
+ if (vm->ioreq_buf && vm->ioreq_page) {
+ unpin_user_page(vm->ioreq_page);
+ vm->ioreq_buf = NULL;
+ }
+}
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
index ff5df7acb551..4ee1a99df4b7 100644
--- a/drivers/virt/acrn/vm.c
+++ b/drivers/virt/acrn/vm.c
@@ -15,9 +15,12 @@
#include "acrn_drv.h"
/* List of VMs */
-static LIST_HEAD(acrn_vm_list);
-/* To protect acrn_vm_list */
-static DEFINE_MUTEX(acrn_vm_list_lock);
+LIST_HEAD(acrn_vm_list);
+/*
+ * acrn_vm_list is read in a tasklet which dispatch I/O requests and is wrote
+ * in VM creation ioctl. Use the rwlock mechanism to protect it.
+ */
+DEFINE_RWLOCK(acrn_vm_list_lock);
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
struct acrn_vm_creation *vm_param)
@@ -32,12 +35,20 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
}
mutex_init(&vm->regions_mapping_lock);
+ INIT_LIST_HEAD(&vm->ioreq_clients);
+ spin_lock_init(&vm->ioreq_clients_lock);
vm->vmid = vm_param->vmid;
vm->vcpu_num = vm_param->vcpu_num;
- mutex_lock(&acrn_vm_list_lock);
+ if (acrn_ioreq_init(vm, vm_param->ioreq_buf) < 0) {
+ hcall_destroy_vm(vm_param->vmid);
+ vm->vmid = ACRN_INVALID_VMID;
+ return NULL;
+ }
+
+ write_lock_bh(&acrn_vm_list_lock);
list_add(&vm->list, &acrn_vm_list);
- mutex_unlock(&acrn_vm_list_lock);
+ write_unlock_bh(&acrn_vm_list_lock);
dev_dbg(acrn_dev.this_device, "VM %u created.\n", vm->vmid);
return vm;
@@ -52,9 +63,11 @@ int acrn_vm_destroy(struct acrn_vm *vm)
return 0;
/* Remove from global VM list */
- mutex_lock(&acrn_vm_list_lock);
+ write_lock_bh(&acrn_vm_list_lock);
list_del_init(&vm->list);
- mutex_unlock(&acrn_vm_list_lock);
+ write_unlock_bh(&acrn_vm_list_lock);
+
+ acrn_ioreq_deinit(vm);
ret = hcall_destroy_vm(vm->vmid);
if (ret < 0) {
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index c260d0e9caea..96ebc879b8d0 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -12,6 +12,129 @@
#include <linux/types.h>
#include <linux/uuid.h>
+#define ACRN_IO_REQUEST_MAX 16
+
+#define ACRN_IOREQ_STATE_PENDING 0
+#define ACRN_IOREQ_STATE_COMPLETE 1
+#define ACRN_IOREQ_STATE_PROCESSING 2
+#define ACRN_IOREQ_STATE_FREE 3
+
+#define ACRN_IOREQ_TYPE_PORTIO 0
+#define ACRN_IOREQ_TYPE_MMIO 1
+
+#define ACRN_IOREQ_DIR_READ 0
+#define ACRN_IOREQ_DIR_WRITE 1
+
+struct acrn_mmio_request {
+ __u32 direction;
+ __u32 reserved;
+ __u64 address;
+ __u64 size;
+ __u64 value;
+};
+
+struct acrn_pio_request {
+ __u32 direction;
+ __u32 reserved;
+ __u64 address;
+ __u64 size;
+ __u32 value;
+};
+
+/**
+ * struct acrn_io_request - 256-byte ACRN I/O request
+ * @type: Type of this request (ACRN_IOREQ_TYPE_*).
+ * @completion_polling: Polling flag. Hypervisor will poll completion of the
+ * I/O request if this flag set.
+ * @reserved0: Reserved fields.
+ * @reqs: Union of different types of request. Byte offset: 64.
+ * @reqs.pio_request: PIO request data of the I/O request.
+ * @reqs.mmio_request: MMIO request data of the I/O request.
+ * @reqs.data: Raw data of the I/O request.
+ * @reserved1: Reserved fields.
+ * @kernel_handled: Flag indicates this request need be handled in kernel.
+ * @processed: The status of this request (ACRN_IOREQ_STATE_*).
+ *
+ * The state transitions of ACRN I/O request:
+ *
+ * FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...
+ *
+ * An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and
+ * ACRN userspace are in charge of processing the others.
+ *
+ * On basis of the states illustrated above, a typical lifecycle of ACRN IO
+ * request would look like:
+ *
+ * Flow (assume the initial state is FREE)
+ * |
+ * | Service VM vCPU 0 Service VM vCPU x User vCPU y
+ * |
+ * | hypervisor:
+ * | fills in type, addr, etc.
+ * | pauses the User VM vCPU y
+ * | sets the state to PENDING (a)
+ * | fires an upcall to Service VM
+ * |
+ * | HSM:
+ * | scans for PENDING requests
+ * | sets the states to PROCESSING (b)
+ * | assigns the requests to clients (c)
+ * V
+ * | client:
+ * | scans for the assigned requests
+ * | handles the requests (d)
+ * | HSM:
+ * | sets states to COMPLETE
+ * | notifies the hypervisor
+ * |
+ * | hypervisor:
+ * | resumes User VM vCPU y (e)
+ * |
+ * | hypervisor:
+ * | post handling (f)
+ * V sets states to FREE
+ *
+ * Note that the procedures (a) to (f) in the illustration above require to be
+ * strictly processed in the order. One vCPU cannot trigger another request of
+ * I/O emulation before completing the previous one.
+ *
+ * Atomic and barriers are required when HSM and hypervisor accessing the state
+ * of &struct acrn_io_request.
+ *
+ */
+struct acrn_io_request {
+ __u32 type;
+ __u32 completion_polling;
+ __u32 reserved0[14];
+ union {
+ struct acrn_pio_request pio_request;
+ struct acrn_mmio_request mmio_request;
+ __u64 data[8];
+ } reqs;
+ __u32 reserved1;
+ __u32 kernel_handled;
+ __u32 processed;
+} __attribute__((aligned(256)));
+
+struct acrn_io_request_buffer {
+ union {
+ struct acrn_io_request req_slot[ACRN_IO_REQUEST_MAX];
+ __u8 reserved[4096];
+ };
+};
+
+/**
+ * struct acrn_ioreq_notify - The structure of ioreq completion notification
+ * @vmid: User VM ID
+ * @reserved: Reserved
+ * @vcpu: vCPU ID
+ */
+struct acrn_ioreq_notify {
+ __u16 vmid;
+ __u16 reserved;
+ __u32 vcpu;
+};
+
/**
* struct acrn_vm_creation - Info to create a User VM
* @vmid: User VM ID returned from the hypervisor
@@ -165,6 +288,17 @@ struct acrn_vm_memmap {
#define ACRN_IOCTL_SET_VCPU_REGS \
_IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs)
+#define ACRN_IOCTL_NOTIFY_REQUEST_FINISH \
+ _IOW(ACRN_IOCTL_TYPE, 0x31, struct acrn_ioreq_notify)
+#define ACRN_IOCTL_CREATE_IOREQ_CLIENT \
+ _IO(ACRN_IOCTL_TYPE, 0x32)
+#define ACRN_IOCTL_ATTACH_IOREQ_CLIENT \
+ _IO(ACRN_IOCTL_TYPE, 0x33)
+#define ACRN_IOCTL_DESTROY_IOREQ_CLIENT \
+ _IO(ACRN_IOCTL_TYPE, 0x34)
+#define ACRN_IOCTL_CLEAR_VM_IOREQ \
+ _IO(ACRN_IOCTL_TYPE, 0x35)
+
#define ACRN_IOCTL_SET_MEMSEG \
_IOW(ACRN_IOCTL_TYPE, 0x41, struct acrn_vm_memmap)
#define ACRN_IOCTL_UNSET_MEMSEG \
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 10/18] virt: acrn: Introduce PCI configuration space PIO accesses combiner
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (8 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 09/18] virt: acrn: Introduce I/O request management shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 11/18] virt: acrn: Introduce interfaces for PCI device passthrough shuo.a.liu
` (7 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu
From: Shuo Liu <shuo.a.liu@intel.com>
A User VM can access its virtual PCI configuration spaces via port IO
approach, which has two following steps:
1) writes address into port 0xCF8
2) put/get data in/from port 0xCFC
To distribute a complete PCI configuration space access one time, HSM
need to combine such two accesses together.
Combine two paired PIO I/O requests into one PCI I/O request and
continue the I/O request distribution.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/acrn_drv.h | 2 +
drivers/virt/acrn/ioreq.c | 76 ++++++++++++++++++++++++++++++++++++
include/uapi/linux/acrn.h | 15 +++++++
3 files changed, 93 insertions(+)
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 7e89e4944d0f..97a7f31cd681 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -154,6 +154,7 @@ extern rwlock_t acrn_vm_list_lock;
* @default_client: The default I/O request client
* @ioreq_buf: I/O request shared buffer
* @ioreq_page: The page of the I/O request shared buffer
+ * @pci_conf_addr: Address of a PCI configuration access emulation
*/
struct acrn_vm {
struct list_head list;
@@ -168,6 +169,7 @@ struct acrn_vm {
struct acrn_ioreq_client *default_client;
struct acrn_io_request_buffer *ioreq_buf;
struct page *ioreq_page;
+ u32 pci_conf_addr;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
diff --git a/drivers/virt/acrn/ioreq.c b/drivers/virt/acrn/ioreq.c
index ed22b1c4775d..b469f273e20c 100644
--- a/drivers/virt/acrn/ioreq.c
+++ b/drivers/virt/acrn/ioreq.c
@@ -220,6 +220,80 @@ int acrn_ioreq_client_wait(struct acrn_ioreq_client *client)
return 0;
}
+static bool is_cfg_addr(struct acrn_io_request *req)
+{
+ return ((req->type == ACRN_IOREQ_TYPE_PORTIO) &&
+ (req->reqs.pio_request.address == 0xcf8));
+}
+
+static bool is_cfg_data(struct acrn_io_request *req)
+{
+ return ((req->type == ACRN_IOREQ_TYPE_PORTIO) &&
+ ((req->reqs.pio_request.address >= 0xcfc) &&
+ (req->reqs.pio_request.address < (0xcfc + 4))));
+}
+
+/* The low 8-bit of supported pci_reg addr.*/
+#define PCI_LOWREG_MASK 0xFC
+/* The high 4-bit of supported pci_reg addr */
+#define PCI_HIGHREG_MASK 0xF00
+/* Max number of supported functions */
+#define PCI_FUNCMAX 7
+/* Max number of supported slots */
+#define PCI_SLOTMAX 31
+/* Max number of supported buses */
+#define PCI_BUSMAX 255
+#define CONF1_ENABLE 0x80000000UL
+/*
+ * A PCI configuration space access via PIO 0xCF8 and 0xCFC normally has two
+ * following steps:
+ * 1) writes address into 0xCF8 port
+ * 2) accesses data in/from 0xCFC
+ * This function combines such paired PCI configuration space I/O requests into
+ * one ACRN_IOREQ_TYPE_PCICFG type I/O request and continues the processing.
+ */
+static bool handle_cf8cfc(struct acrn_vm *vm,
+ struct acrn_io_request *req, u16 vcpu)
+{
+ int offset, pci_cfg_addr, pci_reg;
+ bool is_handled = false;
+
+ if (is_cfg_addr(req)) {
+ WARN_ON(req->reqs.pio_request.size != 4);
+ if (req->reqs.pio_request.direction == ACRN_IOREQ_DIR_WRITE)
+ vm->pci_conf_addr = req->reqs.pio_request.value;
+ else
+ req->reqs.pio_request.value = vm->pci_conf_addr;
+ is_handled = true;
+ } else if (is_cfg_data(req)) {
+ if (!(vm->pci_conf_addr & CONF1_ENABLE)) {
+ if (req->reqs.pio_request.direction ==
+ ACRN_IOREQ_DIR_READ)
+ req->reqs.pio_request.value = 0xffffffff;
+ is_handled = true;
+ } else {
+ offset = req->reqs.pio_request.address - 0xcfc;
+
+ req->type = ACRN_IOREQ_TYPE_PCICFG;
+ pci_cfg_addr = vm->pci_conf_addr;
+ req->reqs.pci_request.bus =
+ (pci_cfg_addr >> 16) & PCI_BUSMAX;
+ req->reqs.pci_request.dev =
+ (pci_cfg_addr >> 11) & PCI_SLOTMAX;
+ req->reqs.pci_request.func =
+ (pci_cfg_addr >> 8) & PCI_FUNCMAX;
+ pci_reg = (pci_cfg_addr & PCI_LOWREG_MASK) +
+ ((pci_cfg_addr >> 16) & PCI_HIGHREG_MASK);
+ req->reqs.pci_request.reg = pci_reg + offset;
+ }
+ }
+
+ if (is_handled)
+ ioreq_complete_request(vm, vcpu, req);
+
+ return is_handled;
+}
+
static bool in_range(struct acrn_ioreq_range *range,
struct acrn_io_request *req)
{
@@ -380,6 +454,8 @@ static int acrn_ioreq_dispatch(struct acrn_vm *vm)
ioreq_complete_request(vm, i, req);
continue;
}
+ if (handle_cf8cfc(vm, req, i))
+ continue;
spin_lock_bh(&vm->ioreq_clients_lock);
client = find_ioreq_client(vm, req);
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index 96ebc879b8d0..66c43bd1e1b3 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -21,6 +21,7 @@
#define ACRN_IOREQ_TYPE_PORTIO 0
#define ACRN_IOREQ_TYPE_MMIO 1
+#define ACRN_IOREQ_TYPE_PCICFG 2
#define ACRN_IOREQ_DIR_READ 0
#define ACRN_IOREQ_DIR_WRITE 1
@@ -41,6 +42,18 @@ struct acrn_pio_request {
__u32 value;
};
+/* Need keep same header fields with pio_request */
+struct acrn_pci_request {
+ __u32 direction;
+ __u32 reserved[3];
+ __u64 size;
+ __u32 value;
+ __u32 bus;
+ __u32 dev;
+ __u32 func;
+ __u32 reg;
+};
+
/**
* struct acrn_io_request - 256-byte ACRN I/O request
* @type: Type of this request (ACRN_IOREQ_TYPE_*).
@@ -49,6 +62,7 @@ struct acrn_pio_request {
* @reserved0: Reserved fields.
* @reqs: Union of different types of request. Byte offset: 64.
* @reqs.pio_request: PIO request data of the I/O request.
+ * @reqs.pci_request: PCI configuration space request data of the I/O request.
* @reqs.mmio_request: MMIO request data of the I/O request.
* @reqs.data: Raw data of the I/O request.
* @reserved1: Reserved fields.
@@ -108,6 +122,7 @@ struct acrn_io_request {
__u32 reserved0[14];
union {
struct acrn_pio_request pio_request;
+ struct acrn_pci_request pci_request;
struct acrn_mmio_request mmio_request;
__u64 data[8];
} reqs;
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 11/18] virt: acrn: Introduce interfaces for PCI device passthrough
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (9 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 10/18] virt: acrn: Introduce PCI configuration space PIO accesses combiner shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 12/18] virt: acrn: Introduce interrupt injection interfaces shuo.a.liu
` (6 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
PCI device passthrough enables an OS in a virtual machine to directly
access a PCI device in the host. It promises almost the native
performance, which is required in performance-critical scenarios of
ACRN.
HSM provides the following ioctls:
- Assign - ACRN_IOCTL_ASSIGN_PCIDEV
Pass data struct acrn_pcidev from userspace to the hypervisor, and
inform the hypervisor to assign a PCI device to a User VM.
- De-assign - ACRN_IOCTL_DEASSIGN_PCIDEV
Pass data struct acrn_pcidev from userspace to the hypervisor, and
inform the hypervisor to de-assign a PCI device from a User VM.
- Set a interrupt of a passthrough device - ACRN_IOCTL_SET_PTDEV_INTR
Pass data struct acrn_ptdev_irq from userspace to the hypervisor,
and inform the hypervisor to map a INTx interrupt of passthrough
device of User VM.
- Reset passthrough device interrupt - ACRN_IOCTL_RESET_PTDEV_INTR
Pass data struct acrn_ptdev_irq from userspace to the hypervisor,
and inform the hypervisor to unmap a INTx interrupt of passthrough
device of User VM.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/hsm.c | 50 ++++++++++++++++++++++++++++
drivers/virt/acrn/hypercall.h | 54 +++++++++++++++++++++++++++++++
include/uapi/linux/acrn.h | 61 +++++++++++++++++++++++++++++++++++
3 files changed, 165 insertions(+)
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index ea86be0ab5e4..c1eb1dc62801 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -49,7 +49,9 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_vm_creation *vm_param;
struct acrn_vcpu_regs *cpu_regs;
struct acrn_ioreq_notify notify;
+ struct acrn_ptdev_irq *irq_info;
struct acrn_vm_memmap memmap;
+ struct acrn_pcidev *pcidev;
int ret = 0;
if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
@@ -131,6 +133,54 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
ret = acrn_vm_memseg_unmap(vm, &memmap);
break;
+ case ACRN_IOCTL_ASSIGN_PCIDEV:
+ pcidev = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_pcidev));
+ if (IS_ERR(pcidev))
+ return PTR_ERR(pcidev);
+
+ ret = hcall_assign_pcidev(vm->vmid, virt_to_phys(pcidev));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to assign pci device!\n");
+ kfree(pcidev);
+ break;
+ case ACRN_IOCTL_DEASSIGN_PCIDEV:
+ pcidev = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_pcidev));
+ if (IS_ERR(pcidev))
+ return PTR_ERR(pcidev);
+
+ ret = hcall_deassign_pcidev(vm->vmid, virt_to_phys(pcidev));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to deassign pci device!\n");
+ kfree(pcidev);
+ break;
+ case ACRN_IOCTL_SET_PTDEV_INTR:
+ irq_info = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_ptdev_irq));
+ if (IS_ERR(irq_info))
+ return PTR_ERR(irq_info);
+
+ ret = hcall_set_ptdev_intr(vm->vmid, virt_to_phys(irq_info));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to configure intr for ptdev!\n");
+ kfree(irq_info);
+ break;
+ case ACRN_IOCTL_RESET_PTDEV_INTR:
+ irq_info = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_ptdev_irq));
+ if (IS_ERR(irq_info))
+ return PTR_ERR(irq_info);
+
+ ret = hcall_reset_ptdev_intr(vm->vmid, virt_to_phys(irq_info));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to reset intr for ptdev!\n");
+ kfree(irq_info);
+ break;
case ACRN_IOCTL_CREATE_IOREQ_CLIENT:
if (vm->default_client)
return -EEXIST;
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index 5eba29e3ed38..f448301832cf 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -28,6 +28,12 @@
#define HC_ID_MEM_BASE 0x40UL
#define HC_VM_SET_MEMORY_REGIONS _HC_ID(HC_ID, HC_ID_MEM_BASE + 0x02)
+#define HC_ID_PCI_BASE 0x50UL
+#define HC_SET_PTDEV_INTR _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x03)
+#define HC_RESET_PTDEV_INTR _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x04)
+#define HC_ASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x05)
+#define HC_DEASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x06)
+
/**
* hcall_create_vm() - Create a User VM
* @vminfo: Service VM GPA of info of User VM creation
@@ -130,4 +136,52 @@ static inline long hcall_set_memory_regions(u64 regions_pa)
return acrn_hypercall1(HC_VM_SET_MEMORY_REGIONS, regions_pa);
}
+/**
+ * hcall_assign_pcidev() - Assign a PCI device to a User VM
+ * @vmid: User VM ID
+ * @addr: Service VM GPA of the &struct acrn_pcidev
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_assign_pcidev(u64 vmid, u64 addr)
+{
+ return acrn_hypercall2(HC_ASSIGN_PCIDEV, vmid, addr);
+}
+
+/**
+ * hcall_deassign_pcidev() - De-assign a PCI device from a User VM
+ * @vmid: User VM ID
+ * @addr: Service VM GPA of the &struct acrn_pcidev
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_deassign_pcidev(u64 vmid, u64 addr)
+{
+ return acrn_hypercall2(HC_DEASSIGN_PCIDEV, vmid, addr);
+}
+
+/**
+ * hcall_set_ptdev_intr() - Configure an interrupt for an assigned PCI device.
+ * @vmid: User VM ID
+ * @irq: Service VM GPA of the &struct acrn_ptdev_irq
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_set_ptdev_intr(u64 vmid, u64 irq)
+{
+ return acrn_hypercall2(HC_SET_PTDEV_INTR, vmid, irq);
+}
+
+/**
+ * hcall_reset_ptdev_intr() - Reset an interrupt for an assigned PCI device.
+ * @vmid: User VM ID
+ * @irq: Service VM GPA of the &struct acrn_ptdev_irq
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_reset_ptdev_intr(u64 vmid, u64 irq)
+{
+ return acrn_hypercall2(HC_RESET_PTDEV_INTR, vmid, irq);
+}
+
#endif /* __ACRN_HSM_HYPERCALL_H */
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index 66c43bd1e1b3..2727f16a7ef4 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -284,6 +284,58 @@ struct acrn_vm_memmap {
__u64 len;
};
+/* Type of interrupt of a passthrough device */
+#define ACRN_PTDEV_IRQ_INTX 0
+#define ACRN_PTDEV_IRQ_MSI 1
+#define ACRN_PTDEV_IRQ_MSIX 2
+/**
+ * struct acrn_ptdev_irq - Interrupt data of a passthrough device.
+ * @type: Type (ACRN_PTDEV_IRQ_*)
+ * @virt_bdf: Virtual Bus/Device/Function
+ * @phys_bdf: Physical Bus/Device/Function
+ * @intx: Info of interrupt
+ * @intx.virt_pin: Virtual IOAPIC pin
+ * @intx.phys_pin: Physical IOAPIC pin
+ * @intx.is_pic_pin: Is PIC pin or not
+ *
+ * This structure will be passed to hypervisor directly.
+ */
+struct acrn_ptdev_irq {
+ __u32 type;
+ __u16 virt_bdf;
+ __u16 phys_bdf;
+
+ struct {
+ __u32 virt_pin;
+ __u32 phys_pin;
+ __u32 is_pic_pin;
+ } intx;
+};
+
+/* Type of PCI device assignment */
+#define ACRN_PTDEV_QUIRK_ASSIGN (1U << 0)
+
+#define ACRN_PCI_NUM_BARS 6
+/**
+ * struct acrn_pcidev - Info for assigning or de-assigning a PCI device
+ * @type: Type of the assignment
+ * @virt_bdf: Virtual Bus/Device/Function
+ * @phys_bdf: Physical Bus/Device/Function
+ * @intr_line: PCI interrupt line
+ * @intr_pin: PCI interrupt pin
+ * @bar: PCI BARs.
+ *
+ * This structure will be passed to hypervisor directly.
+ */
+struct acrn_pcidev {
+ __u32 type;
+ __u16 virt_bdf;
+ __u16 phys_bdf;
+ __u8 intr_line;
+ __u8 intr_pin;
+ __u32 bar[ACRN_PCI_NUM_BARS];
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -319,4 +371,13 @@ struct acrn_vm_memmap {
#define ACRN_IOCTL_UNSET_MEMSEG \
_IOW(ACRN_IOCTL_TYPE, 0x42, struct acrn_vm_memmap)
+#define ACRN_IOCTL_SET_PTDEV_INTR \
+ _IOW(ACRN_IOCTL_TYPE, 0x53, struct acrn_ptdev_irq)
+#define ACRN_IOCTL_RESET_PTDEV_INTR \
+ _IOW(ACRN_IOCTL_TYPE, 0x54, struct acrn_ptdev_irq)
+#define ACRN_IOCTL_ASSIGN_PCIDEV \
+ _IOW(ACRN_IOCTL_TYPE, 0x55, struct acrn_pcidev)
+#define ACRN_IOCTL_DEASSIGN_PCIDEV \
+ _IOW(ACRN_IOCTL_TYPE, 0x56, struct acrn_pcidev)
+
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 12/18] virt: acrn: Introduce interrupt injection interfaces
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (10 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 11/18] virt: acrn: Introduce interfaces for PCI device passthrough shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 13/18] virt: acrn: Introduce interfaces to query C-states and P-states allowed by hypervisor shuo.a.liu
` (5 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
ACRN userspace need to inject virtual interrupts into a User VM in
devices emulation.
HSM needs provide interfaces to do so.
Introduce following interrupt injection interfaces:
ioctl ACRN_IOCTL_SET_IRQLINE:
Pass data from userspace to the hypervisor, and inform the hypervisor
to inject a virtual IOAPIC GSI interrupt to a User VM.
ioctl ACRN_IOCTL_INJECT_MSI:
Pass data struct acrn_msi_entry from userspace to the hypervisor, and
inform the hypervisor to inject a virtual MSI to a User VM.
ioctl ACRN_IOCTL_VM_INTR_MONITOR:
Set a 4-Kbyte aligned shared page for statistics information of
interrupts of a User VM.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/acrn_drv.h | 4 ++++
drivers/virt/acrn/hsm.c | 40 ++++++++++++++++++++++++++++++++++
drivers/virt/acrn/hypercall.h | 41 +++++++++++++++++++++++++++++++++++
drivers/virt/acrn/vm.c | 36 ++++++++++++++++++++++++++++++
include/uapi/linux/acrn.h | 17 +++++++++++++++
5 files changed, 138 insertions(+)
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 97a7f31cd681..542448c88026 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -155,6 +155,7 @@ extern rwlock_t acrn_vm_list_lock;
* @ioreq_buf: I/O request shared buffer
* @ioreq_page: The page of the I/O request shared buffer
* @pci_conf_addr: Address of a PCI configuration access emulation
+ * @monitor_page: Page of interrupt statistics of User VM
*/
struct acrn_vm {
struct list_head list;
@@ -170,6 +171,7 @@ struct acrn_vm {
struct acrn_io_request_buffer *ioreq_buf;
struct page *ioreq_page;
u32 pci_conf_addr;
+ struct page *monitor_page;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -196,4 +198,6 @@ struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm,
const char *name);
void acrn_ioreq_client_destroy(struct acrn_ioreq_client *client);
+int acrn_msi_inject(struct acrn_vm *vm, u64 msi_addr, u64 msi_data);
+
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index c1eb1dc62801..43fb2e32108c 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -51,7 +51,9 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_ioreq_notify notify;
struct acrn_ptdev_irq *irq_info;
struct acrn_vm_memmap memmap;
+ struct acrn_msi_entry *msi;
struct acrn_pcidev *pcidev;
+ struct page *page;
int ret = 0;
if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
@@ -181,6 +183,44 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
"Failed to reset intr for ptdev!\n");
kfree(irq_info);
break;
+ case ACRN_IOCTL_SET_IRQLINE:
+ ret = hcall_set_irqline(vm->vmid, ioctl_param);
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to set interrupt line!\n");
+ break;
+ case ACRN_IOCTL_INJECT_MSI:
+ msi = memdup_user((void __user *)ioctl_param,
+ sizeof(struct acrn_msi_entry));
+ if (IS_ERR(msi))
+ return PTR_ERR(msi);
+
+ ret = hcall_inject_msi(vm->vmid, virt_to_phys(msi));
+ if (ret < 0)
+ dev_dbg(acrn_dev.this_device,
+ "Failed to inject MSI!\n");
+ kfree(msi);
+ break;
+ case ACRN_IOCTL_VM_INTR_MONITOR:
+ ret = pin_user_pages_fast(ioctl_param, 1,
+ FOLL_WRITE | FOLL_LONGTERM, &page);
+ if (unlikely(ret != 1)) {
+ dev_dbg(acrn_dev.this_device,
+ "Failed to pin intr hdr buffer!\n");
+ return -EFAULT;
+ }
+
+ ret = hcall_vm_intr_monitor(vm->vmid, page_to_phys(page));
+ if (ret < 0) {
+ unpin_user_page(page);
+ dev_dbg(acrn_dev.this_device,
+ "Failed to monitor intr data!\n");
+ return ret;
+ }
+ if (vm->monitor_page)
+ unpin_user_page(vm->monitor_page);
+ vm->monitor_page = page;
+ break;
case ACRN_IOCTL_CREATE_IOREQ_CLIENT:
if (vm->default_client)
return -EEXIST;
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index f448301832cf..a8813397a3fe 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -21,6 +21,11 @@
#define HC_RESET_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x05)
#define HC_SET_VCPU_REGS _HC_ID(HC_ID, HC_ID_VM_BASE + 0x06)
+#define HC_ID_IRQ_BASE 0x20UL
+#define HC_INJECT_MSI _HC_ID(HC_ID, HC_ID_IRQ_BASE + 0x03)
+#define HC_VM_INTR_MONITOR _HC_ID(HC_ID, HC_ID_IRQ_BASE + 0x04)
+#define HC_SET_IRQLINE _HC_ID(HC_ID, HC_ID_IRQ_BASE + 0x05)
+
#define HC_ID_IOREQ_BASE 0x30UL
#define HC_SET_IOREQ_BUFFER _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x00)
#define HC_NOTIFY_REQUEST_FINISH _HC_ID(HC_ID, HC_ID_IOREQ_BASE + 0x01)
@@ -101,6 +106,42 @@ static inline long hcall_set_vcpu_regs(u64 vmid, u64 regs_state)
return acrn_hypercall2(HC_SET_VCPU_REGS, vmid, regs_state);
}
+/**
+ * hcall_inject_msi() - Deliver a MSI interrupt to a User VM
+ * @vmid: User VM ID
+ * @msi: Service VM GPA of MSI message
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_inject_msi(u64 vmid, u64 msi)
+{
+ return acrn_hypercall2(HC_INJECT_MSI, vmid, msi);
+}
+
+/**
+ * hcall_vm_intr_monitor() - Set a shared page for User VM interrupt statistics
+ * @vmid: User VM ID
+ * @addr: Service VM GPA of the shared page
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_vm_intr_monitor(u64 vmid, u64 addr)
+{
+ return acrn_hypercall2(HC_VM_INTR_MONITOR, vmid, addr);
+}
+
+/**
+ * hcall_set_irqline() - Set or clear an interrupt line
+ * @vmid: User VM ID
+ * @op: Service VM GPA of interrupt line operations
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_set_irqline(u64 vmid, u64 op)
+{
+ return acrn_hypercall2(HC_SET_IRQLINE, vmid, op);
+}
+
/**
* hcall_set_ioreq_buffer() - Set up the shared buffer for I/O Requests.
* @vmid: User VM ID
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
index 4ee1a99df4b7..38304aeef181 100644
--- a/drivers/virt/acrn/vm.c
+++ b/drivers/virt/acrn/vm.c
@@ -68,6 +68,10 @@ int acrn_vm_destroy(struct acrn_vm *vm)
write_unlock_bh(&acrn_vm_list_lock);
acrn_ioreq_deinit(vm);
+ if (vm->monitor_page) {
+ put_page(vm->monitor_page);
+ vm->monitor_page = NULL;
+ }
ret = hcall_destroy_vm(vm->vmid);
if (ret < 0) {
@@ -83,3 +87,35 @@ int acrn_vm_destroy(struct acrn_vm *vm)
vm->vmid = ACRN_INVALID_VMID;
return 0;
}
+
+/**
+ * acrn_inject_msi() - Inject a MSI interrupt into a User VM
+ * @vm: User VM
+ * @msi_addr: The MSI address
+ * @msi_data: The MSI data
+ *
+ * Return: 0 on success, <0 on error
+ */
+int acrn_msi_inject(struct acrn_vm *vm, u64 msi_addr, u64 msi_data)
+{
+ struct acrn_msi_entry *msi;
+ int ret;
+
+ /* might be used in interrupt context, so use GFP_ATOMIC */
+ msi = kzalloc(sizeof(*msi), GFP_ATOMIC);
+ if (!msi)
+ return -ENOMEM;
+
+ /*
+ * msi_addr: addr[19:12] with dest vcpu id
+ * msi_data: data[7:0] with vector
+ */
+ msi->msi_addr = msi_addr;
+ msi->msi_data = msi_data;
+ ret = hcall_inject_msi(vm->vmid, virt_to_phys(msi));
+ if (ret < 0)
+ dev_err(acrn_dev.this_device,
+ "Failed to inject MSI to VM %u!\n", vm->vmid);
+ kfree(msi);
+ return ret;
+}
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index 2727f16a7ef4..959dfbccc6a4 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -336,6 +336,16 @@ struct acrn_pcidev {
__u32 bar[ACRN_PCI_NUM_BARS];
};
+/**
+ * struct acrn_msi_entry - Info for injecting a MSI interrupt to a VM
+ * @msi_addr: MSI addr[19:12] with dest vCPU ID
+ * @msi_data: MSI data[7:0] with vector
+ */
+struct acrn_msi_entry {
+ __u64 msi_addr;
+ __u64 msi_data;
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -355,6 +365,13 @@ struct acrn_pcidev {
#define ACRN_IOCTL_SET_VCPU_REGS \
_IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs)
+#define ACRN_IOCTL_INJECT_MSI \
+ _IOW(ACRN_IOCTL_TYPE, 0x23, struct acrn_msi_entry)
+#define ACRN_IOCTL_VM_INTR_MONITOR \
+ _IOW(ACRN_IOCTL_TYPE, 0x24, unsigned long)
+#define ACRN_IOCTL_SET_IRQLINE \
+ _IOW(ACRN_IOCTL_TYPE, 0x25, __u64)
+
#define ACRN_IOCTL_NOTIFY_REQUEST_FINISH \
_IOW(ACRN_IOCTL_TYPE, 0x31, struct acrn_ioreq_notify)
#define ACRN_IOCTL_CREATE_IOREQ_CLIENT \
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 13/18] virt: acrn: Introduce interfaces to query C-states and P-states allowed by hypervisor
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (11 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 12/18] virt: acrn: Introduce interrupt injection interfaces shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 14/18] virt: acrn: Introduce I/O ranges operation interfaces shuo.a.liu
` (4 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
The C-states and P-states data are used to support CPU power management.
The hypervisor controls C-states and P-states for a User VM.
ACRN userspace need to query the data from the hypervisor to build ACPI
tables for a User VM.
HSM provides ioctls for ACRN userspace to query C-states and P-states
data obtained from the hypervisor.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/hsm.c | 69 +++++++++++++++++++++++++++++++++++
drivers/virt/acrn/hypercall.h | 12 ++++++
include/uapi/linux/acrn.h | 35 ++++++++++++++++++
3 files changed, 116 insertions(+)
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index 43fb2e32108c..700006e2465b 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -38,6 +38,67 @@ static int acrn_dev_open(struct inode *inode, struct file *filp)
return 0;
}
+static int pmcmd_ioctl(u64 cmd, void __user *uptr)
+{
+ struct acrn_pstate_data *px_data;
+ struct acrn_cstate_data *cx_data;
+ u64 *pm_info;
+ int ret = 0;
+
+ switch (cmd & PMCMD_TYPE_MASK) {
+ case ACRN_PMCMD_GET_PX_CNT:
+ case ACRN_PMCMD_GET_CX_CNT:
+ pm_info = kmalloc(sizeof(u64), GFP_KERNEL);
+ if (!pm_info)
+ return -ENOMEM;
+
+ ret = hcall_get_cpu_state(cmd, virt_to_phys(pm_info));
+ if (ret < 0) {
+ kfree(pm_info);
+ break;
+ }
+
+ if (copy_to_user(uptr, pm_info, sizeof(u64)))
+ ret = -EFAULT;
+ kfree(pm_info);
+ break;
+ case ACRN_PMCMD_GET_PX_DATA:
+ px_data = kmalloc(sizeof(*px_data), GFP_KERNEL);
+ if (!px_data)
+ return -ENOMEM;
+
+ ret = hcall_get_cpu_state(cmd, virt_to_phys(px_data));
+ if (ret < 0) {
+ kfree(px_data);
+ break;
+ }
+
+ if (copy_to_user(uptr, px_data, sizeof(*px_data)))
+ ret = -EFAULT;
+ kfree(px_data);
+ break;
+ case ACRN_PMCMD_GET_CX_DATA:
+ cx_data = kmalloc(sizeof(*cx_data), GFP_KERNEL);
+ if (!cx_data)
+ return -ENOMEM;
+
+ ret = hcall_get_cpu_state(cmd, virt_to_phys(cx_data));
+ if (ret < 0) {
+ kfree(cx_data);
+ break;
+ }
+
+ if (copy_to_user(uptr, cx_data, sizeof(*cx_data)))
+ ret = -EFAULT;
+ kfree(cx_data);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
/*
* HSM relies on hypercall layer of the ACRN hypervisor to do the
* sanity check against the input parameters.
@@ -54,6 +115,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_msi_entry *msi;
struct acrn_pcidev *pcidev;
struct page *page;
+ u64 cstate_cmd;
int ret = 0;
if (vm->vmid == ACRN_INVALID_VMID && cmd != ACRN_IOCTL_CREATE_VM) {
@@ -246,6 +308,13 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
case ACRN_IOCTL_CLEAR_VM_IOREQ:
acrn_ioreq_request_clear(vm);
break;
+ case ACRN_IOCTL_PM_GET_CPU_STATE:
+ if (copy_from_user(&cstate_cmd, (void *)ioctl_param,
+ sizeof(cstate_cmd)))
+ return -EFAULT;
+
+ ret = pmcmd_ioctl(cstate_cmd, (void __user *)ioctl_param);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index a8813397a3fe..e640632366f0 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -39,6 +39,9 @@
#define HC_ASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x05)
#define HC_DEASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x06)
+#define HC_ID_PM_BASE 0x80UL
+#define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00)
+
/**
* hcall_create_vm() - Create a User VM
* @vminfo: Service VM GPA of info of User VM creation
@@ -225,4 +228,13 @@ static inline long hcall_reset_ptdev_intr(u64 vmid, u64 irq)
return acrn_hypercall2(HC_RESET_PTDEV_INTR, vmid, irq);
}
+/*
+ * hcall_get_cpu_state() - Get P-states and C-states info from the hypervisor
+ * @state: Service VM GPA of buffer of P-states and C-states
+ */
+static inline long hcall_get_cpu_state(u64 cmd, u64 state)
+{
+ return acrn_hypercall2(HC_PM_GET_CPU_STATE, cmd, state);
+}
+
#endif /* __ACRN_HSM_HYPERCALL_H */
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index 959dfbccc6a4..cb584f61056b 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -346,6 +346,38 @@ struct acrn_msi_entry {
__u64 msi_data;
};
+struct acrn_acpi_generic_address {
+ __u8 space_id;
+ __u8 bit_width;
+ __u8 bit_offset;
+ __u8 access_size;
+ __u64 address;
+} __attribute__ ((__packed__));
+
+struct acrn_cstate_data {
+ struct acrn_acpi_generic_address cx_reg;
+ __u8 type;
+ __u32 latency;
+ __u64 power;
+};
+
+struct acrn_pstate_data {
+ __u64 core_frequency;
+ __u64 power;
+ __u64 transition_latency;
+ __u64 bus_master_latency;
+ __u64 control;
+ __u64 status;
+};
+
+#define PMCMD_TYPE_MASK 0x000000ff
+enum acrn_pm_cmd_type {
+ ACRN_PMCMD_GET_PX_CNT,
+ ACRN_PMCMD_GET_PX_DATA,
+ ACRN_PMCMD_GET_CX_CNT,
+ ACRN_PMCMD_GET_CX_DATA,
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -397,4 +429,7 @@ struct acrn_msi_entry {
#define ACRN_IOCTL_DEASSIGN_PCIDEV \
_IOW(ACRN_IOCTL_TYPE, 0x56, struct acrn_pcidev)
+#define ACRN_IOCTL_PM_GET_CPU_STATE \
+ _IOWR(ACRN_IOCTL_TYPE, 0x60, __u64)
+
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 14/18] virt: acrn: Introduce I/O ranges operation interfaces
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (12 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 13/18] virt: acrn: Introduce interfaces to query C-states and P-states allowed by hypervisor shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 15/18] virt: acrn: Introduce ioeventfd shuo.a.liu
` (3 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
An I/O request of a User VM, which is constructed by hypervisor, is
distributed by the ACRN Hypervisor Service Module to an I/O client
corresponding to the address range of the I/O request.
I/O client maintains a list of address ranges. Introduce
acrn_ioreq_range_{add,del}() to manage these address ranges.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/acrn_drv.h | 4 +++
drivers/virt/acrn/ioreq.c | 60 ++++++++++++++++++++++++++++++++++++
2 files changed, 64 insertions(+)
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 542448c88026..862d195d4774 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -197,6 +197,10 @@ struct acrn_ioreq_client *acrn_ioreq_client_create(struct acrn_vm *vm,
void *data, bool is_default,
const char *name);
void acrn_ioreq_client_destroy(struct acrn_ioreq_client *client);
+int acrn_ioreq_range_add(struct acrn_ioreq_client *client,
+ u32 type, u64 start, u64 end);
+void acrn_ioreq_range_del(struct acrn_ioreq_client *client,
+ u32 type, u64 start, u64 end);
int acrn_msi_inject(struct acrn_vm *vm, u64 msi_addr, u64 msi_data);
diff --git a/drivers/virt/acrn/ioreq.c b/drivers/virt/acrn/ioreq.c
index b469f273e20c..7d391c026437 100644
--- a/drivers/virt/acrn/ioreq.c
+++ b/drivers/virt/acrn/ioreq.c
@@ -101,6 +101,66 @@ int acrn_ioreq_request_default_complete(struct acrn_vm *vm, u16 vcpu)
return ret;
}
+/**
+ * acrn_ioreq_range_add() - Add an iorange monitored by an ioreq client
+ * @client: The ioreq client
+ * @type: Type (ACRN_IOREQ_TYPE_MMIO or ACRN_IOREQ_TYPE_PORTIO)
+ * @start: Start address of iorange
+ * @end: End address of iorange
+ *
+ * Return: 0 on success, <0 on error
+ */
+int acrn_ioreq_range_add(struct acrn_ioreq_client *client,
+ u32 type, u64 start, u64 end)
+{
+ struct acrn_ioreq_range *range;
+
+ if (end < start) {
+ dev_err(acrn_dev.this_device,
+ "Invalid IO range [0x%llx,0x%llx]\n", start, end);
+ return -EINVAL;
+ }
+
+ range = kzalloc(sizeof(*range), GFP_KERNEL);
+ if (!range)
+ return -ENOMEM;
+
+ range->type = type;
+ range->start = start;
+ range->end = end;
+
+ write_lock_bh(&client->range_lock);
+ list_add(&range->list, &client->range_list);
+ write_unlock_bh(&client->range_lock);
+
+ return 0;
+}
+
+/**
+ * acrn_ioreq_range_del() - Del an iorange monitored by an ioreq client
+ * @client: The ioreq client
+ * @type: Type (ACRN_IOREQ_TYPE_MMIO or ACRN_IOREQ_TYPE_PORTIO)
+ * @start: Start address of iorange
+ * @end: End address of iorange
+ */
+void acrn_ioreq_range_del(struct acrn_ioreq_client *client,
+ u32 type, u64 start, u64 end)
+{
+ struct acrn_ioreq_range *range;
+
+ write_lock_bh(&client->range_lock);
+ list_for_each_entry(range, &client->range_list, list) {
+ if (type == range->type &&
+ start == range->start &&
+ end == range->end) {
+ list_del(&range->list);
+ kfree(range);
+ break;
+ }
+ }
+ write_unlock_bh(&client->range_lock);
+}
+
/*
* ioreq_task() is the execution entity of handler thread of an I/O client.
* The handler callback of the I/O client is called within the handler thread.
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 15/18] virt: acrn: Introduce ioeventfd
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (13 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 14/18] virt: acrn: Introduce I/O ranges operation interfaces shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 16/18] virt: acrn: Introduce irqfd shuo.a.liu
` (2 subsequent siblings)
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
ioeventfd is a mechanism to register PIO/MMIO regions to trigger an
eventfd signal when written to by a User VM. ACRN userspace can register
any arbitrary I/O address with a corresponding eventfd and then pass the
eventfd to a specific end-point of interest for handling.
Vhost is a kernel-level virtio server which uses eventfd for signalling.
To support vhost on ACRN, ioeventfd is introduced in HSM.
A new I/O client dedicated to ioeventfd is associated with a User VM
during VM creation. HSM provides ioctls to associate an I/O region with
a eventfd. The I/O client signals a eventfd once its corresponding I/O
region is matched with an I/O request.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/Kconfig | 1 +
drivers/virt/acrn/Makefile | 2 +-
drivers/virt/acrn/acrn_drv.h | 10 ++
drivers/virt/acrn/hsm.c | 8 +
drivers/virt/acrn/ioeventfd.c | 273 ++++++++++++++++++++++++++++++++++
drivers/virt/acrn/vm.c | 2 +
include/uapi/linux/acrn.h | 29 ++++
7 files changed, 324 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/acrn/ioeventfd.c
diff --git a/drivers/virt/acrn/Kconfig b/drivers/virt/acrn/Kconfig
index 36c80378c30c..3e1a61c9d8d8 100644
--- a/drivers/virt/acrn/Kconfig
+++ b/drivers/virt/acrn/Kconfig
@@ -2,6 +2,7 @@
config ACRN_HSM
tristate "ACRN Hypervisor Service Module"
depends on ACRN_GUEST
+ select EVENTFD
help
ACRN Hypervisor Service Module (HSM) is a kernel module which
communicates with ACRN userspace through ioctls and talks to
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index 21721cbf6a80..755b583b32ca 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o vm.o mm.o ioreq.o
+acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 862d195d4774..74c609cf60ae 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -156,6 +156,9 @@ extern rwlock_t acrn_vm_list_lock;
* @ioreq_page: The page of the I/O request shared buffer
* @pci_conf_addr: Address of a PCI configuration access emulation
* @monitor_page: Page of interrupt statistics of User VM
+ * @ioeventfds_lock: Lock to protect ioeventfds list
+ * @ioeventfds: List to link all hsm_ioeventfd
+ * @ioeventfd_client: I/O client for ioeventfds of the VM
*/
struct acrn_vm {
struct list_head list;
@@ -172,6 +175,9 @@ struct acrn_vm {
struct page *ioreq_page;
u32 pci_conf_addr;
struct page *monitor_page;
+ struct mutex ioeventfds_lock;
+ struct list_head ioeventfds;
+ struct acrn_ioreq_client *ioeventfd_client;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -204,4 +210,8 @@ void acrn_ioreq_range_del(struct acrn_ioreq_client *client,
int acrn_msi_inject(struct acrn_vm *vm, u64 msi_addr, u64 msi_data);
+int acrn_ioeventfd_init(struct acrn_vm *vm);
+int acrn_ioeventfd_config(struct acrn_vm *vm, struct acrn_ioeventfd *args);
+void acrn_ioeventfd_deinit(struct acrn_vm *vm);
+
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index 700006e2465b..e0d48a512017 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -111,6 +111,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_vcpu_regs *cpu_regs;
struct acrn_ioreq_notify notify;
struct acrn_ptdev_irq *irq_info;
+ struct acrn_ioeventfd ioeventfd;
struct acrn_vm_memmap memmap;
struct acrn_msi_entry *msi;
struct acrn_pcidev *pcidev;
@@ -315,6 +316,13 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
ret = pmcmd_ioctl(cstate_cmd, (void __user *)ioctl_param);
break;
+ case ACRN_IOCTL_IOEVENTFD:
+ if (copy_from_user(&ioeventfd, (void __user *)ioctl_param,
+ sizeof(ioeventfd)))
+ return -EFAULT;
+
+ ret = acrn_ioeventfd_config(vm, &ioeventfd);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
diff --git a/drivers/virt/acrn/ioeventfd.c b/drivers/virt/acrn/ioeventfd.c
new file mode 100644
index 000000000000..ac4037e9f947
--- /dev/null
+++ b/drivers/virt/acrn/ioeventfd.c
@@ -0,0 +1,273 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN HSM eventfd - use eventfd objects to signal expected I/O requests
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Shuo Liu <shuo.a.liu@intel.com>
+ * Yakui Zhao <yakui.zhao@intel.com>
+ */
+
+#include <linux/eventfd.h>
+#include <linux/slab.h>
+
+#include "acrn_drv.h"
+
+/**
+ * struct hsm_ioeventfd - Properties of HSM ioeventfd
+ * @list: Entry within &acrn_vm.ioeventfds of ioeventfds of a VM
+ * @eventfd: Eventfd of the HSM ioeventfd
+ * @addr: Address of I/O range
+ * @data: Data for matching
+ * @length: Length of I/O range
+ * @type: Type of I/O range (ACRN_IOREQ_TYPE_MMIO/ACRN_IOREQ_TYPE_PORTIO)
+ * @wildcard: Data matching or not
+ */
+struct hsm_ioeventfd {
+ struct list_head list;
+ struct eventfd_ctx *eventfd;
+ u64 addr;
+ u64 data;
+ int length;
+ int type;
+ bool wildcard;
+};
+
+static inline int ioreq_type_from_flags(int flags)
+{
+ return flags & ACRN_IOEVENTFD_FLAG_PIO ?
+ ACRN_IOREQ_TYPE_PORTIO : ACRN_IOREQ_TYPE_MMIO;
+}
+
+static void acrn_ioeventfd_shutdown(struct acrn_vm *vm, struct hsm_ioeventfd *p)
+{
+ lockdep_assert_held(&vm->ioeventfds_lock);
+
+ eventfd_ctx_put(p->eventfd);
+ list_del(&p->list);
+ kfree(p);
+}
+
+static bool hsm_ioeventfd_is_conflict(struct acrn_vm *vm,
+ struct hsm_ioeventfd *ioeventfd)
+{
+ struct hsm_ioeventfd *p;
+
+ lockdep_assert_held(&vm->ioeventfds_lock);
+
+ /* Either one is wildcard, the data matching will be skipped. */
+ list_for_each_entry(p, &vm->ioeventfds, list)
+ if (p->eventfd == ioeventfd->eventfd &&
+ p->addr == ioeventfd->addr &&
+ p->type == ioeventfd->type &&
+ (p->wildcard || ioeventfd->wildcard ||
+ p->data == ioeventfd->data))
+ return true;
+
+ return false;
+}
+
+/*
+ * Assign an eventfd to a VM and create a HSM ioeventfd associated with the
+ * eventfd. The properties of the HSM ioeventfd are built from a &struct
+ * acrn_ioeventfd.
+ */
+static int acrn_ioeventfd_assign(struct acrn_vm *vm,
+ struct acrn_ioeventfd *args)
+{
+ struct eventfd_ctx *eventfd;
+ struct hsm_ioeventfd *p;
+ int ret;
+
+ /* Check for range overflow */
+ if (args->addr + args->len < args->addr)
+ return -EINVAL;
+
+ /*
+ * Currently, acrn_ioeventfd is used to support vhost. 1,2,4,8 width
+ * accesses can cover vhost's requirements.
+ */
+ if (!(args->len == 1 || args->len == 2 ||
+ args->len == 4 || args->len == 8))
+ return -EINVAL;
+
+ eventfd = eventfd_ctx_fdget(args->fd);
+ if (IS_ERR(eventfd))
+ return PTR_ERR(eventfd);
+
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (!p) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+
+ INIT_LIST_HEAD(&p->list);
+ p->addr = args->addr;
+ p->length = args->len;
+ p->eventfd = eventfd;
+ p->type = ioreq_type_from_flags(args->flags);
+
+ /*
+ * ACRN_IOEVENTFD_FLAG_DATAMATCH flag is set in virtio 1.0 support, the
+ * writing of notification register of each virtqueue may trigger the
+ * notification. There is no data matching requirement.
+ */
+ if (args->flags & ACRN_IOEVENTFD_FLAG_DATAMATCH)
+ p->data = args->data;
+ else
+ p->wildcard = true;
+
+ mutex_lock(&vm->ioeventfds_lock);
+
+ if (hsm_ioeventfd_is_conflict(vm, p)) {
+ ret = -EEXIST;
+ goto unlock_fail;
+ }
+
+ /* register the I/O range into ioreq client */
+ ret = acrn_ioreq_range_add(vm->ioeventfd_client, p->type,
+ p->addr, p->addr + p->length - 1);
+ if (ret < 0)
+ goto unlock_fail;
+
+ list_add_tail(&p->list, &vm->ioeventfds);
+ mutex_unlock(&vm->ioeventfds_lock);
+
+ return 0;
+
+unlock_fail:
+ mutex_unlock(&vm->ioeventfds_lock);
+ kfree(p);
+fail:
+ eventfd_ctx_put(eventfd);
+ return ret;
+}
+
+static int acrn_ioeventfd_deassign(struct acrn_vm *vm,
+ struct acrn_ioeventfd *args)
+{
+ struct hsm_ioeventfd *p;
+ struct eventfd_ctx *eventfd;
+
+ eventfd = eventfd_ctx_fdget(args->fd);
+ if (IS_ERR(eventfd))
+ return PTR_ERR(eventfd);
+
+ mutex_lock(&vm->ioeventfds_lock);
+ list_for_each_entry(p, &vm->ioeventfds, list) {
+ if (p->eventfd != eventfd)
+ continue;
+
+ acrn_ioreq_range_del(vm->ioeventfd_client, p->type,
+ p->addr, p->addr + p->length - 1);
+ acrn_ioeventfd_shutdown(vm, p);
+ break;
+ }
+ mutex_unlock(&vm->ioeventfds_lock);
+
+ eventfd_ctx_put(eventfd);
+ return 0;
+}
+
+static struct hsm_ioeventfd *hsm_ioeventfd_match(struct acrn_vm *vm, u64 addr,
+ u64 data, int len, int type)
+{
+ struct hsm_ioeventfd *p = NULL;
+
+ lockdep_assert_held(&vm->ioeventfds_lock);
+
+ list_for_each_entry(p, &vm->ioeventfds, list) {
+ if (p->type == type && p->addr == addr && p->length >= len &&
+ (p->wildcard || p->data == data))
+ return p;
+ }
+
+ return NULL;
+}
+
+static int acrn_ioeventfd_handler(struct acrn_ioreq_client *client,
+ struct acrn_io_request *req)
+{
+ struct hsm_ioeventfd *p;
+ u64 addr, val;
+ int size;
+
+ if (req->type == ACRN_IOREQ_TYPE_MMIO) {
+ /*
+ * I/O requests are dispatched by range check only, so a
+ * acrn_ioreq_client need process both READ and WRITE accesses
+ * of same range. READ accesses are safe to be ignored here
+ * because virtio PCI devices write the notify registers for
+ * notification.
+ */
+ if (req->reqs.mmio_request.direction == ACRN_IOREQ_DIR_READ) {
+ /* reading does nothing and return 0 */
+ req->reqs.mmio_request.value = 0;
+ return 0;
+ }
+ addr = req->reqs.mmio_request.address;
+ size = req->reqs.mmio_request.size;
+ val = req->reqs.mmio_request.value;
+ } else {
+ if (req->reqs.pio_request.direction == ACRN_IOREQ_DIR_READ) {
+ /* reading does nothing and return 0 */
+ req->reqs.pio_request.value = 0;
+ return 0;
+ }
+ addr = req->reqs.pio_request.address;
+ size = req->reqs.pio_request.size;
+ val = req->reqs.pio_request.value;
+ }
+
+ mutex_lock(&client->vm->ioeventfds_lock);
+ p = hsm_ioeventfd_match(client->vm, addr, val, size, req->type);
+ if (p)
+ eventfd_signal(p->eventfd, 1);
+ mutex_unlock(&client->vm->ioeventfds_lock);
+
+ return 0;
+}
+
+int acrn_ioeventfd_config(struct acrn_vm *vm, struct acrn_ioeventfd *args)
+{
+ int ret;
+
+ if (args->flags & ACRN_IOEVENTFD_FLAG_DEASSIGN)
+ ret = acrn_ioeventfd_deassign(vm, args);
+ else
+ ret = acrn_ioeventfd_assign(vm, args);
+
+ return ret;
+}
+
+int acrn_ioeventfd_init(struct acrn_vm *vm)
+{
+ char name[ACRN_NAME_LEN];
+
+ mutex_init(&vm->ioeventfds_lock);
+ INIT_LIST_HEAD(&vm->ioeventfds);
+ snprintf(name, sizeof(name), "ioeventfd-%u", vm->vmid);
+ vm->ioeventfd_client = acrn_ioreq_client_create(vm,
+ acrn_ioeventfd_handler,
+ NULL, false, name);
+ if (!vm->ioeventfd_client) {
+ dev_err(acrn_dev.this_device, "Failed to create ioeventfd ioreq client!\n");
+ return -EINVAL;
+ }
+
+ dev_dbg(acrn_dev.this_device, "VM %u ioeventfd init.\n", vm->vmid);
+ return 0;
+}
+
+void acrn_ioeventfd_deinit(struct acrn_vm *vm)
+{
+ struct hsm_ioeventfd *p, *next;
+
+ dev_dbg(acrn_dev.this_device, "VM %u ioeventfd deinit.\n", vm->vmid);
+ acrn_ioreq_client_destroy(vm->ioeventfd_client);
+ mutex_lock(&vm->ioeventfds_lock);
+ list_for_each_entry_safe(p, next, &vm->ioeventfds, list)
+ acrn_ioeventfd_shutdown(vm, p);
+ mutex_unlock(&vm->ioeventfds_lock);
+}
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
index 38304aeef181..3c671b03b273 100644
--- a/drivers/virt/acrn/vm.c
+++ b/drivers/virt/acrn/vm.c
@@ -50,6 +50,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
list_add(&vm->list, &acrn_vm_list);
write_unlock_bh(&acrn_vm_list_lock);
+ acrn_ioeventfd_init(vm);
dev_dbg(acrn_dev.this_device, "VM %u created.\n", vm->vmid);
return vm;
}
@@ -67,6 +68,7 @@ int acrn_vm_destroy(struct acrn_vm *vm)
list_del_init(&vm->list);
write_unlock_bh(&acrn_vm_list_lock);
+ acrn_ioeventfd_deinit(vm);
acrn_ioreq_deinit(vm);
if (vm->monitor_page) {
put_page(vm->monitor_page);
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index cb584f61056b..d6d026f7454a 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -378,6 +378,32 @@ enum acrn_pm_cmd_type {
ACRN_PMCMD_GET_CX_DATA,
};
+#define ACRN_IOEVENTFD_FLAG_PIO 0x01
+#define ACRN_IOEVENTFD_FLAG_DATAMATCH 0x02
+#define ACRN_IOEVENTFD_FLAG_DEASSIGN 0x04
+/**
+ * struct acrn_ioeventfd - Data to operate a &struct hsm_ioeventfd
+ * @fd: The fd of eventfd associated with a hsm_ioeventfd
+ * @flags: Logical-OR of ACRN_IOEVENTFD_FLAG_*
+ * @addr: The start address of IO range of ioeventfd
+ * @len: The length of IO range of ioeventfd
+ * @reserved: Reserved
+ * @data: Data for data matching
+ *
+ * Without flag ACRN_IOEVENTFD_FLAG_DEASSIGN, ioctl ACRN_IOCTL_IOEVENTFD
+ * creates a &struct hsm_ioeventfd with properties originated from &struct
+ * acrn_ioeventfd. With flag ACRN_IOEVENTFD_FLAG_DEASSIGN, ioctl
+ * ACRN_IOCTL_IOEVENTFD destroys the &struct hsm_ioeventfd matching the fd.
+ */
+struct acrn_ioeventfd {
+ __u32 fd;
+ __u32 flags;
+ __u64 addr;
+ __u32 len;
+ __u32 reserved;
+ __u64 data;
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -432,4 +458,7 @@ enum acrn_pm_cmd_type {
#define ACRN_IOCTL_PM_GET_CPU_STATE \
_IOWR(ACRN_IOCTL_TYPE, 0x60, __u64)
+#define ACRN_IOCTL_IOEVENTFD \
+ _IOW(ACRN_IOCTL_TYPE, 0x70, struct acrn_ioeventfd)
+
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 16/18] virt: acrn: Introduce irqfd
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (14 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 15/18] virt: acrn: Introduce ioeventfd shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 17/18] virt: acrn: Introduce an interface for Service VM to control vCPU shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 18/18] sample/acrn: Introduce a sample of HSM ioctl interface usage shuo.a.liu
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
irqfd is a mechanism to inject a specific interrupt to a User VM using a
decoupled eventfd mechanism.
Vhost is a kernel-level virtio server which uses eventfd for interrupt
injection. To support vhost on ACRN, irqfd is introduced in HSM.
HSM provides ioctls to associate a virtual Message Signaled Interrupt
(MSI) with an eventfd. The corresponding virtual MSI will be injected
into a User VM once the eventfd got signal.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/Makefile | 2 +-
drivers/virt/acrn/acrn_drv.h | 10 ++
drivers/virt/acrn/hsm.c | 7 ++
drivers/virt/acrn/irqfd.c | 235 +++++++++++++++++++++++++++++++++++
drivers/virt/acrn/vm.c | 3 +
include/uapi/linux/acrn.h | 15 +++
6 files changed, 271 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/acrn/irqfd.c
diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index 755b583b32ca..08ce641dcfa1 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o
+acrn-y := hsm.o vm.o mm.o ioreq.o ioeventfd.o irqfd.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index 74c609cf60ae..be4b5b90277c 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -159,6 +159,9 @@ extern rwlock_t acrn_vm_list_lock;
* @ioeventfds_lock: Lock to protect ioeventfds list
* @ioeventfds: List to link all hsm_ioeventfd
* @ioeventfd_client: I/O client for ioeventfds of the VM
+ * @irqfds_lock: Lock to protect irqfds list
+ * @irqfds: List to link all hsm_irqfd
+ * @irqfd_wq: Workqueue for irqfd async shutdown
*/
struct acrn_vm {
struct list_head list;
@@ -178,6 +181,9 @@ struct acrn_vm {
struct mutex ioeventfds_lock;
struct list_head ioeventfds;
struct acrn_ioreq_client *ioeventfd_client;
+ struct mutex irqfds_lock;
+ struct list_head irqfds;
+ struct workqueue_struct *irqfd_wq;
};
struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
@@ -214,4 +220,8 @@ int acrn_ioeventfd_init(struct acrn_vm *vm);
int acrn_ioeventfd_config(struct acrn_vm *vm, struct acrn_ioeventfd *args);
void acrn_ioeventfd_deinit(struct acrn_vm *vm);
+int acrn_irqfd_init(struct acrn_vm *vm);
+int acrn_irqfd_config(struct acrn_vm *vm, struct acrn_irqfd *args);
+void acrn_irqfd_deinit(struct acrn_vm *vm);
+
#endif /* __ACRN_HSM_DRV_H */
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index e0d48a512017..b8c1255247e5 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -115,6 +115,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
struct acrn_vm_memmap memmap;
struct acrn_msi_entry *msi;
struct acrn_pcidev *pcidev;
+ struct acrn_irqfd irqfd;
struct page *page;
u64 cstate_cmd;
int ret = 0;
@@ -323,6 +324,12 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd,
ret = acrn_ioeventfd_config(vm, &ioeventfd);
break;
+ case ACRN_IOCTL_IRQFD:
+ if (copy_from_user(&irqfd, (void __user *)ioctl_param,
+ sizeof(irqfd)))
+ return -EFAULT;
+ ret = acrn_irqfd_config(vm, &irqfd);
+ break;
default:
dev_dbg(acrn_dev.this_device, "Unknown IOCTL 0x%x!\n", cmd);
ret = -ENOTTY;
diff --git a/drivers/virt/acrn/irqfd.c b/drivers/virt/acrn/irqfd.c
new file mode 100644
index 000000000000..a8766d528e29
--- /dev/null
+++ b/drivers/virt/acrn/irqfd.c
@@ -0,0 +1,235 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ACRN HSM irqfd: use eventfd objects to inject virtual interrupts
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ *
+ * Authors:
+ * Shuo Liu <shuo.a.liu@intel.com>
+ * Yakui Zhao <yakui.zhao@intel.com>
+ */
+
+#include <linux/eventfd.h>
+#include <linux/file.h>
+#include <linux/poll.h>
+#include <linux/slab.h>
+
+#include "acrn_drv.h"
+
+static LIST_HEAD(acrn_irqfd_clients);
+static DEFINE_MUTEX(acrn_irqfds_mutex);
+
+/**
+ * struct hsm_irqfd - Properties of HSM irqfd
+ * @vm: Associated VM pointer
+ * @wait: Entry of wait-queue
+ * @shutdown: Async shutdown work
+ * @eventfd: Associated eventfd
+ * @list: Entry within &acrn_vm.irqfds of irqfds of a VM
+ * @pt: Structure for select/poll on the associated eventfd
+ * @msi: MSI data
+ */
+struct hsm_irqfd {
+ struct acrn_vm *vm;
+ wait_queue_entry_t wait;
+ struct work_struct shutdown;
+ struct eventfd_ctx *eventfd;
+ struct list_head list;
+ poll_table pt;
+ struct acrn_msi_entry msi;
+};
+
+static void acrn_irqfd_inject(struct hsm_irqfd *irqfd)
+{
+ struct acrn_vm *vm = irqfd->vm;
+
+ acrn_msi_inject(vm, irqfd->msi.msi_addr,
+ irqfd->msi.msi_data);
+}
+
+static void hsm_irqfd_shutdown(struct hsm_irqfd *irqfd)
+{
+ u64 cnt;
+
+ lockdep_assert_held(&irqfd->vm->irqfds_lock);
+
+ /* remove from wait queue */
+ list_del_init(&irqfd->list);
+ eventfd_ctx_remove_wait_queue(irqfd->eventfd, &irqfd->wait, &cnt);
+ eventfd_ctx_put(irqfd->eventfd);
+ kfree(irqfd);
+}
+
+static void hsm_irqfd_shutdown_work(struct work_struct *work)
+{
+ struct hsm_irqfd *irqfd;
+ struct acrn_vm *vm;
+
+ irqfd = container_of(work, struct hsm_irqfd, shutdown);
+ vm = irqfd->vm;
+ mutex_lock(&vm->irqfds_lock);
+ if (!list_empty(&irqfd->list))
+ hsm_irqfd_shutdown(irqfd);
+ mutex_unlock(&vm->irqfds_lock);
+}
+
+/* Called with wqh->lock held and interrupts disabled */
+static int hsm_irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode,
+ int sync, void *key)
+{
+ unsigned long poll_bits = (unsigned long)key;
+ struct hsm_irqfd *irqfd;
+ struct acrn_vm *vm;
+
+ irqfd = container_of(wait, struct hsm_irqfd, wait);
+ vm = irqfd->vm;
+ if (poll_bits & POLLIN)
+ /* An event has been signaled, inject an interrupt */
+ acrn_irqfd_inject(irqfd);
+
+ if (poll_bits & POLLHUP)
+ /* Do shutdown work in thread to hold wqh->lock */
+ queue_work(vm->irqfd_wq, &irqfd->shutdown);
+
+ return 0;
+}
+
+static void hsm_irqfd_poll_func(struct file *file, wait_queue_head_t *wqh,
+ poll_table *pt)
+{
+ struct hsm_irqfd *irqfd;
+
+ irqfd = container_of(pt, struct hsm_irqfd, pt);
+ add_wait_queue(wqh, &irqfd->wait);
+}
+
+/*
+ * Assign an eventfd to a VM and create a HSM irqfd associated with the
+ * eventfd. The properties of the HSM irqfd are built from a &struct
+ * acrn_irqfd.
+ */
+static int acrn_irqfd_assign(struct acrn_vm *vm, struct acrn_irqfd *args)
+{
+ struct eventfd_ctx *eventfd = NULL;
+ struct hsm_irqfd *irqfd, *tmp;
+ unsigned int events;
+ struct fd f;
+ int ret = 0;
+
+ irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL);
+ if (!irqfd)
+ return -ENOMEM;
+
+ irqfd->vm = vm;
+ memcpy(&irqfd->msi, &args->msi, sizeof(args->msi));
+ INIT_LIST_HEAD(&irqfd->list);
+ INIT_WORK(&irqfd->shutdown, hsm_irqfd_shutdown_work);
+
+ f = fdget(args->fd);
+ if (!f.file) {
+ ret = -EBADF;
+ goto out;
+ }
+
+ eventfd = eventfd_ctx_fileget(f.file);
+ if (IS_ERR(eventfd)) {
+ ret = PTR_ERR(eventfd);
+ goto fail;
+ }
+
+ irqfd->eventfd = eventfd;
+
+ /*
+ * Install custom wake-up handling to be notified whenever underlying
+ * eventfd is signaled.
+ */
+ init_waitqueue_func_entry(&irqfd->wait, hsm_irqfd_wakeup);
+ init_poll_funcptr(&irqfd->pt, hsm_irqfd_poll_func);
+
+ mutex_lock(&vm->irqfds_lock);
+ list_for_each_entry(tmp, &vm->irqfds, list) {
+ if (irqfd->eventfd != tmp->eventfd)
+ continue;
+ ret = -EBUSY;
+ mutex_unlock(&vm->irqfds_lock);
+ goto fail;
+ }
+ list_add_tail(&irqfd->list, &vm->irqfds);
+ mutex_unlock(&vm->irqfds_lock);
+
+ /* Check the pending event in this stage */
+ events = f.file->f_op->poll(f.file, &irqfd->pt);
+
+ if (events & POLLIN)
+ acrn_irqfd_inject(irqfd);
+
+ fdput(f);
+ return 0;
+fail:
+ if (eventfd && !IS_ERR(eventfd))
+ eventfd_ctx_put(eventfd);
+
+ fdput(f);
+out:
+ kfree(irqfd);
+ return ret;
+}
+
+static int acrn_irqfd_deassign(struct acrn_vm *vm,
+ struct acrn_irqfd *args)
+{
+ struct hsm_irqfd *irqfd, *tmp;
+ struct eventfd_ctx *eventfd;
+
+ eventfd = eventfd_ctx_fdget(args->fd);
+ if (IS_ERR(eventfd))
+ return PTR_ERR(eventfd);
+
+ mutex_lock(&vm->irqfds_lock);
+ list_for_each_entry_safe(irqfd, tmp, &vm->irqfds, list) {
+ if (irqfd->eventfd == eventfd) {
+ hsm_irqfd_shutdown(irqfd);
+ break;
+ }
+ }
+ mutex_unlock(&vm->irqfds_lock);
+ eventfd_ctx_put(eventfd);
+
+ return 0;
+}
+
+int acrn_irqfd_config(struct acrn_vm *vm, struct acrn_irqfd *args)
+{
+ int ret;
+
+ if (args->flags & ACRN_IRQFD_FLAG_DEASSIGN)
+ ret = acrn_irqfd_deassign(vm, args);
+ else
+ ret = acrn_irqfd_assign(vm, args);
+
+ return ret;
+}
+
+int acrn_irqfd_init(struct acrn_vm *vm)
+{
+ INIT_LIST_HEAD(&vm->irqfds);
+ mutex_init(&vm->irqfds_lock);
+ vm->irqfd_wq = alloc_workqueue("acrn_irqfd-%u", 0, 0, vm->vmid);
+ if (!vm->irqfd_wq)
+ return -ENOMEM;
+
+ dev_dbg(acrn_dev.this_device, "VM %u irqfd init.\n", vm->vmid);
+ return 0;
+}
+
+void acrn_irqfd_deinit(struct acrn_vm *vm)
+{
+ struct hsm_irqfd *irqfd, *next;
+
+ dev_dbg(acrn_dev.this_device, "VM %u irqfd deinit.\n", vm->vmid);
+ destroy_workqueue(vm->irqfd_wq);
+ mutex_lock(&vm->irqfds_lock);
+ list_for_each_entry_safe(irqfd, next, &vm->irqfds, list)
+ hsm_irqfd_shutdown(irqfd);
+ mutex_unlock(&vm->irqfds_lock);
+}
diff --git a/drivers/virt/acrn/vm.c b/drivers/virt/acrn/vm.c
index 3c671b03b273..7f152a74b591 100644
--- a/drivers/virt/acrn/vm.c
+++ b/drivers/virt/acrn/vm.c
@@ -51,6 +51,7 @@ struct acrn_vm *acrn_vm_create(struct acrn_vm *vm,
write_unlock_bh(&acrn_vm_list_lock);
acrn_ioeventfd_init(vm);
+ acrn_irqfd_init(vm);
dev_dbg(acrn_dev.this_device, "VM %u created.\n", vm->vmid);
return vm;
}
@@ -69,7 +70,9 @@ int acrn_vm_destroy(struct acrn_vm *vm)
write_unlock_bh(&acrn_vm_list_lock);
acrn_ioeventfd_deinit(vm);
+ acrn_irqfd_deinit(vm);
acrn_ioreq_deinit(vm);
+
if (vm->monitor_page) {
put_page(vm->monitor_page);
vm->monitor_page = NULL;
diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h
index d6d026f7454a..0309b087829f 100644
--- a/include/uapi/linux/acrn.h
+++ b/include/uapi/linux/acrn.h
@@ -404,6 +404,19 @@ struct acrn_ioeventfd {
__u64 data;
};
+#define ACRN_IRQFD_FLAG_DEASSIGN 0x01
+/**
+ * struct acrn_irqfd - Data to operate a &struct hsm_irqfd
+ * @fd: The fd of eventfd associated with a hsm_irqfd
+ * @flags: Logical-OR of ACRN_IRQFD_FLAG_*
+ * @msi: Info of MSI associated with the irqfd
+ */
+struct acrn_irqfd {
+ __s32 fd;
+ __u32 flags;
+ struct acrn_msi_entry msi;
+};
+
/* The ioctl type, documented in ioctl-number.rst */
#define ACRN_IOCTL_TYPE 0xA2
@@ -460,5 +473,7 @@ struct acrn_ioeventfd {
#define ACRN_IOCTL_IOEVENTFD \
_IOW(ACRN_IOCTL_TYPE, 0x70, struct acrn_ioeventfd)
+#define ACRN_IOCTL_IRQFD \
+ _IOW(ACRN_IOCTL_TYPE, 0x71, struct acrn_irqfd)
#endif /* _UAPI_ACRN_H */
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 17/18] virt: acrn: Introduce an interface for Service VM to control vCPU
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (15 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 16/18] virt: acrn: Introduce irqfd shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 18/18] sample/acrn: Introduce a sample of HSM ioctl interface usage shuo.a.liu
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu, Zhi Wang, Zhenyu Wang
From: Shuo Liu <shuo.a.liu@intel.com>
ACRN supports partition mode to achieve real-time requirements. In
partition mode, a CPU core can be dedicated to a vCPU of User VM. The
local APIC of the dedicated CPU core can be passthrough to the User VM.
The Service VM controls the assignment of the CPU cores.
Introduce an interface for the Service VM to remove the control of CPU
core from hypervisor perspective so that the CPU core can be a dedicated
CPU core of User VM.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yu Wang <yu1.wang@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/virt/acrn/hsm.c | 48 +++++++++++++++++++++++++++++++++++
drivers/virt/acrn/hypercall.h | 14 ++++++++++
2 files changed, 62 insertions(+)
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index b8c1255247e5..1bee07a9d3f5 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -9,6 +9,7 @@
* Yakui Zhao <yakui.zhao@intel.com>
*/
+#include <linux/cpu.h>
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/module.h>
@@ -347,6 +348,52 @@ static int acrn_dev_release(struct inode *inode, struct file *filp)
return 0;
}
+static ssize_t remove_cpu_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ u64 cpu, lapicid;
+ int ret;
+
+ if (kstrtoull(buf, 0, &cpu) < 0)
+ return -EINVAL;
+
+ if (cpu >= num_possible_cpus() || cpu == 0 || !cpu_is_hotpluggable(cpu))
+ return -EINVAL;
+
+ if (cpu_online(cpu))
+ remove_cpu(cpu);
+
+ lapicid = cpu_data(cpu).apicid;
+ dev_dbg(dev, "Try to remove cpu %lld with lapicid %lld\n", cpu, lapicid);
+ ret = hcall_sos_remove_cpu(lapicid);
+ if (ret < 0) {
+ dev_err(dev, "Failed to remove cpu %lld!\n", cpu);
+ goto fail_remove;
+ }
+
+ return count;
+
+fail_remove:
+ add_cpu(cpu);
+ return ret;
+}
+static DEVICE_ATTR_WO(remove_cpu);
+
+static struct attribute *acrn_attrs[] = {
+ &dev_attr_remove_cpu.attr,
+ NULL
+};
+
+static struct attribute_group acrn_attr_group = {
+ .attrs = acrn_attrs,
+};
+
+static const struct attribute_group *acrn_attr_groups[] = {
+ &acrn_attr_group,
+ NULL
+};
+
static const struct file_operations acrn_fops = {
.owner = THIS_MODULE,
.open = acrn_dev_open,
@@ -358,6 +405,7 @@ struct miscdevice acrn_dev = {
.minor = MISC_DYNAMIC_MINOR,
.name = "acrn_hsm",
.fops = &acrn_fops,
+ .groups = acrn_attr_groups,
};
static int __init hsm_init(void)
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index e640632366f0..0cfad05bd1a9 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -13,6 +13,9 @@
#define HC_ID 0x80UL
+#define HC_ID_GEN_BASE 0x0UL
+#define HC_SOS_REMOVE_CPU _HC_ID(HC_ID, HC_ID_GEN_BASE + 0x01)
+
#define HC_ID_VM_BASE 0x10UL
#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00)
#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01)
@@ -42,6 +45,17 @@
#define HC_ID_PM_BASE 0x80UL
#define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00)
+/**
+ * hcall_sos_remove_cpu() - Remove a vCPU of Service VM
+ * @cpu: The vCPU to be removed
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_sos_remove_cpu(u64 cpu)
+{
+ return acrn_hypercall1(HC_SOS_REMOVE_CPU, cpu);
+}
+
/**
* hcall_create_vm() - Create a User VM
* @vminfo: Service VM GPA of info of User VM creation
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v6 18/18] sample/acrn: Introduce a sample of HSM ioctl interface usage
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
` (16 preceding siblings ...)
2020-12-01 9:38 ` [PATCH v6 17/18] virt: acrn: Introduce an interface for Service VM to control vCPU shuo.a.liu
@ 2020-12-01 9:38 ` shuo.a.liu
17 siblings, 0 replies; 30+ messages in thread
From: shuo.a.liu @ 2020-12-01 9:38 UTC (permalink / raw)
To: linux-kernel, x86
Cc: Greg Kroah-Hartman, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Sean Christopherson, Yu Wang,
Reinette Chatre, Shuo Liu
From: Shuo Liu <shuo.a.liu@intel.com>
Launch a simple guest (with several instructions as payload) on ACRN
with demonstration ioctl usage.
Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
---
samples/acrn/Makefile | 12 ++++
samples/acrn/guest.ld | 9 +++
samples/acrn/payload.ld | 9 +++
samples/acrn/vm-sample.c | 136 +++++++++++++++++++++++++++++++++++++++
4 files changed, 166 insertions(+)
create mode 100644 samples/acrn/Makefile
create mode 100644 samples/acrn/guest.ld
create mode 100644 samples/acrn/payload.ld
create mode 100644 samples/acrn/vm-sample.c
diff --git a/samples/acrn/Makefile b/samples/acrn/Makefile
new file mode 100644
index 000000000000..c8e3ed9785e9
--- /dev/null
+++ b/samples/acrn/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0
+
+.PHONY: vm-sample
+
+vm-sample: vm-sample.o payload.o
+ $(CC) $^ -o $@
+
+payload.o: payload.ld guest16.o
+ $(LD) -T $< -o $@
+
+clean:
+ rm *.o vm-sample
diff --git a/samples/acrn/guest.ld b/samples/acrn/guest.ld
new file mode 100644
index 000000000000..5127c682bd22
--- /dev/null
+++ b/samples/acrn/guest.ld
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+OUTPUT_FORMAT(binary)
+SECTIONS
+{
+ .start : { *(.start) }
+ .text : { *(.text*) }
+ .rodata : { *(.rodata) }
+ .data : { *(.data) }
+}
diff --git a/samples/acrn/payload.ld b/samples/acrn/payload.ld
new file mode 100644
index 000000000000..e8d9a498ad62
--- /dev/null
+++ b/samples/acrn/payload.ld
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+SECTIONS
+{
+ .payload16 0 : {
+ guest16 = .;
+ guest16.o(.text)
+ guest16_end = .;
+ }
+}
diff --git a/samples/acrn/vm-sample.c b/samples/acrn/vm-sample.c
new file mode 100644
index 000000000000..b2dad47a77a0
--- /dev/null
+++ b/samples/acrn/vm-sample.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A sample program to run a User VM on the ACRN hypervisor
+ *
+ * This sample runs in a Service VM, which is a privileged VM of ACRN.
+ * CONFIG_ACRN_HSM need to be enabled in the Service VM.
+ *
+ * Guest VM code in guest16.s will be executed after the VM launched.
+ *
+ * Copyright (C) 2020 Intel Corporation. All rights reserved.
+ */
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <malloc.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <signal.h>
+#include <sys/ioctl.h>
+#include <linux/acrn.h>
+
+#define GUEST_MEMORY_SIZE (1024*1024)
+void *guest_memory;
+
+extern const unsigned char guest16[], guest16_end[];
+static char io_request_page[4096] __attribute__((aligned(4096)));
+static struct acrn_io_request *io_req_buf = (struct acrn_io_request *)io_request_page;
+
+__u16 vcpu_num;
+__u16 vmid;
+/* POST_STANDARD_VM_UUID1, refer to https://github.com/projectacrn/acrn-hypervisor/blob/master/hypervisor/include/common/vm_uuids.h */
+guid_t vm_uuid = GUID_INIT(0x385479d2, 0xd625, 0xe811, 0x86, 0x4e, 0xcb, 0x7a, 0x18, 0xb3, 0x46, 0x43);
+
+int hsm_fd;
+int is_running = 1;
+
+void vm_exit(int sig)
+{
+ sig = sig;
+
+ is_running = 0;
+ ioctl(hsm_fd, ACRN_IOCTL_PAUSE_VM, vmid);
+ ioctl(hsm_fd, ACRN_IOCTL_DESTROY_IOREQ_CLIENT, 0);
+}
+
+int main(int argc, char **argv)
+{
+ int vcpu_id, ret;
+ struct acrn_vm_creation create_vm = {0};
+ struct acrn_vm_memmap ram_map = {0};
+ struct acrn_vcpu_regs regs;
+ struct acrn_io_request *io_req;
+ struct acrn_ioreq_notify __attribute__((aligned(8))) notify;
+
+ argc = argc;
+ argv = argv;
+
+ guest_memory = memalign(4096, GUEST_MEMORY_SIZE);
+ if (!guest_memory) {
+ printf("No enough memory!\n");
+ return -1;
+ }
+ hsm_fd = open("/dev/acrn_hsm", O_RDWR|O_CLOEXEC);
+
+ memcpy(&create_vm.uuid, &vm_uuid, 16);
+ create_vm.ioreq_buf = (__u64)io_req_buf;
+ ret = ioctl(hsm_fd, ACRN_IOCTL_CREATE_VM, &create_vm);
+ printf("Created VM! [%d]\n", ret);
+ vcpu_num = create_vm.vcpu_num;
+ vmid = create_vm.vmid;
+
+ /* setup guest memory */
+ ram_map.type = ACRN_MEMMAP_RAM;
+ ram_map.vma_base = (__u64)guest_memory;
+ ram_map.len = GUEST_MEMORY_SIZE;
+ ram_map.user_vm_pa = 0;
+ ram_map.attr = ACRN_MEM_ACCESS_RWX;
+ ret = ioctl(hsm_fd, ACRN_IOCTL_SET_MEMSEG, &ram_map);
+ printf("Set up VM memory! [%d]\n", ret);
+
+ memcpy(guest_memory, guest16, guest16_end-guest16);
+
+ /* setup vcpu registers */
+ memset(®s, 0, sizeof(regs));
+ regs.vcpu_id = 0;
+ regs.vcpu_regs.rip = 0;
+
+ /* CR0_ET | CR0_NE */
+ regs.vcpu_regs.cr0 = 0x30U;
+ regs.vcpu_regs.cs_ar = 0x009FU;
+ regs.vcpu_regs.cs_sel = 0xF000U;
+ regs.vcpu_regs.cs_limit = 0xFFFFU;
+ regs.vcpu_regs.cs_base = 0 & 0xFFFF0000UL;
+ regs.vcpu_regs.rip = 0 & 0xFFFFUL;
+
+ ret = ioctl(hsm_fd, ACRN_IOCTL_SET_VCPU_REGS, ®s);
+ printf("Set up VM BSP registers! [%d]\n", ret);
+
+ /* create an ioreq client for this VM */
+ ret = ioctl(hsm_fd, ACRN_IOCTL_CREATE_IOREQ_CLIENT, 0);
+ printf("Created IO request client! [%d]\n", ret);
+
+ /* run vm */
+ ret = ioctl(hsm_fd, ACRN_IOCTL_START_VM, vmid);
+ printf("Start VM! [%d]\n", ret);
+
+ signal(SIGINT, vm_exit);
+ while (is_running) {
+ ret = ioctl(hsm_fd, ACRN_IOCTL_ATTACH_IOREQ_CLIENT, 0);
+
+ for (vcpu_id = 0; vcpu_id < vcpu_num; vcpu_id++) {
+ io_req = &io_req_buf[vcpu_id];
+ if ((__sync_add_and_fetch(&io_req->processed, 0) == ACRN_IOREQ_STATE_PROCESSING)
+ && (!io_req->kernel_handled))
+ if (io_req->type == ACRN_IOREQ_TYPE_PORTIO) {
+ int bytes, port, in;
+
+ port = io_req->reqs.pio_request.address;
+ bytes = io_req->reqs.pio_request.size;
+ in = (io_req->reqs.pio_request.direction == ACRN_IOREQ_DIR_READ);
+ printf("Guest VM %s PIO[%x] with size[%x]\n", in ? "read" : "write", port, bytes);
+
+ notify.vmid = vmid;
+ notify.vcpu = vcpu_id;
+ ioctl(hsm_fd, ACRN_IOCTL_NOTIFY_REQUEST_FINISH, ¬ify);
+ }
+ }
+ }
+
+ ret = ioctl(hsm_fd, ACRN_IOCTL_DESTROY_VM, NULL);
+ printf("Destroy VM! [%d]\n", ret);
+ close(hsm_fd);
+ free(guest_memory);
+ return 0;
+}
--
2.28.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-01 9:38 ` [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces shuo.a.liu
@ 2020-12-01 10:09 ` Greg Kroah-Hartman
2020-12-02 2:14 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-01 10:09 UTC (permalink / raw)
To: shuo.a.liu
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
> + vm_param->reserved0 = 0;
> + vm_param->reserved1 = 0;
NO!
This means that userspace can put whatever crud they want in those
fields, and you will happily zero it out. Then, when those reserved
fields are wanted to be used in the future, you will take those values
from userspace and accept them as a valid value. But, since userspace
was sending crud before, now you will take that crud and do something
with it.
TEST IT to verify that it is zero, that way userspace gets it right the
first time, and you don't get it wrong later, as you can not change it
later.
thaqnks,
greg k-h
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-01 10:09 ` Greg Kroah-Hartman
@ 2020-12-02 2:14 ` Shuo A Liu
2020-12-15 9:52 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Shuo A Liu @ 2020-12-02 2:14 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
>On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
>> + vm_param->reserved0 = 0;
>> + vm_param->reserved1 = 0;
>
>NO!
>
>This means that userspace can put whatever crud they want in those
>fields, and you will happily zero it out. Then, when those reserved
>fields are wanted to be used in the future, you will take those values
>from userspace and accept them as a valid value. But, since userspace
>was sending crud before, now you will take that crud and do something
>with it.
>
>TEST IT to verify that it is zero, that way userspace gets it right the
>first time, and you don't get it wrong later, as you can not change it
>later.
OK. Thanks for the elaboration. I will test it and return -EINVAL if
it is not zero.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 01/18] docs: acrn: Introduce ACRN
2020-12-01 9:38 ` [PATCH v6 01/18] docs: acrn: Introduce ACRN shuo.a.liu
@ 2020-12-08 11:52 ` Borislav Petkov
2020-12-09 1:36 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Borislav Petkov @ 2020-12-08 11:52 UTC (permalink / raw)
To: shuo.a.liu
Cc: linux-kernel, x86, Greg Kroah-Hartman, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, Sean Christopherson, Yu Wang,
Reinette Chatre, Zhi Wang, Dave Hansen, Dan Williams,
Fengwei Yin, Zhenyu Wang, Randy Dunlap
On Tue, Dec 01, 2020 at 05:38:36PM +0800, shuo.a.liu@intel.com wrote:
> From: Shuo Liu <shuo.a.liu@intel.com>
>
> Add documentation on the following aspects of ACRN:
Thanks for doing this!
> +===============
> +ACRN CPUID bits
> +===============
> +
> +A guest VM running on an ACRN hypervisor, can check some of its features using
> +cpuid.
s/cpuid/CPUID/g
> +ACRN Hypervisor Introduction
> +============================
> +
> +The ACRN Hypervisor is a Type 1 hypervisor, running directly on the bare-metal
s/the //
with those fixed:
Reviewed-by: Borislav Petkov <bp@suse.de>
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 01/18] docs: acrn: Introduce ACRN
2020-12-08 11:52 ` Borislav Petkov
@ 2020-12-09 1:36 ` Shuo A Liu
0 siblings, 0 replies; 30+ messages in thread
From: Shuo A Liu @ 2020-12-09 1:36 UTC (permalink / raw)
To: Borislav Petkov
Cc: linux-kernel, x86, Greg Kroah-Hartman, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, Sean Christopherson, Yu Wang,
Reinette Chatre, Zhi Wang, Dave Hansen, Dan Williams,
Fengwei Yin, Zhenyu Wang, Randy Dunlap
On Tue 8.Dec'20 at 12:52:49 +0100, Borislav Petkov wrote:
>On Tue, Dec 01, 2020 at 05:38:36PM +0800, shuo.a.liu@intel.com wrote:
>> From: Shuo Liu <shuo.a.liu@intel.com>
>>
>> Add documentation on the following aspects of ACRN:
>
>Thanks for doing this!
>
>> +===============
>> +ACRN CPUID bits
>> +===============
>> +
>> +A guest VM running on an ACRN hypervisor, can check some of its features using
>> +cpuid.
>
>s/cpuid/CPUID/g
>
>> +ACRN Hypervisor Introduction
>> +============================
>> +
>> +The ACRN Hypervisor is a Type 1 hypervisor, running directly on the bare-metal
>
>s/the //
>
>with those fixed:
>
>Reviewed-by: Borislav Petkov <bp@suse.de>
Thanks for review. Fixed.
Thanks
shuo
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-02 2:14 ` Shuo A Liu
@ 2020-12-15 9:52 ` Shuo A Liu
2020-12-15 10:00 ` Greg Kroah-Hartman
0 siblings, 1 reply; 30+ messages in thread
From: Shuo A Liu @ 2020-12-15 9:52 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Wed 2.Dec'20 at 10:14:29 +0800, Shuo A Liu wrote:
>On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
>>On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
>>>+ vm_param->reserved0 = 0;
>>>+ vm_param->reserved1 = 0;
>>
>>NO!
>>
>>This means that userspace can put whatever crud they want in those
>>fields, and you will happily zero it out. Then, when those reserved
>>fields are wanted to be used in the future, you will take those values
>>from userspace and accept them as a valid value. But, since userspace
>>was sending crud before, now you will take that crud and do something
>>with it.
>>
>>TEST IT to verify that it is zero, that way userspace gets it right the
>>first time, and you don't get it wrong later, as you can not change it
>>later.
>
>OK. Thanks for the elaboration. I will test it and return -EINVAL if
>it is not zero.
>
Hi Greg,
Would you like to review other patches in this series on this version?
This version got only this one comment, so doesn't have too much update.
I can send out a new version if you prefer to.
Thanks for review.
shuo
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-15 9:52 ` Shuo A Liu
@ 2020-12-15 10:00 ` Greg Kroah-Hartman
2020-12-15 10:02 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Greg Kroah-Hartman @ 2020-12-15 10:00 UTC (permalink / raw)
To: Shuo A Liu
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue, Dec 15, 2020 at 05:52:59PM +0800, Shuo A Liu wrote:
> On Wed 2.Dec'20 at 10:14:29 +0800, Shuo A Liu wrote:
> > On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
> > > On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
> > > > + vm_param->reserved0 = 0;
> > > > + vm_param->reserved1 = 0;
> > >
> > > NO!
> > >
> > > This means that userspace can put whatever crud they want in those
> > > fields, and you will happily zero it out. Then, when those reserved
> > > fields are wanted to be used in the future, you will take those values
> > > from userspace and accept them as a valid value. But, since userspace
> > > was sending crud before, now you will take that crud and do something
> > > with it.
> > >
> > > TEST IT to verify that it is zero, that way userspace gets it right the
> > > first time, and you don't get it wrong later, as you can not change it
> > > later.
> >
> > OK. Thanks for the elaboration. I will test it and return -EINVAL if
> > it is not zero.
> >
>
> Hi Greg,
>
> Would you like to review other patches in this series on this version?
Nope, it's the middle of the merge window, I can't do anything with any
new patches until after 5.11-rc1 is out. So I suggest you fix up the
current issues and send a new patch series once 5.11-rc1 is released.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-15 10:00 ` Greg Kroah-Hartman
@ 2020-12-15 10:02 ` Shuo A Liu
2021-01-05 14:03 ` Greg Kroah-Hartman
0 siblings, 1 reply; 30+ messages in thread
From: Shuo A Liu @ 2020-12-15 10:02 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue 15.Dec'20 at 11:00:57 +0100, Greg Kroah-Hartman wrote:
>On Tue, Dec 15, 2020 at 05:52:59PM +0800, Shuo A Liu wrote:
>> On Wed 2.Dec'20 at 10:14:29 +0800, Shuo A Liu wrote:
>> > On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
>> > > On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
>> > > > + vm_param->reserved0 = 0;
>> > > > + vm_param->reserved1 = 0;
>> > >
>> > > NO!
>> > >
>> > > This means that userspace can put whatever crud they want in those
>> > > fields, and you will happily zero it out. Then, when those reserved
>> > > fields are wanted to be used in the future, you will take those values
>> > > from userspace and accept them as a valid value. But, since userspace
>> > > was sending crud before, now you will take that crud and do something
>> > > with it.
>> > >
>> > > TEST IT to verify that it is zero, that way userspace gets it right the
>> > > first time, and you don't get it wrong later, as you can not change it
>> > > later.
>> >
>> > OK. Thanks for the elaboration. I will test it and return -EINVAL if
>> > it is not zero.
>> >
>>
>> Hi Greg,
>>
>> Would you like to review other patches in this series on this version?
>
>Nope, it's the middle of the merge window, I can't do anything with any
>new patches until after 5.11-rc1 is out. So I suggest you fix up the
>current issues and send a new patch series once 5.11-rc1 is released.
Got it, thanks!
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces
2020-12-01 9:38 ` [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces shuo.a.liu
@ 2020-12-30 17:33 ` Borislav Petkov
2021-01-06 7:52 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Borislav Petkov @ 2020-12-30 17:33 UTC (permalink / raw)
To: shuo.a.liu
Cc: linux-kernel, x86, Greg Kroah-Hartman, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, Sean Christopherson, Yu Wang,
Reinette Chatre, Yakui Zhao, Nick Desaulniers, Dave Hansen,
Dan Williams, Fengwei Yin, Zhi Wang, Zhenyu Wang, Arvind Sankar,
Peter Zijlstra, Segher Boessenkool
On Tue, Dec 01, 2020 at 05:38:39PM +0800, shuo.a.liu@intel.com wrote:
> From: Shuo Liu <shuo.a.liu@intel.com>
>
> The Service VM communicates with the hypervisor via conventional
> hypercalls. VMCALL instruction is used to make the hypercalls.
>
> ACRN hypercall ABI:
> * Hypercall number is in R8 register.
> * Up to 2 parameters are in RDI and RSI registers.
> * Return value is in RAX register.
>
> Introduce the ACRN hypercall interfaces. Because GCC doesn't support R8
> register as direct register constraints, use supported constraint as
> input with a explicit MOV to R8 in beginning of asm.
>
> Originally-by: Yakui Zhao <yakui.zhao@intel.com>
> Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
> Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Sean Christopherson <sean.j.christopherson@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Fengwei Yin <fengwei.yin@intel.com>
> Cc: Zhi Wang <zhi.a.wang@intel.com>
> Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
> Cc: Yu Wang <yu1.wang@intel.com>
> Cc: Reinette Chatre <reinette.chatre@intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Arvind Sankar <nivedita@alum.mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Nick Desaulniers <ndesaulniers@google.com>
> Cc: Segher Boessenkool <segher@kernel.crashing.org>
> ---
> arch/x86/include/asm/acrn.h | 54 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 54 insertions(+)
The x86 bits in patches 2-4 look ok now, thanks!
Acked-by: Borislav Petkov <bp@suse.de>
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2020-12-15 10:02 ` Shuo A Liu
@ 2021-01-05 14:03 ` Greg Kroah-Hartman
2021-01-06 7:56 ` Shuo A Liu
0 siblings, 1 reply; 30+ messages in thread
From: Greg Kroah-Hartman @ 2021-01-05 14:03 UTC (permalink / raw)
To: Shuo A Liu
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue, Dec 15, 2020 at 06:02:51PM +0800, Shuo A Liu wrote:
> On Tue 15.Dec'20 at 11:00:57 +0100, Greg Kroah-Hartman wrote:
> > On Tue, Dec 15, 2020 at 05:52:59PM +0800, Shuo A Liu wrote:
> > > On Wed 2.Dec'20 at 10:14:29 +0800, Shuo A Liu wrote:
> > > > On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
> > > > > On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
> > > > > > + vm_param->reserved0 = 0;
> > > > > > + vm_param->reserved1 = 0;
> > > > >
> > > > > NO!
> > > > >
> > > > > This means that userspace can put whatever crud they want in those
> > > > > fields, and you will happily zero it out. Then, when those reserved
> > > > > fields are wanted to be used in the future, you will take those values
> > > > > from userspace and accept them as a valid value. But, since userspace
> > > > > was sending crud before, now you will take that crud and do something
> > > > > with it.
> > > > >
> > > > > TEST IT to verify that it is zero, that way userspace gets it right the
> > > > > first time, and you don't get it wrong later, as you can not change it
> > > > > later.
> > > >
> > > > OK. Thanks for the elaboration. I will test it and return -EINVAL if
> > > > it is not zero.
> > > >
> > >
> > > Hi Greg,
> > >
> > > Would you like to review other patches in this series on this version?
> >
> > Nope, it's the middle of the merge window, I can't do anything with any
> > new patches until after 5.11-rc1 is out. So I suggest you fix up the
> > current issues and send a new patch series once 5.11-rc1 is released.
>
> Got it, thanks!
Did this ever happen? I don't see a new series anywhere, do you have a
lore.kernel.org link?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces
2020-12-30 17:33 ` Borislav Petkov
@ 2021-01-06 7:52 ` Shuo A Liu
0 siblings, 0 replies; 30+ messages in thread
From: Shuo A Liu @ 2021-01-06 7:52 UTC (permalink / raw)
To: Borislav Petkov
Cc: linux-kernel, x86, Greg Kroah-Hartman, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, Sean Christopherson, Yu Wang,
Reinette Chatre, Yakui Zhao, Nick Desaulniers, Dave Hansen,
Dan Williams, Fengwei Yin, Zhi Wang, Zhenyu Wang, Arvind Sankar,
Peter Zijlstra, Segher Boessenkool
On Wed 30.Dec'20 at 18:33:05 +0100, Borislav Petkov wrote:
>On Tue, Dec 01, 2020 at 05:38:39PM +0800, shuo.a.liu@intel.com wrote:
>> From: Shuo Liu <shuo.a.liu@intel.com>
>>
>> The Service VM communicates with the hypervisor via conventional
>> hypercalls. VMCALL instruction is used to make the hypercalls.
>>
>> ACRN hypercall ABI:
>> * Hypercall number is in R8 register.
>> * Up to 2 parameters are in RDI and RSI registers.
>> * Return value is in RAX register.
>>
>> Introduce the ACRN hypercall interfaces. Because GCC doesn't support R8
>> register as direct register constraints, use supported constraint as
>> input with a explicit MOV to R8 in beginning of asm.
>>
>> Originally-by: Yakui Zhao <yakui.zhao@intel.com>
>> Signed-off-by: Shuo Liu <shuo.a.liu@intel.com>
>> Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
>> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
>> Cc: Dave Hansen <dave.hansen@intel.com>
>> Cc: Sean Christopherson <sean.j.christopherson@intel.com>
>> Cc: Dan Williams <dan.j.williams@intel.com>
>> Cc: Fengwei Yin <fengwei.yin@intel.com>
>> Cc: Zhi Wang <zhi.a.wang@intel.com>
>> Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
>> Cc: Yu Wang <yu1.wang@intel.com>
>> Cc: Reinette Chatre <reinette.chatre@intel.com>
>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Cc: Borislav Petkov <bp@alien8.de>
>> Cc: Arvind Sankar <nivedita@alum.mit.edu>
>> Cc: Peter Zijlstra <peterz@infradead.org>
>> Cc: Nick Desaulniers <ndesaulniers@google.com>
>> Cc: Segher Boessenkool <segher@kernel.crashing.org>
>> ---
>> arch/x86/include/asm/acrn.h | 54 +++++++++++++++++++++++++++++++++++++
>> 1 file changed, 54 insertions(+)
>
>The x86 bits in patches 2-4 look ok now, thanks!
>
>Acked-by: Borislav Petkov <bp@suse.de>
Thanks for review!
Shuo
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces
2021-01-05 14:03 ` Greg Kroah-Hartman
@ 2021-01-06 7:56 ` Shuo A Liu
0 siblings, 0 replies; 30+ messages in thread
From: Shuo A Liu @ 2021-01-06 7:56 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: linux-kernel, x86, H . Peter Anvin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Sean Christopherson, Yu Wang, Reinette Chatre,
Zhi Wang, Zhenyu Wang
On Tue 5.Jan'21 at 15:03:06 +0100, Greg Kroah-Hartman wrote:
>On Tue, Dec 15, 2020 at 06:02:51PM +0800, Shuo A Liu wrote:
>> On Tue 15.Dec'20 at 11:00:57 +0100, Greg Kroah-Hartman wrote:
>> > On Tue, Dec 15, 2020 at 05:52:59PM +0800, Shuo A Liu wrote:
>> > > On Wed 2.Dec'20 at 10:14:29 +0800, Shuo A Liu wrote:
>> > > > On Tue 1.Dec'20 at 11:09:47 +0100, Greg Kroah-Hartman wrote:
>> > > > > On Tue, Dec 01, 2020 at 05:38:41PM +0800, shuo.a.liu@intel.com wrote:
>> > > > > > + vm_param->reserved0 = 0;
>> > > > > > + vm_param->reserved1 = 0;
>> > > > >
>> > > > > NO!
>> > > > >
>> > > > > This means that userspace can put whatever crud they want in those
>> > > > > fields, and you will happily zero it out. Then, when those reserved
>> > > > > fields are wanted to be used in the future, you will take those values
>> > > > > from userspace and accept them as a valid value. But, since userspace
>> > > > > was sending crud before, now you will take that crud and do something
>> > > > > with it.
>> > > > >
>> > > > > TEST IT to verify that it is zero, that way userspace gets it right the
>> > > > > first time, and you don't get it wrong later, as you can not change it
>> > > > > later.
>> > > >
>> > > > OK. Thanks for the elaboration. I will test it and return -EINVAL if
>> > > > it is not zero.
>> > > >
>> > >
>> > > Hi Greg,
>> > >
>> > > Would you like to review other patches in this series on this version?
>> >
>> > Nope, it's the middle of the merge window, I can't do anything with any
>> > new patches until after 5.11-rc1 is out. So I suggest you fix up the
>> > current issues and send a new patch series once 5.11-rc1 is released.
>>
>> Got it, thanks!
>
>Did this ever happen? I don't see a new series anywhere, do you have a
>lore.kernel.org link?
Just back to work. :)
Sent new series already. The link is
https://lore.kernel.org/lkml/20210106075055.47226-1-shuo.a.liu@intel.com/
Thanks
shuo
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2021-01-06 7:57 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-01 9:38 [PATCH v6 00/18] HSM driver for ACRN hypervisor shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 01/18] docs: acrn: Introduce ACRN shuo.a.liu
2020-12-08 11:52 ` Borislav Petkov
2020-12-09 1:36 ` Shuo A Liu
2020-12-01 9:38 ` [PATCH v6 02/18] x86/acrn: Introduce acrn_{setup, remove}_intr_handler() shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 03/18] x86/acrn: Introduce acrn_cpuid_base() and hypervisor feature bits shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 04/18] x86/acrn: Introduce hypercall interfaces shuo.a.liu
2020-12-30 17:33 ` Borislav Petkov
2021-01-06 7:52 ` Shuo A Liu
2020-12-01 9:38 ` [PATCH v6 05/18] virt: acrn: Introduce ACRN HSM basic driver shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 06/18] virt: acrn: Introduce VM management interfaces shuo.a.liu
2020-12-01 10:09 ` Greg Kroah-Hartman
2020-12-02 2:14 ` Shuo A Liu
2020-12-15 9:52 ` Shuo A Liu
2020-12-15 10:00 ` Greg Kroah-Hartman
2020-12-15 10:02 ` Shuo A Liu
2021-01-05 14:03 ` Greg Kroah-Hartman
2021-01-06 7:56 ` Shuo A Liu
2020-12-01 9:38 ` [PATCH v6 07/18] virt: acrn: Introduce an ioctl to set vCPU registers state shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 08/18] virt: acrn: Introduce EPT mapping management shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 09/18] virt: acrn: Introduce I/O request management shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 10/18] virt: acrn: Introduce PCI configuration space PIO accesses combiner shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 11/18] virt: acrn: Introduce interfaces for PCI device passthrough shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 12/18] virt: acrn: Introduce interrupt injection interfaces shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 13/18] virt: acrn: Introduce interfaces to query C-states and P-states allowed by hypervisor shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 14/18] virt: acrn: Introduce I/O ranges operation interfaces shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 15/18] virt: acrn: Introduce ioeventfd shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 16/18] virt: acrn: Introduce irqfd shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 17/18] virt: acrn: Introduce an interface for Service VM to control vCPU shuo.a.liu
2020-12-01 9:38 ` [PATCH v6 18/18] sample/acrn: Introduce a sample of HSM ioctl interface usage shuo.a.liu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).