From: Elliot Berman <quic_eberman@quicinc.com>
To: Alex Elder <elder@linaro.org>,
Srinivas Kandagatla <srinivas.kandagatla@linaro.org>,
Elliot Berman <quic_eberman@quicinc.com>,
Prakruthi Deepak Heragu <quic_pheragu@quicinc.com>,
Jonathan Corbet <corbet@lwn.net>
Cc: Murali Nalajala <quic_mnalajal@quicinc.com>,
Trilok Soni <quic_tsoni@quicinc.com>,
Srivatsa Vaddagiri <quic_svaddagi@quicinc.com>,
Carl van Schaik <quic_cvanscha@quicinc.com>,
Dmitry Baryshkov <dmitry.baryshkov@linaro.org>,
Bjorn Andersson <andersson@kernel.org>,
"Konrad Dybcio" <konrad.dybcio@linaro.org>,
Arnd Bergmann <arnd@arndb.de>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
Rob Herring <robh+dt@kernel.org>,
Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
Bagas Sanjaya <bagasdotme@gmail.com>,
Will Deacon <will@kernel.org>, Andy Gross <agross@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Jassi Brar <jassisinghbrar@gmail.com>,
<linux-arm-msm@vger.kernel.org>, <devicetree@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <linux-doc@vger.kernel.org>,
<linux-arm-kernel@lists.infradead.org>
Subject: [PATCH v11 01/26] docs: gunyah: Introduce Gunyah Hypervisor
Date: Fri, 3 Mar 2023 17:06:07 -0800 [thread overview]
Message-ID: <20230304010632.2127470-2-quic_eberman@quicinc.com> (raw)
In-Reply-To: <20230304010632.2127470-1-quic_eberman@quicinc.com>
Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It
does not depend on any lower-privileged OS/kernel code for its core
functionality. This increases its security and can support a smaller
trusted computing based when compared to Type-2 hypervisors.
Add documentation describing the Gunyah hypervisor and the main
components of the Gunyah hypervisor which are of interest to Linux
virtualization development.
Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
Documentation/virt/gunyah/index.rst | 113 ++++++++++++++++++++
Documentation/virt/gunyah/message-queue.rst | 63 +++++++++++
Documentation/virt/index.rst | 1 +
3 files changed, 177 insertions(+)
create mode 100644 Documentation/virt/gunyah/index.rst
create mode 100644 Documentation/virt/gunyah/message-queue.rst
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
new file mode 100644
index 000000000000..74aa345e0a14
--- /dev/null
+++ b/Documentation/virt/gunyah/index.rst
@@ -0,0 +1,113 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Gunyah Hypervisor
+=================
+
+.. toctree::
+ :maxdepth: 1
+
+ message-queue
+
+Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
+a higher CPU privilege level. It does not depend on any lower-privileged operating system
+for its core functionality. This increases its security and can support a much smaller
+trusted computing base than a Type-2 hypervisor.
+
+Gunyah is an open source hypervisor. The source repo is available at
+https://github.com/quic/gunyah-hypervisor.
+
+Gunyah provides these following features.
+
+- Scheduling:
+
+ A scheduler for virtual CPUs (vCPUs) on physical CPUs enables time-sharing
+ of the CPUs. Gunyah supports two models of scheduling:
+
+ 1. "Behind the back" scheduling in which Gunyah hypervisor schedules vCPUS on its own.
+ 2. "Proxy" scheduling in which a delegated VM can donate part of one of its vCPU slice
+ to another VM's vCPU via a hypercall.
+
+- Memory Management:
+
+ APIs handling memory, abstracted as objects, limiting direct use of physical
+ addresses. Memory ownership and usage tracking of all memory under its control.
+ Memory partitioning between VMs is a fundamental security feature.
+
+- Interrupt Virtualization:
+
+ Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled
+ in the hypervisor and routed to the assigned VM.
+
+- Inter-VM Communication:
+
+ There are several different mechanisms provided for communicating between VMs.
+
+- Virtual platform:
+
+ Architectural devices such as interrupt controllers and CPU timers are directly provided
+ by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI.
+
+- Device Virtualization:
+
+ Para-virtualization of devices is supported using inter-VM communication.
+
+Architectures supported
+=======================
+AArch64 with a GIC
+
+Resources and Capabilities
+==========================
+
+Some services or resources provided by the Gunyah hypervisor are described to a virtual machine by
+capability IDs. For instance, inter-VM communication is performed with doorbells and message queues.
+Gunyah allows access to manipulate that doorbell via the capability ID. These resources are
+described in Linux as a struct gh_resource.
+
+High level management of these resources is performed by the resource manager VM. RM informs a
+guest VM about resources it can access through either the device tree or via guest-initiated RPC.
+
+For each virtual machine, Gunyah maintains a table of resources which can be accessed by that VM.
+An entry in this table is called a "capability" and VMs can only access resources via this
+capability table. Hence, virtual Gunyah resources are referenced by a "capability IDs" and not
+"resource IDs". If 2 VMs have access to the same resource, they might not be using the same
+capability ID to access that resource since the capability tables are independent per VM.
+
+Resource Manager
+================
+
+The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor.
+It provides policy enforcement aspects of the virtualization system. The resource manager can
+be treated as an extension of the Hypervisor but is separated to its own partition to ensure
+that the hypervisor layer itself remains small and secure and to maintain a separation of policy
+and mechanism in the platform. RM runs at arm64 NS-EL1 similar to other virtual machines.
+
+Communication with the resource manager from each guest VM happens with message-queue.rst. Details
+about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
+
+::
+
+ +-------+ +--------+ +--------+
+ | RM | | VM_A | | VM_B |
+ +-.-.-.-+ +---.----+ +---.----+
+ | | | |
+ +-.-.-----------.------------.----+
+ | | \==========/ | |
+ | \========================/ |
+ | Gunyah |
+ +---------------------------------+
+
+The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager.
+
+The resource manager provides the following features:
+
+- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs
+- VM access control policy, including memory sharing and lending
+- Interrupt routing configuration
+- Forwarding of system-level events (e.g. VM shutdown) to owner VM
+
+When booting a virtual machine which uses a devicetree such as Linux, resource manager overlays a
+/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM,
+how to communicate with resource manager, and basic description and capabilities of
+this VM. See Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a description
+of this node.
diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
new file mode 100644
index 000000000000..b352918ae54b
--- /dev/null
+++ b/Documentation/virt/gunyah/message-queue.rst
@@ -0,0 +1,63 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Message Queues
+==============
+Message queue is a simple low-capacity IPC channel between two VMs. It is
+intended for sending small control and configuration messages. Each message
+queue is unidirectional, so a full-duplex IPC channel requires a pair of queues.
+
+Messages can be up to 240 bytes in length. Longer messages require a further
+protocol on top of the message queue messages themselves. For instance, communication
+with the resource manager adds a header field for sending longer messages via multiple
+message fragments.
+
+The diagram below shows how message queue works. A typical configuration involves
+2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message
+queue 2 allows VM_B to send messages to VM_A.
+
+1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall
+ with the message to inform the hypervisor to add the message to
+ message queue 1's queue. The hypervisor copies memory into the internal
+ message queue representation; the memory doesn't need to be shared between
+ VM_A and VM_B.
+
+2. Gunyah raises the corresponding interrupt for VM_B (Rx vIRQ) when any of
+ these happens:
+
+ a. gh_msgq_send() has PUSH flag. Queue is immediately flushed. This is the typical case.
+ b. Explicility with gh_msgq_push command from VM_A.
+ c. Message queue has reached a threshold depth.
+
+3. VM_B calls gh_msgq_recv() and Gunyah copies message to requested buffer.
+
+4. Gunyah buffers messages in the queue. If the queue became full when VM_A added a message,
+ the return values for gh_msgq_send() include a flag that indicates the queue is full.
+ Once VM_B receives the message and, thus, there is space in the queue, Gunyah
+ will raise the Tx vIRQ on VM_A to indicate it can continue sending messages.
+
+For VM_B to send a message to VM_A, the process is identical, except that hypercalls
+reference message queue 2's capability ID. Each message queue has its own independent
+vIRQ: two TX message queues will have two vIRQs (and two capability IDs).
+
+::
+
+ +---------------+ +-----------------+ +---------------+
+ | VM_A | |Gunyah hypervisor| | VM_B |
+ | | | | | |
+ | | | | | |
+ | | Tx | | | |
+ | |-------->| | Rx vIRQ | |
+ |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() |
+ | |<------- | | | |
+ | | | | | |
+ | Message Queue | | | | Message Queue |
+ | driver | | | | driver |
+ | | | | | |
+ | | | | | |
+ | | | | Tx | |
+ | | Rx vIRQ | |<--------| |
+ |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() |
+ | | | |-------->| |
+ | | | | | |
+ | | | | | |
+ +---------------+ +-----------------+ +---------------+
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index 7fb55ae08598..15869ee059b3 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -16,6 +16,7 @@ Virtualization Support
coco/sev-guest
coco/tdx-guest
hyperv/index
+ gunyah/index
.. only:: html and subproject
--
2.39.2
next prev parent reply other threads:[~2023-03-04 1:07 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-04 1:06 [PATCH v11 00/26] Drivers for gunyah hypervisor Elliot Berman
2023-03-04 1:06 ` Elliot Berman [this message]
2023-03-04 1:06 ` [PATCH v11 02/26] dt-bindings: Add binding " Elliot Berman
2023-03-04 1:06 ` [PATCH v11 03/26] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-04-03 19:44 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 04/26] virt: gunyah: Add hypercalls to identify Gunyah Elliot Berman
2023-03-21 14:22 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 05/26] virt: gunyah: Identify hypervisor version Elliot Berman
2023-03-21 15:48 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 06/26] virt: gunyah: msgq: Add hypercalls to send and receive messages Elliot Berman
2023-03-21 15:49 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 07/26] mailbox: Add Gunyah message queue mailbox Elliot Berman
2023-03-21 14:22 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-04-03 20:15 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 08/26] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
2023-03-31 14:25 ` Alex Elder
2023-04-03 20:34 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 09/26] gunyah: rsc_mgr: Add VM lifecycle RPC Elliot Berman
2023-03-31 14:25 ` Alex Elder
2023-04-03 21:09 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 10/26] gunyah: vm_mgr: Introduce basic VM Manager Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-04-11 20:48 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 11/26] gunyah: rsc_mgr: Add RPC for sharing memory Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 12/26] gunyah: vm_mgr: Add/remove user memory regions Elliot Berman
2023-03-24 18:37 ` Will Deacon
2023-04-11 20:34 ` Elliot Berman
2023-04-11 21:19 ` Will Deacon
2023-04-12 20:48 ` Elliot Berman
2023-04-13 9:54 ` Will Deacon
2023-03-31 14:26 ` Alex Elder
2023-04-11 21:04 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 13/26] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Elliot Berman
2023-03-21 14:24 ` Srinivas Kandagatla
2023-04-11 21:07 ` Elliot Berman
2023-04-11 21:09 ` Alex Elder
2023-03-31 14:26 ` Alex Elder
2023-04-11 21:16 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 14/26] samples: Add sample userspace Gunyah VM Manager Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 15/26] gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-22 19:17 ` Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 16/26] firmware: qcom_scm: Register Gunyah platform ops Elliot Berman
2023-03-21 14:24 ` Srinivas Kandagatla
2023-03-21 18:40 ` Elliot Berman
2023-03-21 20:19 ` Srinivas Kandagatla
2023-03-04 1:06 ` [PATCH v11 17/26] docs: gunyah: Document Gunyah VM Manager Elliot Berman
2023-03-04 1:06 ` [PATCH v11 18/26] virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-04-18 0:25 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 19/26] gunyah: vm_mgr: Add framework to add VM Functions Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 20/26] virt: gunyah: Add resource tickets Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-04-17 22:57 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 21/26] virt: gunyah: Add IO handlers Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 22/26] virt: gunyah: Add proxy-scheduled vCPUs Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-04-17 22:41 ` Elliot Berman
2023-04-18 12:46 ` Alex Elder
2023-04-18 17:18 ` Elliot Berman
2023-04-18 17:31 ` Alex Elder
2023-04-18 18:35 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 23/26] virt: gunyah: Add hypercalls for sending doorbell Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 24/26] virt: gunyah: Add irqfd interface Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-04-17 22:55 ` Elliot Berman
2023-04-18 12:55 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 25/26] virt: gunyah: Add ioeventfd Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 26/26] MAINTAINERS: Add Gunyah hypervisor drivers section Elliot Berman
2023-03-31 14:24 ` [PATCH v11 00/26] Drivers for gunyah hypervisor Alex Elder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230304010632.2127470-2-quic_eberman@quicinc.com \
--to=quic_eberman@quicinc.com \
--cc=agross@kernel.org \
--cc=andersson@kernel.org \
--cc=arnd@arndb.de \
--cc=bagasdotme@gmail.com \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=devicetree@vger.kernel.org \
--cc=dmitry.baryshkov@linaro.org \
--cc=elder@linaro.org \
--cc=gregkh@linuxfoundation.org \
--cc=jassisinghbrar@gmail.com \
--cc=konrad.dybcio@linaro.org \
--cc=krzysztof.kozlowski+dt@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=quic_cvanscha@quicinc.com \
--cc=quic_mnalajal@quicinc.com \
--cc=quic_pheragu@quicinc.com \
--cc=quic_svaddagi@quicinc.com \
--cc=quic_tsoni@quicinc.com \
--cc=robh+dt@kernel.org \
--cc=srinivas.kandagatla@linaro.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).