* [PATCH 00/11] Gunyah Hypervisor drivers
@ 2022-02-23 23:37 Elliot Berman
2022-02-23 23:37 ` [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
` (11 more replies)
0 siblings, 12 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah is a Type-1 hypervisor independent of any
high-level OS kernel, and runs in a higher CPU privilege level. It does
not depend on any lower-privileged OS kernel/code for its core
functionality. This increases its security and can support a much smaller
trusted computing base than Type-2 hypervisors. This series adds the initial
support for Gunyah hypercalls, IPC via message queues, communication with the
Gunyah Resource Manager to enable Gunyah's paravirtualized console.
Gunyah is an open source hypervisor. The source repo is available at
https://github.com/quic/gunyah-hypervisor.
Elliot Berman (11):
docs: gunyah: Introduce Gunyah Hypervisor
dt-bindings: Add binding for gunyah hypervisor
arm64: gunyah: Add Gunyah hypercalls ABI
gunyah: Common types and error codes for Gunyah hypercalls
virt: gunyah: Add sysfs nodes
virt: gunyah: Add capabilities bus and devices
gunyah: msgq: Add Gunyah message queues
gunyah: rsc_mgr: Add resource manager RPC core
gunyah: rsc_mgr: Add auxiliary devices for console
gunyah: rsc_mgr: Add RPC for console services
gunyah: Add tty console driver for RM Console Serivces
.../ABI/testing/sysfs-hypervisor-gunyah | 37 +
.../bindings/gunyah/message-queue.yml | 100 +++
.../bindings/gunyah/qcom,hypervisor.yml | 122 ++++
Documentation/virt/gunyah/index.rst | 99 +++
Documentation/virt/gunyah/message-queue.rst | 52 ++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 12 +
arch/arm64/include/asm/gunyah/hypercall.h | 199 ++++++
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/gunyah/Kconfig | 27 +
drivers/virt/gunyah/Makefile | 8 +
drivers/virt/gunyah/device.c | 108 +++
drivers/virt/gunyah/gunyah_private.h | 18 +
drivers/virt/gunyah/msgq.c | 295 ++++++++
drivers/virt/gunyah/rsc_mgr.c | 632 ++++++++++++++++++
drivers/virt/gunyah/rsc_mgr.h | 53 ++
drivers/virt/gunyah/rsc_mgr_console.c | 410 ++++++++++++
drivers/virt/gunyah/rsc_mgr_rpc.c | 129 ++++
drivers/virt/gunyah/sysfs.c | 152 +++++
include/linux/gunyah.h | 138 ++++
include/linux/gunyah_rsc_mgr.h | 44 ++
22 files changed, 2639 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah
create mode 100644 Documentation/devicetree/bindings/gunyah/message-queue.yml
create mode 100644 Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
create mode 100644 Documentation/virt/gunyah/index.rst
create mode 100644 Documentation/virt/gunyah/message-queue.rst
create mode 100644 arch/arm64/include/asm/gunyah/hypercall.h
create mode 100644 drivers/virt/gunyah/Kconfig
create mode 100644 drivers/virt/gunyah/Makefile
create mode 100644 drivers/virt/gunyah/device.c
create mode 100644 drivers/virt/gunyah/gunyah_private.h
create mode 100644 drivers/virt/gunyah/msgq.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.h
create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c
create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c
create mode 100644 drivers/virt/gunyah/sysfs.c
create mode 100644 include/linux/gunyah.h
create mode 100644 include/linux/gunyah_rsc_mgr.h
--
2.25.1
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
` (10 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel, linux-doc, Jonathan Corbet
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It
does not depend on any lower-privileged OS/kernel code for its core
functionality. This increases its security and can support a smaller
trusted computing based when compared to Type-2 hypervisors.
Add documentation describing the Gunyah hypervisor and the main
components of the Gunyah hypervisor which are of interest to Linux
virtualization development.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
Documentation/virt/gunyah/index.rst | 92 +++++++++++++++++++++
Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 7 ++
4 files changed, 152 insertions(+)
create mode 100644 Documentation/virt/gunyah/index.rst
create mode 100644 Documentation/virt/gunyah/message-queue.rst
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
new file mode 100644
index 000000000000..e7bb2b14543e
--- /dev/null
+++ b/Documentation/virt/gunyah/index.rst
@@ -0,0 +1,92 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Gunyah Hypervisor
+=================
+
+.. toctree::
+ :maxdepth: 1
+
+ message-queue
+
+Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
+a higher CPU privilege level. It does not depend on any lower-privileged operating system
+for its core functionality. This increases its security and can support a much smaller
+trusted computing base than a Type-2 hypervisor.
+
+Gunyah is an open source hypervisor. The source repo is available at
+https://github.com/quic/gunyah-hypervisor.
+
+Gunyah provides these following features.
+
+- Scheduling:
+ A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing
+ of the CPUs.
+- Memory Management:
+ APIs handling memory, abstracted as objects, limiting direct use of physical
+ addresses. Memory ownership and usage tracking of all memory under its control.
+ Memory partitioning between VMs is a fundamental security feature.
+- Interrupt Virtualization:
+ Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled
+ in the hypervisor and routed to the assigned VM.
+- Inter-VM Communication:
+ There are several different mechanisms provided for communicating between VMs.
+- Virtual platform:
+ Architectural devices such as interrupt controllers and CPU timers are directly provided
+ by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI.
+- Device Virtualization:
+ Para-virtualization of devices is supported using inter-VM communication.
+
+Architectures supported
+=======================
+AArch64 with a GIC
+
+Resources and Capabilities
+==========================
+
+Some services or resources provided by the Gunyah hypervisor are described by capability IDs.
+For instance, inter-VM communication is performed with doorbells and message queues. The specific
+instance of a doorbell is described by a capability ID. These devices are described in Linux as a
+struct gunyah_device.
+
+High level management of these resources is performed by the resource manager VM. RM informs a
+guest VM about resources it can access through either the device tree or via guest-initiated RPC.
+
+Resource Manager
+================
+
+The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor.
+It provides policy enforcement aspects of the virtualization system. The resource manager can
+be treated as an extension of the Hypervisor but is separated to its own partition to ensure
+that the hypervisor layer itself remains small and secure and to maintain a separation of policy
+and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines.
+
+Communication with the resource manager from each guest VM happens with message-queue.rst. Details
+about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
+
+::
+ +-------+ +--------+ +--------+
+ | RM | | VM_A | | VM_B |
+ +-.-.-.-+ +---.----+ +---.----+
+ | | | |
+ +-.-.-----------.------------.----+
+ | | \==========/ | |
+ | \========================/ |
+ | Gunyah |
+ +---------------------------------+
+
+The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager.
+
+The resource manager provides the following features:
+
+- Generate device-tree overlay
+- VM creation and deletion
+- VM device-tree management
+- VM access control policy
+- Interrupt routing configuration
+
+When booting a virtual machine which uses a devicetree, resource manager overlays a
+/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM,
+how to communicate with resource manager, and basic description and capabilities of
+this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description
+of this node.
diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
new file mode 100644
index 000000000000..afd405f3a5e1
--- /dev/null
+++ b/Documentation/virt/gunyah/message-queue.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Message Queues
+==============
+Message queue is a simple low-capacity IPC channel between two VMs. It is
+intended for sending small control and configuration messages. Each message
+queue object is unidirectional, so a full-duplex IPC channel requires a pair of
+objects.
+
+Messages can be up to 240 bytes in length. Longer messages require a further
+protocol on top of the message queue messages themselves. For instance, communication
+with the resource manager adds a header field for sending longer messages via multiple
+message fragments.
+
+The diagram below shows how message queue works. A typical configuration involves
+2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message
+queue 2 allows VM_B to send messages to VM_A.
+
+1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall
+ with the message to inform the hypervisor to add the message to
+ message queue 1's queue.
+2. Gunyah raises the corresponding interrupt for VM_B when any of these happens:
+ a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case.
+ b. Explicility with gh_msgq_push command from VM_A.
+ c. Message queue has reached a threshold depth.
+3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer.
+
+For VM_B to send a message to VM_A, the process is identical, except that hypercalls
+reference message queue 2's capability ID.
+
+::
+
+ +---------------+ +-----------------+ +---------------+
+ | VM_A | |Gunyah hypervisor| | VM_B |
+ | | | | | |
+ | | | | | |
+ | | Tx | | | |
+ | |-------->| | Rx vIRQ | |
+ |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() |
+ | |<------- | | | |
+ | | | | | |
+ | Message Queue | | | | Message Queue |
+ | driver | | | | driver |
+ | | | | | |
+ | | | | | |
+ | | | | Tx | |
+ | | Rx vIRQ | |<--------| |
+ |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() |
+ | | | |-------->| |
+ | | | | | |
+ | | | | | |
+ +---------------+ +-----------------+ +---------------+
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index edea7fea95a8..4080e7f5cad8 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -13,6 +13,7 @@ Linux Virtualization Support
guest-halt-polling
ne_overview
acrn/index
+ gunyah/index
.. only:: html and subproject
diff --git a/MAINTAINERS b/MAINTAINERS
index 777cd6fa2b3d..bed175adc4c3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8395,6 +8395,13 @@ L: linux-efi@vger.kernel.org
S: Maintained
F: block/partitions/efi.*
+GUNYAH HYPERVISOR DRIVER
+M: Elliot Berman <quic_eberman@quicinc.com>
+M: Murali Nalajala <quic_mnalajal@quicinc.com>
+L: linux-arm-msm@vger.kernel.org
+S: Maintained
+F: Documentation/virt/gunyah/
+
H8/300 ARCHITECTURE
M: Yoshinori Sato <ysato@users.sourceforge.jp>
L: uclinux-h8-devel@lists.sourceforge.jp (moderated for non-subscribers)
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-02-23 23:37 ` [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-25 20:01 ` Rob Herring
2022-02-23 23:37 ` [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
` (9 subsequent siblings)
11 siblings, 1 reply; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel, Rob Herring, devicetree
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
When Linux is booted as a guest under the Gunyah hypervisor, Gunyah
applies a devicetree overlay describing the virtual platform
configuration of the guest VM, such as the message queue capability IDs
for communicating with the Resource Manager. Add the DT bindings that
Gunyah adheres for the hypervisor node and message queues.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
.../bindings/gunyah/message-queue.yml | 100 ++++++++++++++
.../bindings/gunyah/qcom,hypervisor.yml | 122 ++++++++++++++++++
MAINTAINERS | 1 +
3 files changed, 223 insertions(+)
create mode 100644 Documentation/devicetree/bindings/gunyah/message-queue.yml
create mode 100644 Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
diff --git a/Documentation/devicetree/bindings/gunyah/message-queue.yml b/Documentation/devicetree/bindings/gunyah/message-queue.yml
new file mode 100644
index 000000000000..1a96d3de2a19
--- /dev/null
+++ b/Documentation/devicetree/bindings/gunyah/message-queue.yml
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Gunyah message queue
+
+maintainers:
+ - Murali Nalajala <quic_mnalajal@quicinc.com>
+ - Elliot Berman <quic_eberman@quicinc.com>
+
+properties:
+ compatible:
+ items:
+ - const: qcom,gunyah-message-queue
+ - const: qcom,gunyah-capability
+ peer:
+ description: VMID of the VM on the other end of message queue
+ $ref: /schemas/types.yaml#/definitions/uint32
+ allOf:
+ - if:
+ anyOf:
+ - properties:
+ qcom,is-sender: true
+ - properties:
+ qcom,is-full-duplex: true
+ then:
+ properties:
+ qcom,tx-message-size:
+ description: Maximum size in bytes of a message which can be sent by this queue
+ $ref: /schemas/types.yaml#/definitions/int32
+ qcom,tx-queue-depth:
+ description: Depth of transmit queue for messages sent by this queue
+ $ref: /schemas/types.yaml#/definitions/int32
+ - if:
+ anyOf:
+ - properties:
+ qcom,is-receiver: true
+ - properties:
+ qcom,is-full-duplex: true
+ then:
+ properties:
+ qcom,rx-message-size:
+ description: Maximum size in bytes of a message which can be received by this queue
+ $ref: /schemas/types.yaml#/definitions/int32
+ qcom,rx-queue-depth:
+ description: Depth of transmit queue for messages received by this queue
+ $ref: /schemas/types.yaml#/definitions/int32
+ - if:
+ anyOf:
+ - properties:
+ qcom,is-receiver: true
+ - properties:
+ qcom,is-sender: true
+ then:
+ properties:
+ reg:
+ description: Hypervisor capability ID of the message queue
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minItems: 1
+ maxItems: 1
+ interrupts:
+ minItems: 1
+ maxItems: 1
+ - if:
+ properties:
+ qcom,is-full-duplex: true
+ then:
+ properties:
+ reg:
+ description:
+ Hypervisor capability IDs of the message queue
+ The first is tx side, the second is rx side
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minItems: 2
+ maxItems: 2
+ interrupts:
+ description: The first is tx interrupt, second is rx interrupt
+ minItems: 2
+ maxItems: 2
+ required:
+ - compatible
+ - reg
+ - interrupts
+
+
+examples:
+ - |
+ display-msgq-pair@abbf0da3c3c965cc {
+ compatible = "qcom,gunyah-message-queue", "qcom,gunyah-capability";
+ interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
+ <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
+ reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; /* TX, RX cap ids */
+ qcom,is-full-duplex;
+ qcom,tx-queue-depth = <8>;
+ qcom,tx-message-size = <0xf0>;
+ qcom,rx-queue-depth = <8>;
+ qcom,rx-message-size = <0xf0>;
+ };
\ No newline at end of file
diff --git a/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
new file mode 100644
index 000000000000..f637d51c52f0
--- /dev/null
+++ b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
@@ -0,0 +1,122 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Hypervisor node to define virtual devices and other services provided by a Gunyah hypervisor
+ to this virtual machine.
+
+maintainers:
+ - Murali Nalajala <quic_mnalajal@quicinc.com>
+ - Elliot Berman <quic_eberman@quicinc.com>
+
+description: |+
+ On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which
+ describes the basic configuration of the hypervisor. Virtual machines use this information for
+ initial discovery that they are running as a Gunyah guest VM.
+ See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: qcom,gunyah-hypervisor-1.0
+ - const: qcom,gunyah-hypervisor
+
+ "#address-cells":
+ description: Number of cells needed to represent 64-bit capability IDs.
+ const: 2
+ "#size-cells":
+ description: must be 0, because capability IDs are not memory address
+ ranges and do not have a size.
+ const: 0
+
+ qcom,gunyah-vm:
+ type: object
+ description:
+ The VM Identification is a virtual node that conveys to the VM information
+ about this virtual machine in the context of the hypervisor-based system
+ properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: qcom,gunyah-vm-id-1.0
+ - const: qcom,gunyah-vm-id
+ qcom,vendor:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: Vendor of the Virtual Machine, e.g. Qualcomm
+ qcom,vmid:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: contains the VMID of this VM as a 32-bit value
+ qcom,owner-vmid:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: Contains the hypervisor VMID of the VM's owner. The owner
+ is the VM that allocated and created the VM. VMs directly
+ managed by the resource manager, such as the primary VM do
+ not have an owner.
+ required:
+ - compatible
+ - qcom,vmid
+ - qcom,owner-vmid
+
+patternProperties:
+ "^qcom,resource-manager-rpc(@.*)?":
+ type: object
+ description:
+ Resource Manager node which is required to communicate to Resource
+ Manager VM using Gunyah Message Queues.
+ allOf: "message-queue.yml#"
+
+ properties:
+ compatible:
+ oneOf:
+ items:
+ - const: qcom,resource-manager-1-0
+ - const: qcom,resource-manager
+ qcom,console-dev:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: if set, the resource-manger will accept console logs from the VM
+ qcom,free-irq-start:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: Set on ARM systems which use a GIC. First VIRQ number which is free
+ for virtual interrupt use.
+ required:
+ - qcom,is-full-duplex
+
+
+required:
+- compatible
+- "#address-cells"
+- "#size-cells"
+
+examples:
+ - |
+ hypervisor {
+ #address-cells = <2>;
+ #size-cells = <0>;
+ compatible = "qcom,gunyah-hypervisor-1.0", "qcom,gunyah-hypervisor", "simple-bus";
+ name = "hypervisor";
+
+ qcom,gunyah-vm {
+ compatible = "qcom,gunyah-vm-id-1.0", "qcom,gunyah-vm-id";
+ qcom,vendor = "Qualcomm Technologies, Inc.";
+ qcom,vmid = <45>;
+ qcom,owner-vmid = <3>;
+ };
+
+ qcom,resource-manager-rpc@0000000000000001 {
+ compatible = "qcom,resource-manager-1-0", "qcom,resource-manager",
+ "qcom,gunyah-message-queue", "qcom,gunyah-capability";
+ interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
+ <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
+ reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>;
+ /* TX, RX cap ids */
+ qcom,is-full-duplex;
+ qcom,free-irq-start = <0>;
+ qcom,tx-queue-depth = <8>;
+ qcom,tx-message-size = <0xf0>;
+ qcom,rx-queue-depth = <8>;
+ qcom,rx-message-size = <0xf0>;
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index bed175adc4c3..6a918f653eac 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8400,6 +8400,7 @@ M: Elliot Berman <quic_eberman@quicinc.com>
M: Murali Nalajala <quic_mnalajal@quicinc.com>
L: linux-arm-msm@vger.kernel.org
S: Maintained
+F: Documentation/devicetree/bindings/gunyah/
F: Documentation/virt/gunyah/
H8/300 ARCHITECTURE
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-02-23 23:37 ` [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
2022-02-23 23:37 ` [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-24 10:10 ` Mark Rutland
2022-02-23 23:37 ` [PATCH 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
` (8 subsequent siblings)
11 siblings, 1 reply; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm, linux-arm-kernel
Add initial support to perform Gunyah hypercalls. The arm64 ABI for
Gunyah hypercalls generally follows the AAPCS64, and can be summarized:
- Function identifier is passed through the imm operand
- [r0,r7] are parameter and result registers
- [r8-r18] are temporary and saved by the caller (VM)
- [r19-r31] are preserved and saved by the hypervisor
The preprocessor macors for creating the necessary HVC instruction
roughly follows the SMCCC 1.1 implementation in
include/linux/arm-smccc.h.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 1 +
arch/arm64/include/asm/gunyah/hypercall.h | 193 ++++++++++++++++++++++
2 files changed, 194 insertions(+)
create mode 100644 arch/arm64/include/asm/gunyah/hypercall.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 6a918f653eac..7e6a8488fa3e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8402,6 +8402,7 @@ L: linux-arm-msm@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/gunyah/
F: Documentation/virt/gunyah/
+F: arch/arm64/include/asm/gunyah/
H8/300 ARCHITECTURE
M: Yoshinori Sato <ysato@users.sourceforge.jp>
diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h
new file mode 100644
index 000000000000..626163500e32
--- /dev/null
+++ b/arch/arm64/include/asm/gunyah/hypercall.h
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef __ASM_GH_HYPERCALL_H
+#define __ASM_GH_HYPERCALL_H
+
+#include <linux/types.h>
+
+#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
+
+#define __gh_count_args(...) \
+ ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0)
+
+#define __gh_skip_0(...) __VA_ARGS__
+#define __gh_skip_1(a, ...) __VA_ARGS__
+#define __gh_skip_2(a, b, ...) __VA_ARGS__
+#define __gh_skip_3(a, b, c, ...) __VA_ARGS__
+#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__
+#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__
+#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__
+#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__
+#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__
+
+#define __gh_declare_arg_0(...)
+
+#define __gh_declare_arg_1(a1, ...) \
+ typeof(a1) __gh_a1 = (a1); \
+ register uintptr_t arg1 asm("r0") = __gh_a1
+
+#define __gh_declare_arg_2(a1, a2, ...) \
+ __gh_declare_arg_1(a1); \
+ typeof(a2) __gh_a2 = (a2); \
+ register uintptr_t arg2 asm("r1") = __gh_a2
+
+#define __gh_declare_arg_3(a1, a2, a3, ...) \
+ __gh_declare_arg_2(a1, a2); \
+ typeof(a3) __gh_a3 = (a3); \
+ register uintptr_t arg3 asm("r2") = __gh_a3
+
+#define __gh_declare_arg_4(a1, a2, a3, a4, ...) \
+ __gh_declare_arg_3(a1, a2, a3); \
+ typeof(a4) __gh_a4 = (a4); \
+ register uintptr_t arg4 asm("r3") = __gh_a4
+
+#define __gh_declare_arg_5(a1, a2, a3, a4, a5, ...) \
+ __gh_declare_arg_4(a1, a2, a3, a4); \
+ typeof(a5) __gh_a5 = (a5); \
+ register uintptr_t arg5 asm("r4") = __gh_a5
+
+#define __gh_declare_arg_6(a1, a2, a3, a4, a5, a6, ...) \
+ __gh_declare_arg_5(a1, a2, a3, a4, a5); \
+ typeof(a6) __gh_a6 = (a6); \
+ register uintptr_t arg6 asm("r5") = __gh_a6
+
+#define __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7, ...) \
+ __gh_declare_arg_6(a1, a2, a3, a4, a5, a6); \
+ typeof(a7) __gh_a7 = (a7); \
+ register uintptr_t arg7 asm("r6") = __gh_a7
+
+#define __gh_declare_arg_8(a1, a2, a3, a4, a5, a6, a7, a8, ...) \
+ __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7); \
+ typeof(a8) __gh_a8 = (a8); \
+ register uintptr_t arg8 asm("r7") = __gh_a8
+
+#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs
+#define __gh_declare_args(nargs) ___gh_declare_args(nargs)
+#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__)
+
+#define __gh_constraint_arg_0
+#define __gh_constraint_arg_1 "r" (arg1),
+#define __gh_constraint_arg_2 __gh_constraint_arg_1 "r" (arg2),
+#define __gh_constraint_arg_3 __gh_constraint_arg_2 "r" (arg3),
+#define __gh_constraint_arg_4 __gh_constraint_arg_3 "r" (arg4),
+#define __gh_constraint_arg_5 __gh_constraint_arg_4 "r" (arg5),
+#define __gh_constraint_arg_6 __gh_constraint_arg_5 "r" (arg6),
+#define __gh_constraint_arg_7 __gh_constraint_arg_6 "r" (arg7),
+#define __gh_constraint_arg_8 __gh_constraint_arg_7 "r" (arg8),
+
+#define _gh_constraint_args(nargs) __gh_constraint_arg_ ## nargs
+
+#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__)
+
+#define __gh_declare_res_0
+
+#define __gh_declare_res_1 \
+ register uintptr_t res1 asm("r0")
+
+#define __gh_declare_res_2 \
+ __gh_declare_res_1; \
+ register uintptr_t res2 asm("r1")
+
+#define __gh_declare_res_3 \
+ __gh_declare_res_2; \
+ register uintptr_t res3 asm("r2")
+
+#define __gh_declare_res_4 \
+ __gh_declare_res_3; \
+ register uintptr_t res4 asm("r3")
+
+#define __gh_declare_res_5 \
+ __gh_declare_res_4; \
+ register uintptr_t res5 asm("r4")
+
+#define __gh_declare_res_6 \
+ __gh_declare_res_5; \
+ register uintptr_t res6 asm("r5")
+
+#define __gh_declare_res_7 \
+ __gh_declare_res_6; \
+ register uintptr_t res7 asm("r6")
+
+#define __gh_declare_res_8 \
+ __gh_declare_res_7; \
+ register uintptr_t res8 asm("r7")
+
+#define ___gh_declare_res(nargs) __gh_declare_res_ ## nargs
+#define __gh_declare_res(nargs) ___gh_declare_res(nargs)
+#define _gh_declare_res(...) __gh_declare_res(__gh_count_args(__VA_ARGS__))
+
+#define __gh_constraint_res_0
+#define __gh_constraint_res_1 "=r" (res1)
+#define __gh_constraint_res_2 __gh_constraint_res_1, "=r" (res2)
+#define __gh_constraint_res_3 __gh_constraint_res_2, "=r" (res3)
+#define __gh_constraint_res_4 __gh_constraint_res_3, "=r" (res4)
+#define __gh_constraint_res_5 __gh_constraint_res_4, "=r" (res5)
+#define __gh_constraint_res_6 __gh_constraint_res_5, "=r" (res6)
+#define __gh_constraint_res_7 __gh_constraint_res_6, "=r" (res7)
+#define __gh_constraint_res_8 __gh_constraint_res_7, "=r" (res8)
+
+#define ___gh_constraint_res(nargs) __gh_constraint_res_ ## nargs
+#define __gh_constraint_res(nargs) ___gh_constraint_res(nargs)
+#define _gh_constraint_res(...) \
+ __gh_constraint_res(__gh_count_args(__VA_ARGS__))
+
+#define __gh_assign_res_0(...)
+
+#define __gh_assign_res_1(r1) \
+ r1 = res1;
+
+#define __gh_assign_res_2(r1, r2) \
+ __gh_assign_res_1(r1); \
+ r2 = res2
+
+#define __gh_assign_res_3(r1, r2, r3) \
+ __gh_assign_res_2(r1, r2); \
+ r3 = res3
+
+#define __gh_assign_res_4(r1, r2, r3, r4) \
+ __gh_assign_res_3(r1, r2, r3); \
+ r4 = res4
+
+#define __gh_assign_res_5(r1, r2, r3, r4, r5) \
+ __gh_assign_res_4(r1, r2, r3, r4); \
+ r5 = res5
+
+#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \
+ __gh_assign_res_5(r1, r2, r3, r4, r5); \
+ r6 = res6
+
+#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \
+ __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \
+ r7 = res7
+
+#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \
+ __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \
+ r8 = res8
+
+#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs
+#define __gh_assign_res(nargs) ___gh_assign_res(nargs)
+#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__)
+
+/**
+ * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI
+ * @hcall_num: Hypercall function ID to invoke
+ * @nargs: Number of input arguments
+ * @...: First nargs are the input arguments. Remaining arguments are output variables.
+ */
+#define arch_gh_hypercall(hcall_num, nargs, ...) \
+ do { \
+ _gh_declare_res(__gh_to_res(nargs, __VA_ARGS__)); \
+ _gh_declare_args(nargs, __VA_ARGS__); \
+ asm volatile( \
+ "hvc %[num]\n" \
+ : _gh_constraint_res(__gh_to_res(nargs, __VA_ARGS__)) \
+ : _gh_constraint_args(nargs) \
+ [num] "i" (hcall_num) \
+ : "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", \
+ "memory"); \
+ _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \
+ } while (0)
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 04/11] gunyah: Common types and error codes for Gunyah hypercalls
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (2 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
` (7 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Add architecture-independent standard error codes, types, and macros for
Gunyah hypercalls.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 1 +
include/linux/gunyah.h | 74 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 75 insertions(+)
create mode 100644 include/linux/gunyah.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 7e6a8488fa3e..59e7070f726a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8403,6 +8403,7 @@ S: Maintained
F: Documentation/devicetree/bindings/gunyah/
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah/
+F: include/linux/gunyah.h
H8/300 ARCHITECTURE
M: Yoshinori Sato <ysato@users.sourceforge.jp>
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
new file mode 100644
index 000000000000..8743bf4978e2
--- /dev/null
+++ b/include/linux/gunyah.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_H
+#define _GUNYAH_H
+
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <asm/gunyah/hypercall.h>
+
+typedef u64 gh_capid_t;
+
+/* Common Gunyah macros */
+#define GH_CAPID_INVAL U64_MAX
+
+#define GH_ERROR_OK 0
+#define GH_ERROR_UNIMPLEMENTED -1
+
+#define GH_ERROR_ARG_INVAL 1
+#define GH_ERROR_ARG_SIZE 2
+#define GH_ERROR_ARG_ALIGN 3
+
+#define GH_ERROR_NOMEM 10
+
+#define GH_ERROR_ADDR_OVFL 20
+#define GH_ERROR_ADDR_UNFL 21
+#define GH_ERROR_ADDR_INVAL 22
+
+#define GH_ERROR_DENIED 30
+#define GH_ERROR_BUSY 31
+#define GH_ERROR_IDLE 32
+
+#define GH_ERROR_IRQ_BOUND 40
+#define GH_ERROR_IRQ_UNBOUND 41
+
+#define GH_ERROR_CSPACE_CAP_NULL 50
+#define GH_ERROR_CSPACE_CAP_REVOKED 51
+#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52
+#define GH_ERROR_CSPACE_INSUF_RIGHTS 53
+#define GH_ERROR_CSPACE_FULL 54
+
+#define GH_ERROR_MSGQUEUE_EMPTY 60
+#define GH_ERROR_MSGQUEUE_FULL 61
+
+static inline int gh_remap_error(int gh_error)
+{
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ return 0;
+ case GH_ERROR_NOMEM:
+ return -ENOMEM;
+ case GH_ERROR_DENIED:
+ case GH_ERROR_CSPACE_CAP_NULL:
+ case GH_ERROR_CSPACE_CAP_REVOKED:
+ case GH_ERROR_CSPACE_WRONG_OBJ_TYPE:
+ case GH_ERROR_CSPACE_INSUF_RIGHTS:
+ case GH_ERROR_CSPACE_FULL:
+ return -EACCES;
+ case GH_ERROR_BUSY:
+ case GH_ERROR_IDLE:
+ return -EBUSY;
+ case GH_ERROR_IRQ_BOUND:
+ case GH_ERROR_IRQ_UNBOUND:
+ case GH_ERROR_MSGQUEUE_FULL:
+ case GH_ERROR_MSGQUEUE_EMPTY:
+ return -EPERM;
+ default:
+ return -EINVAL;
+ }
+}
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 05/11] virt: gunyah: Add sysfs nodes
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (3 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
` (6 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Add /sys/hypervisor support when detecting that Linux is running in a
Gunyah environment. Export the version of Gunyah which is reported via
the hyp_identify hypercall.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
.../ABI/testing/sysfs-hypervisor-gunyah | 37 ++++++
MAINTAINERS | 2 +
arch/arm64/include/asm/gunyah/hypercall.h | 2 +
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/gunyah/Kconfig | 13 ++
drivers/virt/gunyah/Makefile | 4 +
drivers/virt/gunyah/sysfs.c | 116 ++++++++++++++++++
8 files changed, 177 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah
create mode 100644 drivers/virt/gunyah/Kconfig
create mode 100644 drivers/virt/gunyah/Makefile
create mode 100644 drivers/virt/gunyah/sysfs.c
diff --git a/Documentation/ABI/testing/sysfs-hypervisor-gunyah b/Documentation/ABI/testing/sysfs-hypervisor-gunyah
new file mode 100644
index 000000000000..ebbdd0aead7b
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-hypervisor-gunyah
@@ -0,0 +1,37 @@
+What: /sys/hypervisor/type
+Date: January 2022
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ Type of hypervisor:
+ "gunyah": Gunyah hypervisor
+
+What: /sys/hypervisor/features
+Date: January 2022
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ Space separated list of features supported by Linux and Gunyah:
+ "cspace": Gunyah devices
+ "doorbell": Sending/receiving virtual interrupts via Gunyah doorbells
+ "message-queue": Sending/receiving messages via Gunyah message queues
+ "vic": Interrupt lending
+ "vpm": Virtual platform management
+ "vcpu": Virtual CPU management
+ "memextent": Memory lending/management
+ "trace": Gunyah hypervisor tracing
+
+
+What: /sys/hypervisor/version/api
+Date: April 2020
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ The Gunyah API version.
+
+What: /sys/hypervisor/version/variant
+Date: April 2020
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ The Gunyah variant (build) version.
\ No newline at end of file
diff --git a/MAINTAINERS b/MAINTAINERS
index 59e7070f726a..10c59c8767ff 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8400,9 +8400,11 @@ M: Elliot Berman <quic_eberman@quicinc.com>
M: Murali Nalajala <quic_mnalajal@quicinc.com>
L: linux-arm-msm@vger.kernel.org
S: Maintained
+F: Documentation/ABI/testing/sysfs-hypervisor-gunyah
F: Documentation/devicetree/bindings/gunyah/
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah/
+F: drivers/virt/gunyah/
F: include/linux/gunyah.h
H8/300 ARCHITECTURE
diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h
index 626163500e32..a8e68ece074e 100644
--- a/arch/arm64/include/asm/gunyah/hypercall.h
+++ b/arch/arm64/include/asm/gunyah/hypercall.h
@@ -7,6 +7,8 @@
#include <linux/types.h>
+#define GH_HYPERCALL_HYP_IDENTIFY 0x6000
+
#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
#define __gh_count_args(...) \
diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig
index 8061e8ef449f..823663d67a95 100644
--- a/drivers/virt/Kconfig
+++ b/drivers/virt/Kconfig
@@ -36,4 +36,6 @@ source "drivers/virt/vboxguest/Kconfig"
source "drivers/virt/nitro_enclaves/Kconfig"
source "drivers/virt/acrn/Kconfig"
+
+source "drivers/virt/gunyah/Kconfig"
endif
diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile
index 3e272ea60cd9..ca2141b6837e 100644
--- a/drivers/virt/Makefile
+++ b/drivers/virt/Makefile
@@ -5,6 +5,7 @@
obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o
obj-y += vboxguest/
+obj-$(CONFIG_GUNYAH) += gunyah/
obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/
obj-$(CONFIG_ACRN_HSM) += acrn/
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
new file mode 100644
index 000000000000..e88289963518
--- /dev/null
+++ b/drivers/virt/gunyah/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config GUNYAH
+ tristate "Gunyah Virtualization drivers"
+ depends on ARM64
+ select SYS_HYPERVISOR
+ help
+ The Gunyah drivers are the helper interfaces that runs in a guest VM
+ such as basic inter-VM IPC and signaling mechanism,s and higher level
+ services such as memory/device sharing, IRQ sharing, and so on.
+
+ Say Y here to enable the drivers needed to interact in a Gunyah
+ virtual environment.
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
new file mode 100644
index 000000000000..0aa086f9149f
--- /dev/null
+++ b/drivers/virt/gunyah/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+gunyah-y += sysfs.o
+obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
new file mode 100644
index 000000000000..3d22f08360db
--- /dev/null
+++ b/drivers/virt/gunyah/sysfs.c
@@ -0,0 +1,116 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gunyah: " fmt
+
+#include <linux/kobject.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+#include <linux/of.h>
+
+#include "gunyah_private.h"
+
+#define GH_API_INFO_API_VERSION(x) (((x) >> 0) & 0x3fff)
+#define GH_API_INFO_BIG_ENDIAN(x) (((x) >> 14) & 1)
+#define GH_API_INFO_IS_64BIT(x) (((x) >> 15) & 1)
+#define GH_API_INFO_VARIANT(x) (((x) >> 56) & 0xff)
+
+#define GH_IDENTIFY_PARTITION_CSPACE(flags) (((flags)[0] >> 0) & 1)
+#define GH_IDENTIFY_DOORBELL(flags) (((flags)[0] >> 1) & 1)
+#define GH_IDENTIFY_MSGQUEUE(flags) (((flags)[0] >> 2) & 1)
+#define GH_IDENTIFY_VIC(flags) (((flags)[0] >> 3) & 1)
+#define GH_IDENTIFY_VPM(flags) (((flags)[0] >> 4) & 1)
+#define GH_IDENTIFY_VCPU(flags) (((flags)[0] >> 5) & 1)
+#define GH_IDENTIFY_MEMEXTENT(flags) (((flags)[0] >> 6) & 1)
+#define GH_IDENTIFY_TRACE_CTRL(flags) (((flags)[0] >> 7) & 1)
+
+struct gh_hypercall_hyp_identify_resp {
+ u64 api_info;
+ u64 flags[3];
+};
+
+static struct gh_hypercall_hyp_identify_resp gunyah_api;
+
+static ssize_t type_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "gunyah\n");
+}
+static struct kobj_attribute type_attr = __ATTR_RO(type);
+
+static ssize_t api_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_API_VERSION(gunyah_api.api_info));
+}
+static struct kobj_attribute api_attr = __ATTR_RO(api);
+
+static ssize_t variant_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_VARIANT(gunyah_api.api_info));
+}
+static struct kobj_attribute variant_attr = __ATTR_RO(variant);
+
+static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "\n");
+}
+static struct kobj_attribute features_attr = __ATTR_RO(features);
+
+static struct attribute *version_attrs[] = {
+ &api_attr.attr,
+ &variant_attr.attr,
+ NULL
+};
+
+static const struct attribute_group version_group = {
+ .name = "version",
+ .attrs = version_attrs,
+};
+
+static int __init gh_sysfs_register(void)
+{
+ int ret;
+
+ ret = sysfs_create_file(hypervisor_kobj, &type_attr.attr);
+ if (ret)
+ return ret;
+
+ ret = sysfs_create_group(hypervisor_kobj, &version_group);
+ if (ret)
+ return ret;
+
+ return sysfs_create_file(hypervisor_kobj, &features_attr.attr);
+}
+
+static void gh_sysfs_unregister(void)
+{
+ sysfs_remove_file(hypervisor_kobj, &type_attr.attr);
+ sysfs_remove_group(hypervisor_kobj, &version_group);
+}
+
+static int __init gunyah_init(void)
+{
+ arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info,
+ gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]);
+
+ if (GH_API_INFO_API_VERSION(gunyah_api.api_info) != 1) {
+ pr_warn("Unrecognized gunyah version: %llu. Currently supported: 1\n",
+ GH_API_INFO_API_VERSION(gunyah_api.api_info));
+ return 0;
+ }
+
+ return gh_sysfs_register();
+}
+module_init(gunyah_init);
+
+static void __exit gunyah_exit(void)
+{
+ gh_sysfs_unregister();
+}
+module_exit(gunyah_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Gunyah Hypervisor Driver");
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 06/11] virt: gunyah: Add capabilities bus and devices
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (4 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
` (5 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Some resources provided by the Gunyah hypervisor are described as
objects. The objects are identified with a capability ID. For instance,
Inter-VM communication is performed with doorbells and message queues.
Each doorbell and message queue endpoint can be described consisely as a
Linux device.
These resources are discovered either on the devicetree or reported by
the Resource Manager. Devices on the Gunyah bus are matched with drivers
according to the type ID reported by resource manager. Most resources
will be discovered directly from the resource manager, so matching
directly on type ID seems like sensible design.
Each resource may also optionally have an interrupt associated with it
and a known partner VM (e.g. which VM is the receiver of a message
queue).
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/Makefile | 2 +-
drivers/virt/gunyah/device.c | 108 +++++++++++++++++++++++++++
drivers/virt/gunyah/gunyah_private.h | 12 +++
drivers/virt/gunyah/sysfs.c | 24 +++++-
include/linux/gunyah.h | 45 +++++++++++
5 files changed, 188 insertions(+), 3 deletions(-)
create mode 100644 drivers/virt/gunyah/device.c
create mode 100644 drivers/virt/gunyah/gunyah_private.h
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 0aa086f9149f..3869fb7371df 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-gunyah-y += sysfs.o
+gunyah-y += sysfs.o device.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/device.c b/drivers/virt/gunyah/device.c
new file mode 100644
index 000000000000..93595f9a65b9
--- /dev/null
+++ b/drivers/virt/gunyah/device.c
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "ghdev: " fmt
+
+#include <linux/interrupt.h>
+#include <linux/gunyah.h>
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+
+#include "gunyah_private.h"
+
+static int gunyah_match(struct device *dev, struct device_driver *drv)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(drv);
+
+ return ghdev->type == ghdrv->type;
+}
+
+static int gunyah_probe(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver);
+
+ return ghdrv->probe ? ghdrv->probe(ghdev) : 0;
+}
+
+static void gunyah_remove(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver);
+
+ if (ghdrv->remove)
+ ghdrv->remove(ghdev);
+}
+
+static struct bus_type gunyah_bus = {
+ .name = "gunyah",
+ .match = gunyah_match,
+ .probe = gunyah_probe,
+ .remove = gunyah_remove,
+};
+
+int gunyah_register_driver(struct gunyah_driver *ghdrv)
+{
+ ghdrv->driver.bus = &gunyah_bus;
+ return driver_register(&ghdrv->driver);
+}
+
+void gunyah_unregister_driver(struct gunyah_driver *ghdrv)
+{
+ driver_unregister(&ghdrv->driver);
+}
+
+static void gunyah_device_release(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+
+ kfree(ghdev);
+}
+
+struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type)
+{
+ struct gunyah_device *ghdev;
+
+ ghdev = kzalloc(sizeof(*ghdev), GFP_KERNEL);
+ if (!ghdev)
+ return NULL;
+
+ ghdev->capid = capid;
+ ghdev->type = type;
+ ghdev->irq = IRQ_NOTCONNECTED;
+ ghdev->dev.parent = parent;
+ ghdev->dev.release = gunyah_device_release;
+ ghdev->dev.bus = &gunyah_bus;
+ device_initialize(&ghdev->dev);
+ return ghdev;
+}
+
+int gunyah_device_add(struct gunyah_device *ghdev)
+{
+ int ret;
+
+ ret = dev_set_name(&ghdev->dev, "%u.%08llx", ghdev->type, ghdev->capid);
+ if (ret)
+ return ret;
+
+ return device_add(&ghdev->dev);
+}
+
+void gunyah_device_remove(struct gunyah_device *ghdev)
+{
+ device_unregister(&ghdev->dev);
+}
+
+int __init gunyah_bus_init(void)
+{
+ return bus_register(&gunyah_bus);
+}
+
+void gunyah_bus_exit(void)
+{
+ bus_unregister(&gunyah_bus);
+}
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
new file mode 100644
index 000000000000..5f3832608020
--- /dev/null
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_PRIVATE_H
+#define _GUNYAH_PRIVATE_H
+
+int __init gunyah_bus_init(void);
+void gunyah_bus_exit(void);
+
+#endif
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index 3d22f08360db..d66e6275fe32 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -55,7 +55,13 @@ static struct kobj_attribute variant_attr = __ATTR_RO(variant);
static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
{
- return sysfs_emit(buffer, "\n");
+ int len = 0;
+
+ if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags))
+ len += sysfs_emit_at(buffer, len, "cspace ");
+
+ len += sysfs_emit_at(buffer, len, "\n");
+ return len;
}
static struct kobj_attribute features_attr = __ATTR_RO(features);
@@ -93,6 +99,8 @@ static void gh_sysfs_unregister(void)
static int __init gunyah_init(void)
{
+ int ret;
+
arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info,
gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]);
@@ -102,12 +110,24 @@ static int __init gunyah_init(void)
return 0;
}
- return gh_sysfs_register();
+ ret = gh_sysfs_register();
+ if (ret)
+ return ret;
+
+ ret = gunyah_bus_init();
+ if (ret)
+ goto err_sysfs;
+
+ return ret;
+err_sysfs:
+ gh_sysfs_unregister();
+ return ret;
}
module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gunyah_bus_exit();
gh_sysfs_unregister();
}
module_exit(gunyah_exit);
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
index 8743bf4978e2..f169c78881cb 100644
--- a/include/linux/gunyah.h
+++ b/include/linux/gunyah.h
@@ -8,6 +8,7 @@
#include <linux/types.h>
#include <linux/errno.h>
+#include <linux/device.h>
#include <asm/gunyah/hypercall.h>
typedef u64 gh_capid_t;
@@ -71,4 +72,48 @@ static inline int gh_remap_error(int gh_error)
}
}
+/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */
+#define GUNYAH_DEVICE_TYPE_BELL_TX 0
+#define GUNYAH_DEVICE_TYPE_BELL_RX 1
+#define GUNYAH_DEVICE_TYPE_MSGQ_TX 2
+#define GUNYAH_DEVICE_TYPE_MSGQ_RX 3
+#define GUNYAH_DEVICE_TYPE_VCPU 4
+
+struct gunyah_device {
+ u8 type;
+ gh_capid_t capid;
+ int irq;
+
+ struct device dev;
+};
+
+#define to_gunyah_device(dev) container_of(dev, struct gunyah_device, dev)
+
+static inline void *ghdev_get_drvdata(const struct gunyah_device *ghdev)
+{
+ return dev_get_drvdata(&ghdev->dev);
+}
+
+static inline void ghdev_set_drvdata(struct gunyah_device *ghdev, void *data)
+{
+ dev_set_drvdata(&ghdev->dev, data);
+}
+
+struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type);
+
+int gunyah_device_add(struct gunyah_device *ghdev);
+void gunyah_device_remove(struct gunyah_device *ghdev);
+
+struct gunyah_driver {
+ struct device_driver driver;
+ u8 type;
+ int (*probe)(struct gunyah_device *ghdev);
+ int (*remove)(struct gunyah_device *ghdev);
+};
+
+#define to_gunyah_driver(drv) container_of(drv, struct gunyah_driver, driver)
+
+int gunyah_register_driver(struct gunyah_driver *ghdrv);
+void gunyah_unregister_driver(struct gunyah_driver *ghdrv);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 07/11] gunyah: msgq: Add Gunyah message queues
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (5 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
` (4 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah message queues are unidirectional pipelines to communicate
between 2 virtual machines, but are typically paired to allow
bidirectional communication. The intended use case is for small control
messages between 2 VMs, as they support a maximum of 240 bytes.
Message queues can be discovered either by resource manager or on the
devicetree. To support discovery on the devicetree, client drivers can
use gh_msgq_platform_host_attach to allocate the tx and rx message
queues according to
Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
arch/arm64/include/asm/gunyah/hypercall.h | 4 +
drivers/virt/gunyah/Makefile | 2 +-
drivers/virt/gunyah/gunyah_private.h | 3 +
drivers/virt/gunyah/msgq.c | 295 ++++++++++++++++++++++
drivers/virt/gunyah/sysfs.c | 9 +
include/linux/gunyah.h | 19 ++
6 files changed, 331 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/gunyah/msgq.c
diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h
index a8e68ece074e..7c6eb82ecd88 100644
--- a/arch/arm64/include/asm/gunyah/hypercall.h
+++ b/arch/arm64/include/asm/gunyah/hypercall.h
@@ -8,6 +8,10 @@
#include <linux/types.h>
#define GH_HYPERCALL_HYP_IDENTIFY 0x6000
+#define GH_HYPERCALL_MSGQ_SEND 0x601B
+#define GH_HYPERCALL_MSGQ_RECV 0x601C
+
+#define GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH BIT(0)
#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 3869fb7371df..94dc8e738911 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-gunyah-y += sysfs.o device.o
+gunyah-y += sysfs.o device.o msgq.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
index 5f3832608020..2ade32bd9bdf 100644
--- a/drivers/virt/gunyah/gunyah_private.h
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -9,4 +9,7 @@
int __init gunyah_bus_init(void);
void gunyah_bus_exit(void);
+int __init gh_msgq_init(void);
+void gh_msgq_exit(void);
+
#endif
diff --git a/drivers/virt/gunyah/msgq.c b/drivers/virt/gunyah/msgq.c
new file mode 100644
index 000000000000..1c79b3fff30c
--- /dev/null
+++ b/drivers/virt/gunyah/msgq.c
@@ -0,0 +1,295 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/interrupt.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+#include <linux/of.h>
+
+#include "gunyah_private.h"
+
+struct gh_msgq {
+ bool ready;
+ wait_queue_head_t wq;
+ spinlock_t lock;
+};
+
+static irqreturn_t gh_msgq_irq_handler(int irq, void *dev)
+{
+ struct gh_msgq *msgq = dev;
+
+ spin_lock(&msgq->lock);
+ msgq->ready = true;
+ spin_unlock(&msgq->lock);
+ wake_up_interruptible(&msgq->wq);
+
+ return IRQ_HANDLED;
+}
+
+static int __gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, u64 tx_flags)
+{
+ unsigned long flags, gh_error;
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+ bool ready;
+
+ spin_lock_irqsave(&msgq->lock, flags);
+ arch_gh_hypercall(GH_HYPERCALL_MSGQ_SEND, 5,
+ ghdev->capid, size, (uintptr_t)buff, tx_flags, 0,
+ gh_error, ready);
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ ret = 0;
+ msgq->ready = ready;
+ break;
+ case GH_ERROR_MSGQUEUE_FULL:
+ ret = -EAGAIN;
+ msgq->ready = false;
+ break;
+ default:
+ ret = gh_remap_error(gh_error);
+ break;
+ }
+
+ spin_unlock_irqrestore(&msgq->lock, flags);
+
+ return ret;
+}
+
+/**
+ * gh_msgq_send() - Send a message to the client running on a different VM
+ * @client: The client descriptor that was obtained via gh_msgq_register()
+ * @buff: Pointer to the buffer where the received data must be placed
+ * @buff_size: The size of the buffer space available
+ * @flags: Optional flags to pass to receive the data. For the list of flags,
+ * see linux/gunyah/gh_msgq.h
+ *
+ * Returns: The number of bytes copied to buff. <0 if there was an error.
+ *
+ * Note: this function may sleep and should not be called from interrupt context
+ */
+int gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags)
+{
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ int ret;
+ u64 tx_flags = 0;
+
+ if (flags & GH_MSGQ_TX_PUSH)
+ tx_flags |= GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH;
+
+ do {
+ ret = __gh_msgq_send(ghdev, buff, size, tx_flags);
+
+ if (ret == -EAGAIN) {
+ if (flags & GH_MSGQ_NONBLOCK)
+ goto out;
+ if (wait_event_interruptible(msgq->wq, msgq->ready))
+ ret = -ERESTARTSYS;
+ }
+ } while (ret == -EAGAIN);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_msgq_send);
+
+static ssize_t __gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size)
+{
+ unsigned long flags, gh_error;
+ size_t recv_size;
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+ bool ready;
+
+ spin_lock_irqsave(&msgq->lock, flags);
+
+ arch_gh_hypercall(GH_HYPERCALL_MSGQ_RECV, 4,
+ ghdev->capid, (uintptr_t)buff, size, 0,
+ gh_error, recv_size, ready);
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ ret = recv_size;
+ msgq->ready = ready;
+ break;
+ case GH_ERROR_MSGQUEUE_EMPTY:
+ ret = -EAGAIN;
+ msgq->ready = false;
+ break;
+ default:
+ ret = gh_remap_error(gh_error);
+ break;
+ }
+
+ spin_unlock_irqrestore(&msgq->lock, flags);
+
+ return ret;
+}
+
+/**
+ * gh_msgq_recv() - Receive a message from the client running on a different VM
+ * @client: The client descriptor that was obtained via gh_msgq_register()
+ * @buff: Pointer to the buffer where the received data must be placed
+ * @buff_size: The size of the buffer space available
+ * @flags: Optional flags to pass to receive the data. For the list of flags,
+ * see linux/gunyah/gh_msgq.h
+ *
+ * Returns: The number of bytes copied to buff. <0 if there was an error.
+ *
+ * Note: this function may sleep and should not be called from interrupt context
+ */
+ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags)
+{
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+
+ do {
+ ret = __gh_msgq_recv(ghdev, buff, size);
+
+ if (ret == -EAGAIN) {
+ if (flags & GH_MSGQ_NONBLOCK)
+ goto out;
+ if (wait_event_interruptible(msgq->wq, msgq->ready))
+ ret = -ERESTARTSYS;
+ }
+ } while (ret == -EAGAIN);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_msgq_recv);
+
+static int gh_msgq_probe(struct gunyah_device *ghdev)
+{
+ struct gh_msgq *msgq;
+
+ msgq = devm_kzalloc(&ghdev->dev, sizeof(*msgq), GFP_KERNEL);
+ ghdev_set_drvdata(ghdev, msgq);
+
+ msgq->ready = true; /* Assume we can use the message queue right away */
+ init_waitqueue_head(&msgq->wq);
+
+ return devm_request_irq(&ghdev->dev, ghdev->irq, gh_msgq_irq_handler, 0,
+ dev_name(&ghdev->dev), msgq);
+}
+
+static struct gunyah_driver gh_msgq_tx_driver = {
+ .driver = {
+ .name = "gh_msgq_tx",
+ .owner = THIS_MODULE,
+ },
+ .type = GUNYAH_DEVICE_TYPE_MSGQ_TX,
+ .probe = gh_msgq_probe,
+};
+
+static struct gunyah_driver gh_msgq_rx_driver = {
+ .driver = {
+ .name = "gh_msgq_rx",
+ .owner = THIS_MODULE,
+ },
+ .type = GUNYAH_DEVICE_TYPE_MSGQ_RX,
+ .probe = gh_msgq_probe,
+};
+
+static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev,
+ u8 gh_type, int idx)
+{
+ int irq, ret;
+ u64 capid;
+ struct device_node *node = pdev->dev.of_node;
+ struct gunyah_device *ghdev;
+
+ irq = platform_get_irq(pdev, idx);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, irq);
+ return ERR_PTR(irq);
+ }
+
+ ret = of_property_read_u64_index(node, "reg", idx, &capid);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret);
+ return ERR_PTR(ret);
+ }
+
+ ghdev = gunyah_device_alloc(&pdev->dev, capid, gh_type);
+ ghdev->irq = irq;
+ ret = gunyah_device_add(ghdev);
+ if (ret) {
+ kfree(ghdev);
+ return ERR_PTR(ret);
+ }
+
+ return ghdev;
+}
+
+int gh_msgq_platform_host_attach(struct platform_device *pdev, struct gh_msgq_platform_host *host)
+{
+ struct gunyah_device *tx_dev = NULL, *rx_dev = NULL;
+ struct device_node *node = pdev->dev.of_node;
+ int idx = 0;
+ bool duplex;
+
+ duplex = of_property_read_bool(node, "qcom,is-full-duplex");
+
+ if (duplex || of_property_read_bool(node, "qcom,is-sender")) {
+ tx_dev = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, idx);
+ if (IS_ERR(tx_dev))
+ return PTR_ERR(tx_dev);
+ idx++;
+ }
+
+ if (duplex || of_property_read_bool(node, "qcom,is-receiver")) {
+ rx_dev = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_RX, idx);
+ if (IS_ERR(rx_dev)) {
+ if (!IS_ERR_OR_NULL(tx_dev))
+ gunyah_device_remove(tx_dev);
+ return PTR_ERR(rx_dev);
+ }
+ }
+
+ host->tx = tx_dev;
+ host->rx = rx_dev;
+ return 0;
+}
+
+void gh_msgq_platform_host_unattach(struct gh_msgq_platform_host *host)
+{
+ if (host->tx) {
+ gunyah_device_remove(host->tx);
+ host->tx = NULL;
+ }
+ if (host->rx) {
+ gunyah_device_remove(host->rx);
+ host->rx = NULL;
+ }
+}
+
+int __init gh_msgq_init(void)
+{
+ int ret;
+
+ ret = gunyah_register_driver(&gh_msgq_tx_driver);
+ if (ret)
+ return ret;
+
+ ret = gunyah_register_driver(&gh_msgq_rx_driver);
+ if (ret)
+ goto err_rx;
+
+ return ret;
+err_rx:
+ gunyah_unregister_driver(&gh_msgq_tx_driver);
+ return ret;
+}
+
+void gh_msgq_exit(void)
+{
+ gunyah_unregister_driver(&gh_msgq_rx_driver);
+ gunyah_unregister_driver(&gh_msgq_tx_driver);
+}
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index d66e6275fe32..7bf39fe1b6e6 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -59,6 +59,8 @@ static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr,
if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags))
len += sysfs_emit_at(buffer, len, "cspace ");
+ if (GH_IDENTIFY_MSGQUEUE(gunyah_api.flags))
+ len += sysfs_emit_at(buffer, len, "message-queue ");
len += sysfs_emit_at(buffer, len, "\n");
return len;
@@ -118,7 +120,13 @@ static int __init gunyah_init(void)
if (ret)
goto err_sysfs;
+ ret = gh_msgq_init();
+ if (ret)
+ goto err_bus;
+
return ret;
+err_bus:
+ gunyah_bus_exit();
err_sysfs:
gh_sysfs_unregister();
return ret;
@@ -127,6 +135,7 @@ module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gh_msgq_exit();
gunyah_bus_exit();
gh_sysfs_unregister();
}
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
index f169c78881cb..66c1dba73cc5 100644
--- a/include/linux/gunyah.h
+++ b/include/linux/gunyah.h
@@ -10,6 +10,7 @@
#include <linux/errno.h>
#include <linux/device.h>
#include <asm/gunyah/hypercall.h>
+#include <linux/platform_device.h>
typedef u64 gh_capid_t;
@@ -116,4 +117,22 @@ struct gunyah_driver {
int gunyah_register_driver(struct gunyah_driver *ghdrv);
void gunyah_unregister_driver(struct gunyah_driver *ghdrv);
+#define GH_MSGQ_MAX_MSG_SIZE 1024
+
+/* Possible flags to pass for Tx or Rx */
+#define GH_MSGQ_TX_PUSH BIT(0)
+#define GH_MSGQ_NONBLOCK BIT(32)
+
+int gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags);
+ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, unsigned long flags);
+
+struct gh_msgq_platform_host {
+ struct gunyah_device *tx;
+ struct gunyah_device *rx;
+};
+
+int gh_msgq_platform_host_attach(struct platform_device *pdev, struct gh_msgq_platform_host *host);
+void gh_msgq_platform_host_unattach(struct gh_msgq_platform_host *host);
+
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 08/11] gunyah: rsc_mgr: Add resource manager RPC core
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (6 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
` (3 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
The resource manager is a special virtual machine which is always
running on a Gunyah system. It provides APIs for creating and destroying
VMs, secure memory management, sharing/lending of memory between VMs,
and setup of inter-VM communication. Calls to the resource manager are
made via message queues.
This patch implements the basic probing and RPC mechanism to make those
API calls. Request/response calls can be made with gh_rm_call.
Drivers can also register to notifications pushed by RM via
gh_rm_register_notifier
Specific API calls that resource manager supports will be implemented in
subsequent patches.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 2 +-
drivers/virt/gunyah/Kconfig | 1 +
drivers/virt/gunyah/Makefile | 1 +
drivers/virt/gunyah/gunyah_private.h | 3 +
drivers/virt/gunyah/rsc_mgr.c | 574 +++++++++++++++++++++++++++
drivers/virt/gunyah/rsc_mgr.h | 34 ++
drivers/virt/gunyah/sysfs.c | 7 +
include/linux/gunyah_rsc_mgr.h | 29 ++
8 files changed, 650 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/gunyah/rsc_mgr.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.h
create mode 100644 include/linux/gunyah_rsc_mgr.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 10c59c8767ff..b05359adc4f7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8405,7 +8405,7 @@ F: Documentation/devicetree/bindings/gunyah/
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah/
F: drivers/virt/gunyah/
-F: include/linux/gunyah.h
+F: include/linux/gunyah*.h
H8/300 ARCHITECTURE
M: Yoshinori Sato <ysato@users.sourceforge.jp>
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
index e88289963518..2ef4887e280d 100644
--- a/drivers/virt/gunyah/Kconfig
+++ b/drivers/virt/gunyah/Kconfig
@@ -4,6 +4,7 @@ config GUNYAH
tristate "Gunyah Virtualization drivers"
depends on ARM64
select SYS_HYPERVISOR
+ select AUXILIARY_BUS
help
The Gunyah drivers are the helper interfaces that runs in a guest VM
such as basic inter-VM IPC and signaling mechanism,s and higher level
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 94dc8e738911..86655bca8944 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
gunyah-y += sysfs.o device.o msgq.o
+gunyah-y += rsc_mgr.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
index 2ade32bd9bdf..6483ffa8c15d 100644
--- a/drivers/virt/gunyah/gunyah_private.h
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -12,4 +12,7 @@ void gunyah_bus_exit(void);
int __init gh_msgq_init(void);
void gh_msgq_exit(void);
+int __init gh_rsc_mgr_init(void);
+void gh_rsc_mgr_exit(void);
+
#endif
diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c
new file mode 100644
index 000000000000..c8c4e1617566
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr.c
@@ -0,0 +1,574 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr: " fmt
+
+#include <linux/of.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/of_irq.h>
+#include <linux/kthread.h>
+#include <linux/notifier.h>
+#include <linux/irqdomain.h>
+#include <linux/workqueue.h>
+#include <linux/completion.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/gunyah_rsc_mgr.h>
+#include <linux/mod_devicetable.h>
+#include <linux/platform_device.h>
+
+#include "gunyah_private.h"
+#include "rsc_mgr.h"
+
+/* Resource Manager Header */
+struct gh_rm_rpc_hdr {
+ u8 version:4,
+ hdr_words:4;
+ u8 type:2,
+ fragments:6;
+ u16 seq;
+ u32 msg_id;
+} __packed;
+
+/* Standard reply header */
+struct gh_rm_rpc_reply_hdr {
+ struct gh_rm_rpc_hdr rpc_hdr;
+ u32 err_code;
+} __packed;
+
+/* RPC Header versions */
+#define GH_RM_RPC_HDR_VERSION_ONE 0x1
+
+/* RPC Header words */
+#define GH_RM_RPC_HDR_WORDS 0x2
+
+/* RPC Message types */
+#define GH_RM_RPC_TYPE_CONT 0x0
+#define GH_RM_RPC_TYPE_REQ 0x1
+#define GH_RM_RPC_TYPE_RPLY 0x2
+#define GH_RM_RPC_TYPE_NOTIF 0x3
+
+#define GH_RM_MAX_NUM_FRAGMENTS 62
+
+#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr))
+
+/**
+ * struct gh_rm_connection - Represents a complete message from resource manager
+ * @buff: Combined payload of all the fragments (i.e. msg headers stripped off).
+ * @size: Size of the payload.
+ * @ret: Linux return code, set in case there was an error processing the connection.
+ * @msg_id: Message ID from the header.
+ * @type: GH_RM_RPC_TYPE_RPLY or GH_RM_RPC_TYPE_NOTIF.
+ * @num_fragments: total number of fragments expected to be received for this connection.
+ * @fragments_recieved: fragments received so far.
+ * @rm_error: For request/reply sequences with standard replies.
+ * @seq: Sequence ID for the main message.
+ */
+struct gh_rm_connection {
+ void *buff;
+ size_t size;
+ int ret;
+ u32 msg_id;
+ u8 type;
+
+ u8 num_fragments;
+ u8 fragments_received;
+
+ /* only for req/reply sequence */
+ u32 rm_error;
+ u16 seq;
+ struct completion seq_done;
+};
+
+struct gh_rm_notif_complete {
+ struct gh_rm_connection *conn;
+ struct work_struct work;
+};
+
+struct gh_rsc_mgr {
+ struct task_struct *recv_task;
+ struct gh_msgq_platform_host msgq;
+
+ struct idr call_idr;
+ struct mutex call_idr_lock;
+
+ struct mutex send_lock;
+};
+
+static struct gh_rsc_mgr *__rsc_mgr;
+SRCU_NOTIFIER_HEAD_STATIC(gh_rm_notifier);
+
+static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type)
+{
+ struct gh_rm_connection *connection;
+
+ connection = kzalloc(sizeof(*connection), GFP_KERNEL);
+ if (!connection)
+ return NULL;
+
+ connection->type = type;
+ connection->msg_id = msg_id;
+
+ return connection;
+}
+
+/**
+ * gh_rm_init_connection_buff() - Fills the first message for a connection.
+ */
+static int gh_rm_init_connection_buff(struct gh_rm_connection *connection, void *msg,
+ size_t hdr_size, size_t payload_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ size_t max_buf_size;
+
+ connection->num_fragments = hdr->fragments;
+ connection->fragments_received = 0;
+ connection->type = hdr->type;
+
+ /* There's not going to be any payload, no need to allocate buffer. */
+ if (!payload_size && !connection->num_fragments)
+ return 0;
+
+ /*
+ * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size
+ * and can received (hdr->fragments + 1) of those
+ */
+ max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (hdr->fragments + 1);
+
+ connection->buff = kzalloc(max_buf_size, GFP_KERNEL);
+ if (!connection->buff)
+ return -ENOMEM;
+
+ memcpy(connection->buff, msg + hdr_size, payload_size);
+ connection->size = payload_size;
+ return 0;
+}
+
+static void gh_rm_notif_work(struct work_struct *work)
+{
+ struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work);
+ struct gh_rm_connection *connection = notif->conn;
+ u32 notif_id = connection->msg_id;
+ struct gh_rm_notification notification = {
+ .buff = connection->buff,
+ .size = connection->size,
+ };
+
+ srcu_notifier_call_chain(&gh_rm_notifier, notif_id, ¬ification);
+
+ kfree(connection->buff);
+ kfree(connection);
+ kfree(notif);
+}
+
+static struct gh_rm_connection *gh_rm_process_notif(struct gh_rsc_mgr *rsc_mgr,
+ void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ struct gh_rm_connection *connection;
+
+ connection = gh_rm_alloc_connection(hdr->msg_id, hdr->type);
+ if (!connection) {
+ pr_err("Failed to alloc connection for notification, dropping.\n");
+ return NULL;
+ }
+
+ if (gh_rm_init_connection_buff(connection, msg, sizeof(*hdr), msg_size - sizeof(*hdr))) {
+ pr_err("Failed to alloc connection buffer for notification, dropping.\n");
+ kfree(connection);
+ return NULL;
+ }
+
+ return connection;
+}
+
+static struct gh_rm_connection *gh_rm_process_rply(struct gh_rsc_mgr *rsc_mgr,
+ void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_reply_hdr *reply_hdr = msg;
+ struct gh_rm_rpc_hdr *hdr = msg;
+ struct gh_rm_connection *connection;
+
+ if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock))
+ return ERR_PTR(-ERESTARTSYS);
+
+ connection = idr_find(&rsc_mgr->call_idr, hdr->seq);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ if (!connection) {
+ pr_err("Failed to find connection for sequence %u\n", hdr->seq);
+ return NULL;
+ }
+ if (connection->msg_id != hdr->msg_id) {
+ pr_err("Reply for sequence %u expected msg_id: %x but got %x\n", hdr->seq,
+ connection->msg_id, hdr->msg_id);
+ /*
+ * Don't complete connection and error the client, maybe resource manager will
+ * send us the expected reply sequence soon.
+ */
+ return NULL;
+ }
+
+ if (gh_rm_init_connection_buff(connection, msg, sizeof(*reply_hdr),
+ msg_size - sizeof(*reply_hdr))) {
+ pr_err("Failed to alloc connection buffer for sequence %d\n", hdr->seq);
+ /* Send connection complete and error the client. */
+ connection->ret = -ENOMEM;
+ complete(&connection->seq_done);
+ return NULL;
+ }
+
+ connection->rm_error = reply_hdr->err_code;
+ return connection;
+}
+
+static void gh_rm_process_cont(struct gh_rm_connection *connection, void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ size_t payload_size = msg_size - sizeof(*hdr);
+
+ /*
+ * hdr->fragments and hdr->msg_id preserves the value from first reply or notif message.
+ * For sake of sanity, check if it's still intact.
+ */
+ if (connection->msg_id != hdr->msg_id)
+ pr_warn("Appending mismatched continuation with id %d to connection with id %d\n",
+ hdr->msg_id, connection->msg_id);
+ if (connection->num_fragments != hdr->fragments)
+ pr_warn("Number of fragments mismatch for seq: %d\n", hdr->seq);
+
+ memcpy(connection->buff + connection->size, msg + sizeof(*hdr), payload_size);
+ connection->size += payload_size;
+ connection->fragments_received++;
+}
+
+static bool gh_rm_complete_connection(struct gh_rm_connection *connection)
+{
+ struct gh_rm_notif_complete *notif_work;
+
+ if (!connection)
+ return false;
+
+ if (connection->fragments_received != connection->num_fragments)
+ return false;
+
+ switch (connection->type) {
+ case GH_RM_RPC_TYPE_RPLY:
+ complete(&connection->seq_done);
+ break;
+ case GH_RM_RPC_TYPE_NOTIF:
+ notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL);
+ if (notif_work == NULL)
+ break;
+
+ notif_work->conn = connection;
+ INIT_WORK(¬if_work->work, gh_rm_notif_work);
+
+ schedule_work(¬if_work->work);
+ break;
+ default:
+ pr_err("Invalid message type (%d) received\n", connection->type);
+ break;
+ }
+
+ return true;
+}
+
+static int gh_rm_recv_task_fn(void *data)
+{
+ struct gh_rsc_mgr *rsc_mgr = data;
+ struct gh_rm_connection *connection = NULL;
+ struct gh_rm_rpc_hdr *hdr = NULL;
+ ssize_t msg_size;
+ void *msg;
+
+ msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
+ while (!kthread_should_stop()) {
+ /* Block until a new message is received */
+ msg_size = gh_msgq_recv(rsc_mgr->msgq.rx, msg, GH_MSGQ_MAX_MSG_SIZE, 0);
+ if (msg_size < 0) {
+ pr_err("Failed to receive the message: %ld\n", msg_size);
+ continue;
+ } else if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) {
+ pr_err("Invalid message size received: %ld is too small\n", msg_size);
+ continue;
+ }
+
+ hdr = msg;
+ switch (hdr->type) {
+ case GH_RM_RPC_TYPE_NOTIF:
+ if (connection) {
+ /* Not possible per protocol. Do something better than BUG_ON */
+ pr_warn("Received start of new notification without finishing existing message series.\n");
+ kfree(connection->buff);
+ kfree(connection);
+ }
+ connection = gh_rm_process_notif(rsc_mgr, msg, msg_size);
+ break;
+ case GH_RM_RPC_TYPE_RPLY:
+ if (connection) {
+ /* Not possible per protocol. Do something better than BUG_ON */
+ pr_warn("Received start of new reply without finishing existing message series.\n");
+ kfree(connection->buff);
+ kfree(connection);
+ }
+ connection = gh_rm_process_rply(rsc_mgr, msg, msg_size);
+ break;
+ case GH_RM_RPC_TYPE_CONT:
+ if (!connection) {
+ pr_warn("Received a continuation message without receiving initial message\n");
+ break;
+ }
+ gh_rm_process_cont(connection, msg, msg_size);
+ break;
+ default:
+ pr_err("Invalid message type (%d) received\n", hdr->type);
+ continue;
+ }
+
+ if (gh_rm_complete_connection(connection))
+ connection = NULL;
+ }
+
+ return 0;
+}
+
+static int gh_rm_send_request(struct gh_rsc_mgr *rsc_mgr, u32 message_id,
+ const void *req_buff, size_t req_buff_size,
+ struct gh_rm_connection *connection)
+{
+ size_t buff_size_remaining = req_buff_size;
+ const void *req_buff_curr = req_buff;
+ struct gh_rm_rpc_hdr *hdr;
+ unsigned long tx_flags;
+ u32 num_fragments = 0;
+ size_t payload_size;
+ void *msg;
+ int i, ret = 0;
+
+ if (req_buff_size > GH_RM_MAX_MSG_SIZE)
+ num_fragments = req_buff_size / GH_RM_MAX_MSG_SIZE;
+
+ if (num_fragments > GH_RM_MAX_NUM_FRAGMENTS) {
+ pr_err("Limit exceeded for the number of fragments: %u\n", num_fragments);
+ return -E2BIG;
+ }
+
+ /*
+ * The above calculation also includes the count for the 'request' packet.
+ * Exclude it as the header needs to fill the num. of fragments to follow.
+ */
+ if (num_fragments)
+ num_fragments--;
+
+ if (mutex_lock_interruptible(&rsc_mgr->send_lock))
+ return -ERESTARTSYS;
+
+ msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!msg) {
+ mutex_unlock(&rsc_mgr->send_lock);
+ return -ENOMEM;
+ }
+
+ /* Consider also the 'request' packet for the loop count */
+ for (i = 0; i <= num_fragments; i++) {
+ if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) {
+ payload_size = GH_RM_MAX_MSG_SIZE;
+ buff_size_remaining -= payload_size;
+ } else {
+ payload_size = buff_size_remaining;
+ }
+
+ memset(msg, 0, GH_MSGQ_MAX_MSG_SIZE);
+
+ /* Fill header */
+ hdr = msg;
+ hdr->version = GH_RM_RPC_HDR_VERSION_ONE;
+ hdr->hdr_words = GH_RM_RPC_HDR_WORDS;
+ hdr->type = i == 0 ? GH_RM_RPC_TYPE_REQ : GH_RM_RPC_TYPE_CONT;
+ hdr->fragments = num_fragments;
+ hdr->seq = connection->seq;
+ hdr->msg_id = message_id;
+
+ /* Copy payload */
+ memcpy(msg + sizeof(*hdr), req_buff_curr, payload_size);
+ req_buff_curr += payload_size;
+
+ /* Force the last fragment to be sent immediately to the receiver */
+ tx_flags = (i == num_fragments) ? GH_MSGQ_TX_PUSH : 0;
+
+ ret = gh_msgq_send(rsc_mgr->msgq.tx, msg, sizeof(*hdr) + payload_size, tx_flags);
+
+ if (ret)
+ break;
+ }
+
+ mutex_unlock(&rsc_mgr->send_lock);
+ return ret;
+}
+
+/**
+ * gh_rm_call: Achieve request-response type communication with RPC
+ * @message_id: The RM RPC message-id
+ * @req_buff: Request buffer that contains the payload
+ * @req_buff_size: Total size of the payload
+ * @resp_buf: Pointer to a response buffer
+ * @resp_buff_size: Size of the response buffer
+ * @reply_err_code: Returns Gunyah standard error code for the response
+ *
+ * Make a request to the RM-VM and wait for reply back. For a successful
+ * response, the function returns the payload. The size of the payload is set in resp_buff_size.
+ * The resp_buf should be freed by the caller.
+ *
+ * Context: Process context. Will sleep waiting for reply.
+ * Return: >0 is standard reply error from RM. <0 on internal error.
+ */
+int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
+ void **resp_buf, size_t *resp_buff_size)
+{
+ struct gh_rm_connection *connection;
+ int ret;
+ struct gh_rsc_mgr *rsc_mgr = __rsc_mgr;
+
+ /* messaged_id 0 is reserved */
+ if (!message_id)
+ return -EINVAL;
+
+ if (!rsc_mgr)
+ return -EPROBE_DEFER;
+
+ connection = gh_rm_alloc_connection(message_id, GH_RM_RPC_TYPE_RPLY);
+ if (!connection)
+ return -ENOMEM;
+
+ init_completion(&connection->seq_done);
+
+ /* Allocate a new seq number for this connection */
+ if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) {
+ kfree(connection);
+ return -ERESTARTSYS;
+ }
+ connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ /* Send the request to the Resource Manager */
+ ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection);
+ if (ret < 0)
+ goto out;
+
+ /* Wait for response */
+ wait_for_completion(&connection->seq_done);
+
+ if (connection->ret) {
+ ret = connection->ret;
+ kfree(connection->buff);
+ goto out;
+ }
+
+ mutex_lock(&rsc_mgr->call_idr_lock);
+ idr_remove(&rsc_mgr->call_idr, connection->seq);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ *resp_buf = connection->buff;
+ *resp_buff_size = connection->size;
+ ret = connection->rm_error;
+
+out:
+ kfree(connection);
+ return ret;
+}
+
+int gh_rm_register_notifier(struct notifier_block *nb)
+{
+ return srcu_notifier_chain_register(&gh_rm_notifier, nb);
+}
+EXPORT_SYMBOL_GPL(gh_rm_register_notifier);
+
+int gh_rm_unregister_notifier(struct notifier_block *nb)
+{
+ return srcu_notifier_chain_unregister(&gh_rm_notifier, nb);
+}
+EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier);
+
+static int gh_rm_drv_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct gh_rsc_mgr *rsc_mgr;
+ struct list_head *l, *n;
+ int ret, i;
+
+ rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL);
+ if (!rsc_mgr)
+ return -ENOMEM;
+ platform_set_drvdata(pdev, rsc_mgr);
+
+ mutex_init(&rsc_mgr->call_idr_lock);
+ idr_init(&rsc_mgr->call_idr);
+ mutex_init(&rsc_mgr->send_lock);
+
+ ret = gh_msgq_platform_host_attach(pdev, &rsc_mgr->msgq);
+ if (ret)
+ return ret;
+ if (!rsc_mgr->msgq.tx || !rsc_mgr->msgq.rx) {
+ dev_warn(dev, "Expected both tx and rx message queues\n");
+ ret = -ENODEV;
+ goto err_msgq;
+ }
+
+ rsc_mgr->recv_task = kthread_run(gh_rm_recv_task_fn, rsc_mgr, "gh_rm_recv_task");
+ if (IS_ERR_OR_NULL(rsc_mgr->recv_task)) {
+ ret = PTR_ERR(rsc_mgr->recv_task);
+ goto err_msgq;
+ }
+
+ __rsc_mgr = rsc_mgr;
+
+ return 0;
+
+err_msgq:
+ gh_msgq_platform_host_unattach(&rsc_mgr->msgq);
+ return ret;
+}
+
+static int gh_rm_drv_remove(struct platform_device *pdev)
+{
+ struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev);
+
+ gh_msgq_platform_host_unattach(&rsc_mgr->msgq);
+
+ return 0;
+}
+
+static const struct of_device_id gh_rm_of_match[] = {
+ { .compatible = "qcom,resource-manager-1-0" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, gh_rm_of_match);
+
+static struct platform_driver gh_rsc_mgr_driver = {
+ .probe = gh_rm_drv_probe,
+ .remove = gh_rm_drv_remove,
+ .driver = {
+ .name = "gh_rsc_mgr",
+ .of_match_table = gh_rm_of_match,
+ },
+};
+
+int __init gh_rsc_mgr_init(void)
+{
+ return platform_driver_register(&gh_rsc_mgr_driver);
+}
+
+void gh_rsc_mgr_exit(void)
+{
+ platform_driver_unregister(&gh_rsc_mgr_driver);
+}
diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h
new file mode 100644
index 000000000000..e4f2499267bf
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef __GH_RSC_MGR_PRIV_H
+#define __GH_RSC_MGR_PRIV_H
+
+#include <linux/gunyah.h>
+
+/* RM Error codes */
+#define GH_RM_ERROR_OK 0x0
+#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF
+#define GH_RM_ERROR_NOMEM 0x1
+#define GH_RM_ERROR_NORESOURCE 0x2
+#define GH_RM_ERROR_DENIED 0x3
+#define GH_RM_ERROR_INVALID 0x4
+#define GH_RM_ERROR_BUSY 0x5
+#define GH_RM_ERROR_ARGUMENT_INVALID 0x6
+#define GH_RM_ERROR_HANDLE_INVALID 0x7
+#define GH_RM_ERROR_VALIDATE_FAILED 0x8
+#define GH_RM_ERROR_MAP_FAILED 0x9
+#define GH_RM_ERROR_MEM_INVALID 0xA
+#define GH_RM_ERROR_MEM_INUSE 0xB
+#define GH_RM_ERROR_MEM_RELEASED 0xC
+#define GH_RM_ERROR_VMID_INVALID 0xD
+#define GH_RM_ERROR_LOOKUP_FAILED 0xE
+#define GH_RM_ERROR_IRQ_INVALID 0xF
+#define GH_RM_ERROR_IRQ_INUSE 0x10
+#define GH_RM_ERROR_IRQ_RELEASED 0x11
+
+int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
+ void **resp_buf, size_t *resp_buff_size);
+
+#endif
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index 7bf39fe1b6e6..389feeb5cc03 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -124,7 +124,13 @@ static int __init gunyah_init(void)
if (ret)
goto err_bus;
+ ret = gh_rsc_mgr_init();
+ if (ret)
+ goto err_msgq;
+
return ret;
+err_msgq:
+ gh_msgq_exit();
err_bus:
gunyah_bus_exit();
err_sysfs:
@@ -135,6 +141,7 @@ module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gh_rsc_mgr_exit();
gh_msgq_exit();
gunyah_bus_exit();
gh_sysfs_unregister();
diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h
new file mode 100644
index 000000000000..015bd851e1a3
--- /dev/null
+++ b/include/linux/gunyah_rsc_mgr.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_RSC_MGR_H
+#define _GUNYAH_RSC_MGR_H
+
+#include <linux/list.h>
+#include <linux/notifier.h>
+#include <linux/gunyah.h>
+
+typedef u16 gh_vmid_t;
+typedef u32 gh_virq_handle_t;
+
+#define GH_VMID_INVAL U16_MAX
+
+/* Gunyah recognizes VMID0 as an alias to the current VM's ID */
+#define GH_VMID_SELF 0
+
+struct gh_rm_notification {
+ const void *buff;
+ const size_t size;
+};
+
+int gh_rm_register_notifier(struct notifier_block *nb);
+int gh_rm_unregister_notifier(struct notifier_block *nb);
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 09/11] gunyah: rsc_mgr: Add auxiliary devices for console
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (7 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
` (2 subsequent siblings)
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah resource manager exposes a concrete functionalities which
complicate a single resource manager driver. Use auxiliary bus
to help split high level functions for the resource manager and keep the
primary resource manager driver focused on the RPC with RM itself.
Delegate Resource Manager's console functionality to the auxiliary bus.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/rsc_mgr.c | 58 +++++++++++++++++++++++++++++++++++
1 file changed, 58 insertions(+)
diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c
index c8c4e1617566..9ca63053dd38 100644
--- a/drivers/virt/gunyah/rsc_mgr.c
+++ b/drivers/virt/gunyah/rsc_mgr.c
@@ -91,6 +91,11 @@ struct gh_rm_notif_complete {
struct work_struct work;
};
+struct gh_rsc_mgr_adev {
+ struct auxiliary_device adev;
+ struct list_head list;
+};
+
struct gh_rsc_mgr {
struct task_struct *recv_task;
struct gh_msgq_platform_host msgq;
@@ -99,6 +104,13 @@ struct gh_rsc_mgr {
struct mutex call_idr_lock;
struct mutex send_lock;
+
+ struct list_head adevs;
+};
+
+/* List of auxiliary devices which resource manager creates */
+static const char *adev_names[] = {
+ "console",
};
static struct gh_rsc_mgr *__rsc_mgr;
@@ -499,10 +511,19 @@ int gh_rm_unregister_notifier(struct notifier_block *nb)
}
EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier);
+static void gh_rm_adev_release(struct device *dev)
+{
+ struct gh_rsc_mgr_adev *rm_adev = container_of(dev, struct gh_rsc_mgr_adev, adev.dev);
+
+ list_del(&rm_adev->list);
+ kfree(rm_adev);
+}
+
static int gh_rm_drv_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct gh_rsc_mgr *rsc_mgr;
+ struct gh_rsc_mgr_adev *rm_adev;
struct list_head *l, *n;
int ret, i;
@@ -514,6 +535,7 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
mutex_init(&rsc_mgr->call_idr_lock);
idr_init(&rsc_mgr->call_idr);
mutex_init(&rsc_mgr->send_lock);
+ INIT_LIST_HEAD(&rsc_mgr->adevs);
ret = gh_msgq_platform_host_attach(pdev, &rsc_mgr->msgq);
if (ret)
@@ -530,10 +552,38 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
goto err_msgq;
}
+ for (i = 0; i < ARRAY_SIZE(adev_names); i++) {
+ rm_adev = kzalloc(sizeof(*rm_adev), GFP_KERNEL);
+
+ rm_adev->adev.dev.parent = dev;
+ rm_adev->adev.dev.release = gh_rm_adev_release;
+ rm_adev->adev.name = adev_names[i];
+ ret = auxiliary_device_init(&rm_adev->adev);
+ if (ret) {
+ kfree(rm_adev);
+ goto err_adevs;
+ }
+
+ list_add(&rm_adev->list, &rsc_mgr->adevs);
+
+ ret = auxiliary_device_add(&rm_adev->adev);
+ if (ret) {
+ auxiliary_device_uninit(&rm_adev->adev);
+ goto err_adevs;
+ }
+ }
+
__rsc_mgr = rsc_mgr;
return 0;
+err_adevs:
+ list_for_each_safe(l, n, &rsc_mgr->adevs) {
+ rm_adev = container_of(l, struct gh_rsc_mgr_adev, list);
+ auxiliary_device_delete(&rm_adev->adev);
+ auxiliary_device_uninit(&rm_adev->adev);
+ }
+
err_msgq:
gh_msgq_platform_host_unattach(&rsc_mgr->msgq);
return ret;
@@ -542,6 +592,14 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
static int gh_rm_drv_remove(struct platform_device *pdev)
{
struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev);
+ struct gh_rsc_mgr_adev *rm_adev;
+ struct list_head *l, *n;
+
+ list_for_each_safe(l, n, &rsc_mgr->adevs) {
+ rm_adev = container_of(l, struct gh_rsc_mgr_adev, list);
+ auxiliary_device_delete(&rm_adev->adev);
+ auxiliary_device_uninit(&rm_adev->adev);
+ }
gh_msgq_platform_host_unattach(&rsc_mgr->msgq);
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 10/11] gunyah: rsc_mgr: Add RPC for console services
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (8 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah resource manager defines a simple API for virtual machine log
sharing with the console service. A VM's own log can be opened by using
GH_VMID_SELF. Another VM's log can be accessed via its VMID. Once
opened, characters can be written to the log with a write command.
Characters are received with resource manager notifications (using ID
GH_RM_NOTIF_VM_CONSOLE_CHARS).
These high level rpc calls are kept in
drivers/virt/gunyah/rsc_mgr_rpc.c. Future RPC calls, e.g. to launch a VM
will also be maintained in this file.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/Makefile | 4 +-
drivers/virt/gunyah/rsc_mgr.h | 19 +++++
drivers/virt/gunyah/rsc_mgr_rpc.c | 129 ++++++++++++++++++++++++++++++
include/linux/gunyah_rsc_mgr.h | 15 ++++
4 files changed, 165 insertions(+), 2 deletions(-)
create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 86655bca8944..b3f15c052297 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
gunyah-y += sysfs.o device.o msgq.o
-gunyah-y += rsc_mgr.o
-obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
+gunyah-y += rsc_mgr.o rsc_mgr_rpc.o
+obj-$(CONFIG_GUNYAH) += gunyah.o
diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h
index e4f2499267bf..8cbc04b9938e 100644
--- a/drivers/virt/gunyah/rsc_mgr.h
+++ b/drivers/virt/gunyah/rsc_mgr.h
@@ -28,6 +28,25 @@
#define GH_RM_ERROR_IRQ_INUSE 0x10
#define GH_RM_ERROR_IRQ_RELEASED 0x11
+/* Message IDs: VM Services */
+#define GH_RM_RPC_VM_CONSOLE_OPEN_ID 0x56000081
+#define GH_RM_RPC_VM_CONSOLE_CLOSE_ID 0x56000082
+#define GH_RM_RPC_VM_CONSOLE_WRITE_ID 0x56000083
+#define GH_RM_RPC_VM_CONSOLE_FLUSH_ID 0x56000084
+
+/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */
+struct gh_vm_console_common_req {
+ gh_vmid_t vmid;
+ u16 reserved0;
+} __packed;
+
+/* Call: CONSOLE_WRITE */
+struct gh_vm_console_write_req {
+ gh_vmid_t vmid;
+ u16 num_bytes;
+ u8 data[0];
+} __packed;
+
int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
void **resp_buf, size_t *resp_buff_size);
diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c
new file mode 100644
index 000000000000..23e0febc1567
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr_rpc.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr: " fmt
+
+/* Contains the high level interface used by other drivers (or RM driver itself) */
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/printk.h>
+#include <linux/gunyah_rsc_mgr.h>
+
+#include "rsc_mgr.h"
+
+/**
+ * gh_rm_console_open: Open a console with a VM
+ * @vmid: VMID of the other vmid whose console to open. If VMID is GH_VMID_SELF, the
+ * console associated with this VM is opened.
+ */
+int gh_rm_console_open(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_OPEN_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_OPEN: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_open);
+
+/**
+ * gh_rm_console_close: Close a console with a VM
+ * @vmid: The vmid of the vm whose console to close.
+ */
+int gh_rm_console_close(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_CLOSE_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_CLOSE: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_close);
+
+/**
+ * gh_rm_console_write: Write to a VM's console
+ * @vmid: The vmid of the vm whose console to write to.
+ * @buf: Buffer to write to the VM's console
+ * @size: Size of the buffer
+ */
+int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size)
+{
+ void *resp;
+ struct gh_vm_console_write_req *req_payload;
+ size_t resp_size;
+ int ret = 0;
+ size_t req_payload_size = sizeof(*req_payload) + size;
+
+ if (size < 1 || size > (U32_MAX - sizeof(*req_payload)))
+ return -EINVAL;
+
+ req_payload = kzalloc(req_payload_size, GFP_KERNEL);
+
+ if (!req_payload)
+ return -ENOMEM;
+
+ req_payload->vmid = vmid;
+ req_payload->num_bytes = size;
+ memcpy(req_payload->data, buf, size);
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_WRITE_ID,
+ req_payload, req_payload_size,
+ &resp, &resp_size);
+ kfree(req_payload);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_WRITE: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_write);
+
+/**
+ * gh_rm_console_flush: Flush a console with a VM
+ * @vmid: The vmid of the vm whose console to flush
+ */
+int gh_rm_console_flush(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_FLUSH_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_FLUSH: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_flush);
diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h
index 015bd851e1a3..035bbf601e1f 100644
--- a/include/linux/gunyah_rsc_mgr.h
+++ b/include/linux/gunyah_rsc_mgr.h
@@ -26,4 +26,19 @@ struct gh_rm_notification {
int gh_rm_register_notifier(struct notifier_block *nb);
int gh_rm_unregister_notifier(struct notifier_block *nb);
+/* Notification type Message IDs */
+#define GH_RM_NOTIF_VM_CONSOLE_CHARS 0X56100080
+
+struct gh_rm_notif_vm_console_chars {
+ gh_vmid_t vmid;
+ u16 num_bytes;
+ u8 bytes[0];
+} __packed;
+
+/* RPC Calls */
+int gh_rm_console_open(gh_vmid_t vmid);
+int gh_rm_console_close(gh_vmid_t vmid);
+int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size);
+int gh_rm_console_flush(gh_vmid_t vmid);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 11/11] gunyah: Add tty console driver for RM Console Serivces
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (9 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
@ 2022-02-23 23:37 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
11 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-02-23 23:37 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-msm
Gunyah provides a console for each VM using the VM console resource
manager APIs. This driver allows console data from other
VMs to be accessed via a TTY device and exports a console device to dump
Linux's own logs to our console.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
Documentation/virt/gunyah/index.rst | 7 +
drivers/virt/gunyah/Kconfig | 13 +
drivers/virt/gunyah/Makefile | 3 +
drivers/virt/gunyah/rsc_mgr_console.c | 410 ++++++++++++++++++++++++++
4 files changed, 433 insertions(+)
create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
index e7bb2b14543e..95ba9b71ab30 100644
--- a/Documentation/virt/gunyah/index.rst
+++ b/Documentation/virt/gunyah/index.rst
@@ -90,3 +90,10 @@ When booting a virtual machine which uses a devicetree, resource manager overlay
how to communicate with resource manager, and basic description and capabilities of
this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description
of this node.
+
+Resource Manager Consoles
+-------------------------
+RM provides infrastructure for virtual machines to share an interactive console. This can be used to
+interact with a VM which may not have access to a serial port. Linux will register a printk console:
+ttyGH0. That console and other VM's consoles can be accessed via ttyGHX.
+/sys/class/tty/ttyGHX/vmid will print the VM which is associated with that TTY.
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
index 2ef4887e280d..86d5ca292c76 100644
--- a/drivers/virt/gunyah/Kconfig
+++ b/drivers/virt/gunyah/Kconfig
@@ -12,3 +12,16 @@ config GUNYAH
Say Y here to enable the drivers needed to interact in a Gunyah
virtual environment.
+
+if GUNYAH
+config GUNYAH_RESOURCE_MANAGER_CONSOLE
+ tristate "Gunyah Resource Manager Consoles"
+ depends on TTY
+ help
+ This enables support for console output using Gunyah's Resource Manager RPC.
+ This is normally used when a secondary VM which does not have exclusive access
+ to a real serial device.
+
+ If you don't have Gunyah or have other console options for secondary VMs,
+ you probably don't want this option.
+endif
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index b3f15c052297..001cf1630c03 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -3,3 +3,6 @@
gunyah-y += sysfs.o device.o msgq.o
gunyah-y += rsc_mgr.o rsc_mgr_rpc.o
obj-$(CONFIG_GUNYAH) += gunyah.o
+
+gunyah_console-y += rsc_mgr_console.o
+obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER_CONSOLE) += gunyah_console.o
diff --git a/drivers/virt/gunyah/rsc_mgr_console.c b/drivers/virt/gunyah/rsc_mgr_console.c
new file mode 100644
index 000000000000..72267bc9a315
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr_console.c
@@ -0,0 +1,410 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr_console: " fmt
+
+#include <linux/gunyah_rsc_mgr.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+#include <linux/tty_flip.h>
+#include <linux/console.h>
+#include <linux/module.h>
+#include <linux/kfifo.h>
+#include <linux/kref.h>
+#include <linux/slab.h>
+#include <linux/tty.h>
+#include <linux/of.h>
+
+/*
+ * The Linux TTY code does not support dynamic addition of tty derived devices
+ * so we need to know how many tty devices we might need when space is allocated
+ * for the tty device. Since this driver supports hotplug of vty adapters we
+ * need to make sure we have enough allocated.
+ */
+#define RSC_MGR_TTY_ADAPTERS 16
+
+/* # of payload bytes that can fit in a 1-fragment CONSOLE_WRITE message */
+#define RM_CONS_WRITE_MSG_SIZE ((1 * (GH_MSGQ_MAX_MSG_SIZE - 8)) - 4)
+
+struct rm_cons_port {
+ struct tty_port port;
+ gh_vmid_t vmid;
+ bool open;
+ unsigned int index;
+
+ DECLARE_KFIFO(put_fifo, char, 1024);
+ spinlock_t fifo_lock;
+ struct work_struct put_work;
+
+ struct rm_cons_data *cons_data;
+};
+
+struct rm_cons_data {
+ struct tty_driver *tty_driver;
+ struct device *dev;
+
+ spinlock_t ports_lock;
+ struct rm_cons_port *ports[RSC_MGR_TTY_ADAPTERS];
+
+ struct notifier_block rsc_mgr_notif;
+ struct console console;
+};
+
+static void put_work_fn(struct work_struct *ws)
+{
+ char buf[RM_CONS_WRITE_MSG_SIZE];
+ int count, ret;
+ struct rm_cons_port *port = container_of(ws, struct rm_cons_port, put_work);
+
+ while (!kfifo_is_empty(&port->put_fifo)) {
+ count = kfifo_out_spinlocked(&port->put_fifo, buf, sizeof(buf), &port->fifo_lock);
+ if (count <= 0)
+ continue;
+
+ ret = gh_rm_console_write(port->vmid, buf, count);
+ if (ret) {
+ pr_warn_once("failed to send characters: %d\n", ret);
+ break;
+ }
+ }
+}
+
+static int rsc_mgr_console_notif(struct notifier_block *nb, unsigned long cmd, void *data)
+{
+ int count, i;
+ struct rm_cons_port *rm_port;
+ struct tty_port *tty_port = NULL;
+ struct rm_cons_data *cons_data = container_of(nb, struct rm_cons_data, rsc_mgr_notif);
+ const struct gh_rm_notification *notif = data;
+ struct gh_rm_notif_vm_console_chars const * const msg = notif->buff;
+
+ if (cmd != GH_RM_NOTIF_VM_CONSOLE_CHARS ||
+ notif->size < sizeof(*msg))
+ return NOTIFY_DONE;
+
+ spin_lock(&cons_data->ports_lock);
+ for (i = 0; i < RSC_MGR_TTY_ADAPTERS; i++) {
+ if (!cons_data->ports[i])
+ continue;
+ if (cons_data->ports[i]->vmid == msg->vmid) {
+ rm_port = cons_data->ports[i];
+ break;
+ }
+ }
+ if (rm_port)
+ tty_port = tty_port_get(&rm_port->port);
+ spin_unlock(&cons_data->ports_lock);
+
+ if (!rm_port)
+ pr_warn("Received unexpected console characters for VMID %u\n", msg->vmid);
+ if (!tty_port)
+ return NOTIFY_DONE;
+
+ count = tty_buffer_request_room(tty_port, msg->num_bytes);
+ tty_insert_flip_string(tty_port, msg->bytes, count);
+ tty_flip_buffer_push(tty_port);
+
+ tty_port_put(tty_port);
+ return NOTIFY_OK;
+}
+
+static ssize_t vmid_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(dev);
+
+ if (rm_port->vmid == GH_VMID_SELF)
+ return sysfs_emit(buf, "self\n");
+
+ return sysfs_emit(buf, "%u\n", rm_port->vmid);
+}
+
+static DEVICE_ATTR_RO(vmid);
+
+static struct attribute *rsc_mgr_tty_dev_attrs[] = {
+ &dev_attr_vmid.attr,
+ NULL
+};
+
+static const struct attribute_group rsc_mgr_tty_dev_attr_group = {
+ .attrs = rsc_mgr_tty_dev_attrs,
+};
+
+static const struct attribute_group *rsc_mgr_tty_dev_attr_groups[] = {
+ &rsc_mgr_tty_dev_attr_group,
+ NULL
+};
+
+static int rsc_mgr_tty_open(struct tty_struct *tty, struct file *filp)
+{
+ int ret;
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ if (!rm_port->open) {
+ ret = gh_rm_console_open(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = true;
+ }
+
+ return tty_port_open(&rm_port->port, tty, filp);
+}
+
+static void rsc_mgr_tty_close(struct tty_struct *tty, struct file *filp)
+{
+ int ret;
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ if (rm_port->open) {
+ if (rm_port->vmid != GH_VMID_SELF) {
+ ret = gh_rm_console_close(rm_port->vmid);
+ if (ret)
+ pr_warn("Failed to close RM console for vmid %d: %d\n",
+ rm_port->vmid, ret);
+ }
+ rm_port->open = false;
+
+ tty_port_close(&rm_port->port, tty, filp);
+ }
+
+}
+
+static int rsc_mgr_tty_write(struct tty_struct *tty, const unsigned char *buf, int count)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+ int ret;
+
+ ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock);
+ if (ret > 0)
+ schedule_work(&rm_port->put_work);
+
+ return ret;
+}
+
+static unsigned int rsc_mgr_mgr_tty_write_room(struct tty_struct *tty)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ return kfifo_avail(&rm_port->put_fifo);
+}
+
+static void rsc_mgr_console_write(struct console *co, const char *buf, unsigned count)
+{
+ struct rm_cons_port *rm_port = co->data;
+ int ret;
+
+ ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock);
+ if (ret > 0)
+ schedule_work(&rm_port->put_work);
+}
+
+static struct tty_driver *rsc_mgr_console_device(struct console *co, int *index)
+{
+ struct rm_cons_port *rm_port = co->data;
+
+ *index = rm_port->index;
+ return rm_port->port.tty->driver;
+}
+
+static int rsc_mgr_console_setup(struct console *co, char *unused)
+{
+ int ret;
+ struct rm_cons_port *rm_port = co->data;
+
+ if (!rm_port->open) {
+ ret = gh_rm_console_open(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = true;
+ }
+
+ return 0;
+}
+
+static int rsc_mgr_console_exit(struct console *co)
+{
+ int ret;
+ struct rm_cons_port *rm_port = co->data;
+
+ if (rm_port->open) {
+ ret = gh_rm_console_close(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to close RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = false;
+ }
+
+ return 0;
+}
+
+static const struct tty_operations rsc_mgr_tty_ops = {
+ .open = rsc_mgr_tty_open,
+ .close = rsc_mgr_tty_close,
+ .write = rsc_mgr_tty_write,
+ .write_room = rsc_mgr_mgr_tty_write_room,
+};
+
+static void rsc_mgr_port_destruct(struct tty_port *port)
+{
+ struct rm_cons_port *rm_port = container_of(port, struct rm_cons_port, port);
+ struct rm_cons_data *cons_data = rm_port->cons_data;
+
+ spin_lock(&cons_data->ports_lock);
+ WARN_ON(cons_data->ports[rm_port->index] != rm_port);
+ cons_data->ports[rm_port->index] = NULL;
+ spin_unlock(&cons_data->ports_lock);
+ kfree(rm_port);
+}
+
+static const struct tty_port_operations rsc_mgr_port_ops = {
+ .destruct = rsc_mgr_port_destruct,
+};
+
+static struct rm_cons_port *rsc_mgr_port_create(struct rm_cons_data *cons_data, gh_vmid_t vmid)
+{
+ struct rm_cons_port *rm_port;
+ struct device *ttydev;
+ unsigned int index;
+ int ret;
+
+ rm_port = kzalloc(sizeof(*rm_port), GFP_KERNEL);
+ rm_port->vmid = vmid;
+ INIT_KFIFO(rm_port->put_fifo);
+ spin_lock_init(&rm_port->fifo_lock);
+ INIT_WORK(&rm_port->put_work, put_work_fn);
+ tty_port_init(&rm_port->port);
+ rm_port->port.ops = &rsc_mgr_port_ops;
+
+ spin_lock(&cons_data->ports_lock);
+ for (index = 0; index < RSC_MGR_TTY_ADAPTERS; index++) {
+ if (!cons_data->ports[index]) {
+ cons_data->ports[index] = rm_port;
+ rm_port->index = index;
+ break;
+ }
+ }
+ spin_unlock(&cons_data->ports_lock);
+ if (index >= RSC_MGR_TTY_ADAPTERS) {
+ ret = -ENOSPC;
+ goto err_put_port;
+ }
+
+ ttydev = tty_port_register_device_attr(&rm_port->port, cons_data->tty_driver, index,
+ cons_data->dev, rm_port, rsc_mgr_tty_dev_attr_groups);
+ if (IS_ERR(ttydev)) {
+ ret = PTR_ERR(ttydev);
+ goto err_put_port;
+ }
+
+ return rm_port;
+err_put_port:
+ tty_port_put(&rm_port->port);
+ return ERR_PTR(ret);
+}
+
+static int rsc_mgr_console_probe(struct auxiliary_device *auxdev,
+ const struct auxiliary_device_id *aux_dev_id)
+{
+ struct rm_cons_data *cons_data;
+ struct rm_cons_port *rm_port;
+ int ret;
+ struct device_node *hyp_node, *vm_of_node;
+ u32 vmid;
+
+ cons_data = devm_kzalloc(&auxdev->dev, sizeof(*cons_data), GFP_KERNEL);
+ if (!cons_data)
+ return -ENOMEM;
+ dev_set_drvdata(&auxdev->dev, cons_data);
+ cons_data->dev = &auxdev->dev;
+
+ cons_data->tty_driver = tty_alloc_driver(RSC_MGR_TTY_ADAPTERS,
+ TTY_DRIVER_REAL_RAW | TTY_DRIVER_DYNAMIC_DEV);
+ if (IS_ERR(cons_data->tty_driver))
+ return PTR_ERR(cons_data->tty_driver);
+
+ cons_data->tty_driver->driver_name = "gh";
+ cons_data->tty_driver->name = "ttyGH";
+ cons_data->tty_driver->type = TTY_DRIVER_TYPE_SYSTEM;
+ cons_data->tty_driver->init_termios = tty_std_termios;
+ tty_set_operations(cons_data->tty_driver, &rsc_mgr_tty_ops);
+
+ ret = tty_register_driver(cons_data->tty_driver);
+ if (ret) {
+ dev_err(&auxdev->dev, "Could not register tty driver: %d\n", ret);
+ goto err_put_tty;
+ }
+
+ spin_lock_init(&cons_data->ports_lock);
+
+ cons_data->rsc_mgr_notif.notifier_call = rsc_mgr_console_notif;
+ ret = gh_rm_register_notifier(&cons_data->rsc_mgr_notif);
+ if (ret) {
+ dev_err(&auxdev->dev, "Could not register for resource manager notifications: %d\n",
+ ret);
+ goto err_put_tty;
+ }
+
+ rm_port = rsc_mgr_port_create(cons_data, GH_VMID_SELF);
+ if (IS_ERR(rm_port)) {
+ ret = PTR_ERR(rm_port);
+ dev_err(&auxdev->dev, "Could not create own console: %d\n", ret);
+ goto err_unreg_notif;
+ }
+
+ strncpy(cons_data->console.name, "ttyGH", sizeof(cons_data->console.name));
+ cons_data->console.write = rsc_mgr_console_write;
+ cons_data->console.device = rsc_mgr_console_device;
+ cons_data->console.setup = rsc_mgr_console_setup;
+ cons_data->console.exit = rsc_mgr_console_exit;
+ cons_data->console.index = rm_port->index;
+ cons_data->console.data = rm_port;
+ register_console(&cons_data->console);
+
+ hyp_node = of_get_parent(auxdev->dev.parent->of_node);
+ vm_of_node = of_find_compatible_node(hyp_node, NULL, "qcom,gunyah-vm-id");
+ if (vm_of_node) {
+ ret = of_property_read_u32(vm_of_node, "qcom,vmid", &vmid);
+ if (ret)
+ return 0;
+ rm_port = rsc_mgr_port_create(cons_data, vmid);
+ }
+
+ return 0;
+err_unreg_notif:
+ gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif);
+err_put_tty:
+ tty_driver_kref_put(cons_data->tty_driver);
+ return ret;
+}
+
+static void rsc_mgr_console_remove(struct auxiliary_device *auxdev)
+{
+ struct rm_cons_data *cons_data = dev_get_drvdata(&auxdev->dev);
+
+ unregister_console(&cons_data->console);
+ gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif);
+ tty_driver_kref_put(cons_data->tty_driver);
+}
+
+static struct auxiliary_device_id rsc_mgr_console_ids[] = {
+ { .name = "gunyah.console" },
+ {}
+};
+MODULE_DEVICE_TABLE(auxiliary, rsc_mgr_console_ids);
+
+static struct auxiliary_driver rsc_mgr_console_drv = {
+ .probe = rsc_mgr_console_probe,
+ .remove = rsc_mgr_console_remove,
+ .id_table = rsc_mgr_console_ids,
+};
+module_auxiliary_driver(rsc_mgr_console_drv);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Gunyah Console");
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI
2022-02-23 23:37 ` [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
@ 2022-02-24 10:10 ` Mark Rutland
2022-02-24 10:26 ` Marc Zyngier
0 siblings, 1 reply; 30+ messages in thread
From: Mark Rutland @ 2022-02-24 10:10 UTC (permalink / raw)
To: Elliot Berman
Cc: Bjorn Andersson, linux-kernel, Trilok Soni, Murali Nalajala,
Srivatsa Vaddagiri, Carl van Schaik, Andy Gross, linux-arm-msm,
linux-arm-kernel, Lorenzo Pieralisi, Sudeep Holla, Marc Zyngier
Hi,
As a general thing, this is the *only* patch from this series which has
been Cc'd to linux-arm-kernel, which makes it practically impossible to
understand the context for this, which is somewhat frustrating.
Looking on lore.kernel.org I see that the entire series was Cc'd to
linux-arm-msm, but most people don't subscribe to that list. If you send
one patch in a series to a list, please send the *entire* series there.
On Wed, Feb 23, 2022 at 03:37:21PM -0800, Elliot Berman wrote:
> Add initial support to perform Gunyah hypercalls. The arm64 ABI for
> Gunyah hypercalls generally follows the AAPCS64, and can be summarized:
> - Function identifier is passed through the imm operand
> - [r0,r7] are parameter and result registers
> - [r8-r18] are temporary and saved by the caller (VM)
> - [r19-r31] are preserved and saved by the hypervisor
>
> The preprocessor macors for creating the necessary HVC instruction
> roughly follows the SMCCC 1.1 implementation in
> include/linux/arm-smccc.h.
I've added the SMCCC maintainers (myself, Lorenzo, and SUdeep) to Cc,
and also Marc who was involvedi n prior discussions in this area. Please
Cc us on any future patches adding HVC or SMCC interfaces (SMCCC or
otherwise).
We've previously said NO to any new hypercall mechanisms which do not
follow SMCCC. There is no reason to fragment this space further; please
use SMCCC (which your hypervisor must already implement in part if it
exposes PSCI to a guest).
NAK to this non-SMCCC interface.
Thanks,
Mark.
>
> Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
> ---
> MAINTAINERS | 1 +
> arch/arm64/include/asm/gunyah/hypercall.h | 193 ++++++++++++++++++++++
> 2 files changed, 194 insertions(+)
> create mode 100644 arch/arm64/include/asm/gunyah/hypercall.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 6a918f653eac..7e6a8488fa3e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8402,6 +8402,7 @@ L: linux-arm-msm@vger.kernel.org
> S: Maintained
> F: Documentation/devicetree/bindings/gunyah/
> F: Documentation/virt/gunyah/
> +F: arch/arm64/include/asm/gunyah/
>
> H8/300 ARCHITECTURE
> M: Yoshinori Sato <ysato@users.sourceforge.jp>
> diff --git a/arch/arm64/include/asm/gunyah/hypercall.h b/arch/arm64/include/asm/gunyah/hypercall.h
> new file mode 100644
> index 000000000000..626163500e32
> --- /dev/null
> +++ b/arch/arm64/include/asm/gunyah/hypercall.h
> @@ -0,0 +1,193 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +#ifndef __ASM_GH_HYPERCALL_H
> +#define __ASM_GH_HYPERCALL_H
> +
> +#include <linux/types.h>
> +
> +#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
> +
> +#define __gh_count_args(...) \
> + ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0)
> +
> +#define __gh_skip_0(...) __VA_ARGS__
> +#define __gh_skip_1(a, ...) __VA_ARGS__
> +#define __gh_skip_2(a, b, ...) __VA_ARGS__
> +#define __gh_skip_3(a, b, c, ...) __VA_ARGS__
> +#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__
> +#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__
> +#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__
> +#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__
> +#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__
> +
> +#define __gh_declare_arg_0(...)
> +
> +#define __gh_declare_arg_1(a1, ...) \
> + typeof(a1) __gh_a1 = (a1); \
> + register uintptr_t arg1 asm("r0") = __gh_a1
> +
> +#define __gh_declare_arg_2(a1, a2, ...) \
> + __gh_declare_arg_1(a1); \
> + typeof(a2) __gh_a2 = (a2); \
> + register uintptr_t arg2 asm("r1") = __gh_a2
> +
> +#define __gh_declare_arg_3(a1, a2, a3, ...) \
> + __gh_declare_arg_2(a1, a2); \
> + typeof(a3) __gh_a3 = (a3); \
> + register uintptr_t arg3 asm("r2") = __gh_a3
> +
> +#define __gh_declare_arg_4(a1, a2, a3, a4, ...) \
> + __gh_declare_arg_3(a1, a2, a3); \
> + typeof(a4) __gh_a4 = (a4); \
> + register uintptr_t arg4 asm("r3") = __gh_a4
> +
> +#define __gh_declare_arg_5(a1, a2, a3, a4, a5, ...) \
> + __gh_declare_arg_4(a1, a2, a3, a4); \
> + typeof(a5) __gh_a5 = (a5); \
> + register uintptr_t arg5 asm("r4") = __gh_a5
> +
> +#define __gh_declare_arg_6(a1, a2, a3, a4, a5, a6, ...) \
> + __gh_declare_arg_5(a1, a2, a3, a4, a5); \
> + typeof(a6) __gh_a6 = (a6); \
> + register uintptr_t arg6 asm("r5") = __gh_a6
> +
> +#define __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7, ...) \
> + __gh_declare_arg_6(a1, a2, a3, a4, a5, a6); \
> + typeof(a7) __gh_a7 = (a7); \
> + register uintptr_t arg7 asm("r6") = __gh_a7
> +
> +#define __gh_declare_arg_8(a1, a2, a3, a4, a5, a6, a7, a8, ...) \
> + __gh_declare_arg_7(a1, a2, a3, a4, a5, a6, a7); \
> + typeof(a8) __gh_a8 = (a8); \
> + register uintptr_t arg8 asm("r7") = __gh_a8
> +
> +#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs
> +#define __gh_declare_args(nargs) ___gh_declare_args(nargs)
> +#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__)
> +
> +#define __gh_constraint_arg_0
> +#define __gh_constraint_arg_1 "r" (arg1),
> +#define __gh_constraint_arg_2 __gh_constraint_arg_1 "r" (arg2),
> +#define __gh_constraint_arg_3 __gh_constraint_arg_2 "r" (arg3),
> +#define __gh_constraint_arg_4 __gh_constraint_arg_3 "r" (arg4),
> +#define __gh_constraint_arg_5 __gh_constraint_arg_4 "r" (arg5),
> +#define __gh_constraint_arg_6 __gh_constraint_arg_5 "r" (arg6),
> +#define __gh_constraint_arg_7 __gh_constraint_arg_6 "r" (arg7),
> +#define __gh_constraint_arg_8 __gh_constraint_arg_7 "r" (arg8),
> +
> +#define _gh_constraint_args(nargs) __gh_constraint_arg_ ## nargs
> +
> +#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__)
> +
> +#define __gh_declare_res_0
> +
> +#define __gh_declare_res_1 \
> + register uintptr_t res1 asm("r0")
> +
> +#define __gh_declare_res_2 \
> + __gh_declare_res_1; \
> + register uintptr_t res2 asm("r1")
> +
> +#define __gh_declare_res_3 \
> + __gh_declare_res_2; \
> + register uintptr_t res3 asm("r2")
> +
> +#define __gh_declare_res_4 \
> + __gh_declare_res_3; \
> + register uintptr_t res4 asm("r3")
> +
> +#define __gh_declare_res_5 \
> + __gh_declare_res_4; \
> + register uintptr_t res5 asm("r4")
> +
> +#define __gh_declare_res_6 \
> + __gh_declare_res_5; \
> + register uintptr_t res6 asm("r5")
> +
> +#define __gh_declare_res_7 \
> + __gh_declare_res_6; \
> + register uintptr_t res7 asm("r6")
> +
> +#define __gh_declare_res_8 \
> + __gh_declare_res_7; \
> + register uintptr_t res8 asm("r7")
> +
> +#define ___gh_declare_res(nargs) __gh_declare_res_ ## nargs
> +#define __gh_declare_res(nargs) ___gh_declare_res(nargs)
> +#define _gh_declare_res(...) __gh_declare_res(__gh_count_args(__VA_ARGS__))
> +
> +#define __gh_constraint_res_0
> +#define __gh_constraint_res_1 "=r" (res1)
> +#define __gh_constraint_res_2 __gh_constraint_res_1, "=r" (res2)
> +#define __gh_constraint_res_3 __gh_constraint_res_2, "=r" (res3)
> +#define __gh_constraint_res_4 __gh_constraint_res_3, "=r" (res4)
> +#define __gh_constraint_res_5 __gh_constraint_res_4, "=r" (res5)
> +#define __gh_constraint_res_6 __gh_constraint_res_5, "=r" (res6)
> +#define __gh_constraint_res_7 __gh_constraint_res_6, "=r" (res7)
> +#define __gh_constraint_res_8 __gh_constraint_res_7, "=r" (res8)
> +
> +#define ___gh_constraint_res(nargs) __gh_constraint_res_ ## nargs
> +#define __gh_constraint_res(nargs) ___gh_constraint_res(nargs)
> +#define _gh_constraint_res(...) \
> + __gh_constraint_res(__gh_count_args(__VA_ARGS__))
> +
> +#define __gh_assign_res_0(...)
> +
> +#define __gh_assign_res_1(r1) \
> + r1 = res1;
> +
> +#define __gh_assign_res_2(r1, r2) \
> + __gh_assign_res_1(r1); \
> + r2 = res2
> +
> +#define __gh_assign_res_3(r1, r2, r3) \
> + __gh_assign_res_2(r1, r2); \
> + r3 = res3
> +
> +#define __gh_assign_res_4(r1, r2, r3, r4) \
> + __gh_assign_res_3(r1, r2, r3); \
> + r4 = res4
> +
> +#define __gh_assign_res_5(r1, r2, r3, r4, r5) \
> + __gh_assign_res_4(r1, r2, r3, r4); \
> + r5 = res5
> +
> +#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \
> + __gh_assign_res_5(r1, r2, r3, r4, r5); \
> + r6 = res6
> +
> +#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \
> + __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \
> + r7 = res7
> +
> +#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \
> + __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \
> + r8 = res8
> +
> +#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs
> +#define __gh_assign_res(nargs) ___gh_assign_res(nargs)
> +#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__)
> +
> +/**
> + * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI
> + * @hcall_num: Hypercall function ID to invoke
> + * @nargs: Number of input arguments
> + * @...: First nargs are the input arguments. Remaining arguments are output variables.
> + */
> +#define arch_gh_hypercall(hcall_num, nargs, ...) \
> + do { \
> + _gh_declare_res(__gh_to_res(nargs, __VA_ARGS__)); \
> + _gh_declare_args(nargs, __VA_ARGS__); \
> + asm volatile( \
> + "hvc %[num]\n" \
> + : _gh_constraint_res(__gh_to_res(nargs, __VA_ARGS__)) \
> + : _gh_constraint_args(nargs) \
> + [num] "i" (hcall_num) \
> + : "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", \
> + "memory"); \
> + _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \
> + } while (0)
> +
> +#endif
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI
2022-02-24 10:10 ` Mark Rutland
@ 2022-02-24 10:26 ` Marc Zyngier
2022-04-13 17:09 ` Elliot Berman
0 siblings, 1 reply; 30+ messages in thread
From: Marc Zyngier @ 2022-02-24 10:26 UTC (permalink / raw)
To: Elliot Berman, Mark Rutland
Cc: Bjorn Andersson, linux-kernel, Trilok Soni, Murali Nalajala,
Srivatsa Vaddagiri, Carl van Schaik, Andy Gross, linux-arm-msm,
linux-arm-kernel, Lorenzo Pieralisi, Sudeep Holla
Thanks Mark for roping me in.
On Thu, 24 Feb 2022 10:10:02 +0000,
Mark Rutland <mark.rutland@arm.com> wrote:
>
> Hi,
>
> As a general thing, this is the *only* patch from this series which has
> been Cc'd to linux-arm-kernel, which makes it practically impossible to
> understand the context for this, which is somewhat frustrating.
>
> Looking on lore.kernel.org I see that the entire series was Cc'd to
> linux-arm-msm, but most people don't subscribe to that list. If you send
> one patch in a series to a list, please send the *entire* series there.
>
> On Wed, Feb 23, 2022 at 03:37:21PM -0800, Elliot Berman wrote:
> > Add initial support to perform Gunyah hypercalls. The arm64 ABI for
> > Gunyah hypercalls generally follows the AAPCS64, and can be summarized:
> > - Function identifier is passed through the imm operand
> > - [r0,r7] are parameter and result registers
> > - [r8-r18] are temporary and saved by the caller (VM)
> > - [r19-r31] are preserved and saved by the hypervisor
> >
> > The preprocessor macors for creating the necessary HVC instruction
> > roughly follows the SMCCC 1.1 implementation in
> > include/linux/arm-smccc.h.
>
> I've added the SMCCC maintainers (myself, Lorenzo, and SUdeep) to Cc,
> and also Marc who was involvedi n prior discussions in this area. Please
> Cc us on any future patches adding HVC or SMCC interfaces (SMCCC or
> otherwise).
In general, please Cc all the interested parties with the whole
series. Random patches randomly cc'd out of context are pretty useless
and only lead to them being ignored.
>
> We've previously said NO to any new hypercall mechanisms which do not
> follow SMCCC. There is no reason to fragment this space further; please
> use SMCCC (which your hypervisor must already implement in part if it
> exposes PSCI to a guest).
>
> NAK to this non-SMCCC interface.
Agreed. We pushed back on that for Hyper-V, and I don't see a reason
for changing tack on that.
The calling convention exists for a reason: portability. If this
hypervisor is to be "independent of any high-level OS kernel" (as it
is being advertised), then it must already implement SMCCC.
What is the issue with properly supporting SMCCC for all interactions
with the hypervisor and not reinventing a square wheel?
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor
2022-02-23 23:37 ` [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
@ 2022-02-25 20:01 ` Rob Herring
2022-03-02 23:53 ` Elliot Berman
0 siblings, 1 reply; 30+ messages in thread
From: Rob Herring @ 2022-02-25 20:01 UTC (permalink / raw)
To: Elliot Berman
Cc: Bjorn Andersson, linux-kernel, devicetree, Trilok Soni,
Murali Nalajala, Srivatsa Vaddagiri, Carl van Schaik, Andy Gross,
linux-arm-msm
On Wed, Feb 23, 2022 at 03:37:20PM -0800, Elliot Berman wrote:
> When Linux is booted as a guest under the Gunyah hypervisor, Gunyah
> applies a devicetree overlay describing the virtual platform
> configuration of the guest VM, such as the message queue capability IDs
> for communicating with the Resource Manager. Add the DT bindings that
> Gunyah adheres for the hypervisor node and message queues.
>
> Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
> ---
> .../bindings/gunyah/message-queue.yml | 100 ++++++++++++++
> .../bindings/gunyah/qcom,hypervisor.yml | 122 ++++++++++++++++++
How did testing these files work? It didn't because .yml files are
ignored. Get 'make dt_binding_check' actually working and resubmit.
No, you don't get your own directory.
In general, DT is for undiscoverable hardware that we are stuck with
because it was not made discoverable. As this is not h/w and you control
each side of the interface, make it discoverable and don't use DT for
your discovery mechanism.
Incomplete review follows...
> MAINTAINERS | 1 +
> 3 files changed, 223 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/gunyah/message-queue.yml
> create mode 100644 Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
>
> diff --git a/Documentation/devicetree/bindings/gunyah/message-queue.yml b/Documentation/devicetree/bindings/gunyah/message-queue.yml
> new file mode 100644
> index 000000000000..1a96d3de2a19
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/gunyah/message-queue.yml
> @@ -0,0 +1,100 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Gunyah message queue
> +
> +maintainers:
> + - Murali Nalajala <quic_mnalajal@quicinc.com>
> + - Elliot Berman <quic_eberman@quicinc.com>
> +
> +properties:
> + compatible:
> + items:
> + - const: qcom,gunyah-message-queue
> + - const: qcom,gunyah-capability
I'm not following how capability is a fallback to message-queue.
> + peer:
> + description: VMID of the VM on the other end of message queue
> + $ref: /schemas/types.yaml#/definitions/uint32
> + allOf:
Check your indentation.
> + - if:
> + anyOf:
> + - properties:
> + qcom,is-sender: true
> + - properties:
> + qcom,is-full-duplex: true
> + then:
> + properties:
> + qcom,tx-message-size:
> + description: Maximum size in bytes of a message which can be sent by this queue
> + $ref: /schemas/types.yaml#/definitions/int32
> + qcom,tx-queue-depth:
> + description: Depth of transmit queue for messages sent by this queue
> + $ref: /schemas/types.yaml#/definitions/int32
> + - if:
> + anyOf:
> + - properties:
> + qcom,is-receiver: true
> + - properties:
> + qcom,is-full-duplex: true
> + then:
> + properties:
> + qcom,rx-message-size:
> + description: Maximum size in bytes of a message which can be received by this queue
> + $ref: /schemas/types.yaml#/definitions/int32
> + qcom,rx-queue-depth:
> + description: Depth of transmit queue for messages received by this queue
> + $ref: /schemas/types.yaml#/definitions/int32
> + - if:
> + anyOf:
> + - properties:
> + qcom,is-receiver: true
> + - properties:
> + qcom,is-sender: true
> + then:
> + properties:
> + reg:
> + description: Hypervisor capability ID of the message queue
> + $ref: /schemas/types.yaml#/definitions/uint32
> + minItems: 1
> + maxItems: 1
> + interrupts:
> + minItems: 1
> + maxItems: 1
> + - if:
> + properties:
> + qcom,is-full-duplex: true
> + then:
> + properties:
> + reg:
> + description:
> + Hypervisor capability IDs of the message queue
> + The first is tx side, the second is rx side
> + $ref: /schemas/types.yaml#/definitions/uint32
> + minItems: 2
> + maxItems: 2
> + interrupts:
> + description: The first is tx interrupt, second is rx interrupt
> + minItems: 2
> + maxItems: 2
> + required:
> + - compatible
> + - reg
> + - interrupts
> +
> +
> +examples:
> + - |
> + display-msgq-pair@abbf0da3c3c965cc {
> + compatible = "qcom,gunyah-message-queue", "qcom,gunyah-capability";
> + interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
> + <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
> + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; /* TX, RX cap ids */
> + qcom,is-full-duplex;
> + qcom,tx-queue-depth = <8>;
> + qcom,tx-message-size = <0xf0>;
> + qcom,rx-queue-depth = <8>;
> + qcom,rx-message-size = <0xf0>;
> + };
> \ No newline at end of file
Fix this.
> diff --git a/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
> new file mode 100644
> index 000000000000..f637d51c52f0
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
> @@ -0,0 +1,122 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Hypervisor node to define virtual devices and other services provided by a Gunyah hypervisor
> + to this virtual machine.
> +
> +maintainers:
> + - Murali Nalajala <quic_mnalajal@quicinc.com>
> + - Elliot Berman <quic_eberman@quicinc.com>
> +
> +description: |+
> + On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which
> + describes the basic configuration of the hypervisor. Virtual machines use this information for
> + initial discovery that they are running as a Gunyah guest VM.
> + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c
> +
> +properties:
> + compatible:
> + oneOf:
> + - items:
> + - const: qcom,gunyah-hypervisor-1.0
> + - const: qcom,gunyah-hypervisor
> +
> + "#address-cells":
> + description: Number of cells needed to represent 64-bit capability IDs.
> + const: 2
> + "#size-cells":
> + description: must be 0, because capability IDs are not memory address
> + ranges and do not have a size.
> + const: 0
> +
> + qcom,gunyah-vm:
> + type: object
> + description:
> + The VM Identification is a virtual node that conveys to the VM information
> + about this virtual machine in the context of the hypervisor-based system
> + properties:
> + compatible:
> + oneOf:
> + - items:
> + - const: qcom,gunyah-vm-id-1.0
> + - const: qcom,gunyah-vm-id
> + qcom,vendor:
> + $ref: /schemas/types.yaml#/definitions/string
> + description: Vendor of the Virtual Machine, e.g. Qualcomm
Doesn't the compatible say this already?
> + qcom,vmid:
> + $ref: /schemas/types.yaml#/definitions/uint32
> + description: contains the VMID of this VM as a 32-bit value
> + qcom,owner-vmid:
> + $ref: /schemas/types.yaml#/definitions/uint32
> + description: Contains the hypervisor VMID of the VM's owner. The owner
> + is the VM that allocated and created the VM. VMs directly
> + managed by the resource manager, such as the primary VM do
> + not have an owner.
> + required:
> + - compatible
> + - qcom,vmid
> + - qcom,owner-vmid
> +
> +patternProperties:
> + "^qcom,resource-manager-rpc(@.*)?":
We don't use vendor prefixes in node names. QCom really liked to though.
> + type: object
> + description:
> + Resource Manager node which is required to communicate to Resource
> + Manager VM using Gunyah Message Queues.
> + allOf: "message-queue.yml#"
Not valid json-schema...
> +
> + properties:
> + compatible:
> + oneOf:
> + items:
> + - const: qcom,resource-manager-1-0
> + - const: qcom,resource-manager
> + qcom,console-dev:
> + $ref: /schemas/types.yaml#/definitions/flag
> + description: if set, the resource-manger will accept console logs from the VM
> + qcom,free-irq-start:
> + $ref: /schemas/types.yaml#/definitions/uint32
> + description: Set on ARM systems which use a GIC. First VIRQ number which is free
> + for virtual interrupt use.
> + required:
> + - qcom,is-full-duplex
> +
> +
> +required:
> +- compatible
> +- "#address-cells"
> +- "#size-cells"
> +
> +examples:
> + - |
> + hypervisor {
> + #address-cells = <2>;
> + #size-cells = <0>;
> + compatible = "qcom,gunyah-hypervisor-1.0", "qcom,gunyah-hypervisor", "simple-bus";
> + name = "hypervisor";
> +
> + qcom,gunyah-vm {
> + compatible = "qcom,gunyah-vm-id-1.0", "qcom,gunyah-vm-id";
> + qcom,vendor = "Qualcomm Technologies, Inc.";
> + qcom,vmid = <45>;
> + qcom,owner-vmid = <3>;
> + };
> +
> + qcom,resource-manager-rpc@0000000000000001 {
> + compatible = "qcom,resource-manager-1-0", "qcom,resource-manager",
> + "qcom,gunyah-message-queue", "qcom,gunyah-capability";
> + interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
> + <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
> + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>;
> + /* TX, RX cap ids */
> + qcom,is-full-duplex;
> + qcom,free-irq-start = <0>;
> + qcom,tx-queue-depth = <8>;
> + qcom,tx-message-size = <0xf0>;
> + qcom,rx-queue-depth = <8>;
> + qcom,rx-message-size = <0xf0>;
> + };
> + };
> diff --git a/MAINTAINERS b/MAINTAINERS
> index bed175adc4c3..6a918f653eac 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8400,6 +8400,7 @@ M: Elliot Berman <quic_eberman@quicinc.com>
> M: Murali Nalajala <quic_mnalajal@quicinc.com>
> L: linux-arm-msm@vger.kernel.org
> S: Maintained
> +F: Documentation/devicetree/bindings/gunyah/
> F: Documentation/virt/gunyah/
>
> H8/300 ARCHITECTURE
> --
> 2.25.1
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor
2022-02-25 20:01 ` Rob Herring
@ 2022-03-02 23:53 ` Elliot Berman
0 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-03-02 23:53 UTC (permalink / raw)
To: Rob Herring
Cc: Bjorn Andersson, linux-kernel, devicetree, Trilok Soni,
Carl van Schaik, Andy Gross, linux-arm-msm, Murali Nalajala
On 2/25/2022 12:01 PM, Rob Herring wrote:
> On Wed, Feb 23, 2022 at 03:37:20PM -0800, Elliot Berman wrote:
>> When Linux is booted as a guest under the Gunyah hypervisor, Gunyah
>> applies a devicetree overlay describing the virtual platform
>> configuration of the guest VM, such as the message queue capability IDs
>> for communicating with the Resource Manager. Add the DT bindings that
>> Gunyah adheres for the hypervisor node and message queues.
>>
>> Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
>> ---
>> .../bindings/gunyah/message-queue.yml | 100 ++++++++++++++
>> .../bindings/gunyah/qcom,hypervisor.yml | 122 ++++++++++++++++++
>
> How did testing these files work? It didn't because .yml files are
> ignored. Get 'make dt_binding_check' actually working and resubmit.
>
I'll double check this, thanks!
> No, you don't get your own directory.
>
Do you have a suggestion for an alternate directory? I'm not sure if
misc/ makes sense here?
> In general, DT is for undiscoverable hardware that we are stuck with
> because it was not made discoverable. As this is not h/w and you control
> each side of the interface, make it discoverable and don't use DT for
> your discovery mechanism.
>
Gunyah follows a micro-kernel architecture. The EL2 hypervisor doesn't
keep the necessary context to share initial message queue information in
order to communicate with the resource manager VM, and it's considered
undiscoverable in this regard.
This feedback is good for the other properties, though. I'll improve the
discoverability for other properties and drop the bindings for them.
In summary, I can remove all of the properties and subnodes from
/hypervisor node, except for /hypervisor/resource-manager-rpc. For that
node, I only need to keep a few fields (reg and interrupts) because that
information isn't discoverable.
> Incomplete review follows...
>
>> MAINTAINERS | 1 +
>> 3 files changed, 223 insertions(+)
>> create mode 100644 Documentation/devicetree/bindings/gunyah/message-queue.yml
>> create mode 100644 Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
>>
>> diff --git a/Documentation/devicetree/bindings/gunyah/message-queue.yml b/Documentation/devicetree/bindings/gunyah/message-queue.yml
>> new file mode 100644
>> index 000000000000..1a96d3de2a19
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/gunyah/message-queue.yml
>> @@ -0,0 +1,100 @@
>> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Gunyah message queue
>> +
>> +maintainers:
>> + - Murali Nalajala <quic_mnalajal@quicinc.com>
>> + - Elliot Berman <quic_eberman@quicinc.com>
>> +
>> +properties:
>> + compatible:
>> + items:
>> + - const: qcom,gunyah-message-queue
>> + - const: qcom,gunyah-capability
>
> I'm not following how capability is a fallback to message-queue.
>
This node shares common properties with other Gunyah capabilities. Linux
drivers don't need to make use of this compatible, but Gunyah needs to
make it present for other operating systems. In that case, should I drop
"gunyah-capability" from the bindings?
>> + peer:
>> + description: VMID of the VM on the other end of message queue
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + allOf:
>
> Check your indentation.
>
>> + - if:
>> + anyOf:
>> + - properties:
>> + qcom,is-sender: true
>> + - properties:
>> + qcom,is-full-duplex: true
>> + then:
>> + properties:
>> + qcom,tx-message-size:
>> + description: Maximum size in bytes of a message which can be sent by this queue
>> + $ref: /schemas/types.yaml#/definitions/int32
>> + qcom,tx-queue-depth:
>> + description: Depth of transmit queue for messages sent by this queue
>> + $ref: /schemas/types.yaml#/definitions/int32
>> + - if:
>> + anyOf:
>> + - properties:
>> + qcom,is-receiver: true
>> + - properties:
>> + qcom,is-full-duplex: true
>> + then:
>> + properties:
>> + qcom,rx-message-size:
>> + description: Maximum size in bytes of a message which can be received by this queue
>> + $ref: /schemas/types.yaml#/definitions/int32
>> + qcom,rx-queue-depth:
>> + description: Depth of transmit queue for messages received by this queue
>> + $ref: /schemas/types.yaml#/definitions/int32
>> + - if:
>> + anyOf:
>> + - properties:
>> + qcom,is-receiver: true
>> + - properties:
>> + qcom,is-sender: true
>> + then:
>> + properties:
>> + reg:
>> + description: Hypervisor capability ID of the message queue
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + minItems: 1
>> + maxItems: 1
>> + interrupts:
>> + minItems: 1
>> + maxItems: 1
>> + - if:
>> + properties:
>> + qcom,is-full-duplex: true
>> + then:
>> + properties:
>> + reg:
>> + description:
>> + Hypervisor capability IDs of the message queue
>> + The first is tx side, the second is rx side
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + minItems: 2
>> + maxItems: 2
>> + interrupts:
>> + description: The first is tx interrupt, second is rx interrupt
>> + minItems: 2
>> + maxItems: 2
>> + required:
>> + - compatible
>> + - reg
>> + - interrupts
>> +
>> +
>> +examples:
>> + - |
>> + display-msgq-pair@abbf0da3c3c965cc {
>> + compatible = "qcom,gunyah-message-queue", "qcom,gunyah-capability";
>> + interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
>> + <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
>> + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; /* TX, RX cap ids */
>> + qcom,is-full-duplex;
>> + qcom,tx-queue-depth = <8>;
>> + qcom,tx-message-size = <0xf0>;
>> + qcom,rx-queue-depth = <8>;
>> + qcom,rx-message-size = <0xf0>;
>> + };
>> \ No newline at end of file
>
> Fix this.
>
>> diff --git a/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
>> new file mode 100644
>> index 000000000000..f637d51c52f0
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml
>> @@ -0,0 +1,122 @@
>> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/gunyah/qcom,hypervisor.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Hypervisor node to define virtual devices and other services provided by a Gunyah hypervisor
>> + to this virtual machine.
>> +
>> +maintainers:
>> + - Murali Nalajala <quic_mnalajal@quicinc.com>
>> + - Elliot Berman <quic_eberman@quicinc.com>
>> +
>> +description: |+
>> + On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which
>> + describes the basic configuration of the hypervisor. Virtual machines use this information for
>> + initial discovery that they are running as a Gunyah guest VM.
>> + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c
>> +
>> +properties:
>> + compatible:
>> + oneOf:
>> + - items:
>> + - const: qcom,gunyah-hypervisor-1.0
>> + - const: qcom,gunyah-hypervisor
>> +
>> + "#address-cells":
>> + description: Number of cells needed to represent 64-bit capability IDs.
>> + const: 2
>> + "#size-cells":
>> + description: must be 0, because capability IDs are not memory address
>> + ranges and do not have a size.
>> + const: 0
>> +
>> + qcom,gunyah-vm:
>> + type: object
>> + description:
>> + The VM Identification is a virtual node that conveys to the VM information
>> + about this virtual machine in the context of the hypervisor-based system
>> + properties:
>> + compatible:
>> + oneOf:
>> + - items:
>> + - const: qcom,gunyah-vm-id-1.0
>> + - const: qcom,gunyah-vm-id
>> + qcom,vendor:
>> + $ref: /schemas/types.yaml#/definitions/string
>> + description: Vendor of the Virtual Machine, e.g. Qualcomm
>
> Doesn't the compatible say this already?
>
We'll drop "qcom," vendor prefix.
>> + qcom,vmid:
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + description: contains the VMID of this VM as a 32-bit value
>> + qcom,owner-vmid:
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + description: Contains the hypervisor VMID of the VM's owner. The owner
>> + is the VM that allocated and created the VM. VMs directly
>> + managed by the resource manager, such as the primary VM do
>> + not have an owner.
>> + required:
>> + - compatible
>> + - qcom,vmid
>> + - qcom,owner-vmid
>> +
>> +patternProperties:
>> + "^qcom,resource-manager-rpc(@.*)?":
>
> We don't use vendor prefixes in node names. QCom really liked to though.
>
We'll drop "qcom," vendor prefix.
>> + type: object
>> + description:
>> + Resource Manager node which is required to communicate to Resource
>> + Manager VM using Gunyah Message Queues.
>> + allOf: "message-queue.yml#"
>
> Not valid json-schema...
>
Will double check this in next patch.
>> +
>> + properties:
>> + compatible:
>> + oneOf:
>> + items:
>> + - const: qcom,resource-manager-1-0
>> + - const: qcom,resource-manager
>> + qcom,console-dev:
>> + $ref: /schemas/types.yaml#/definitions/flag
>> + description: if set, the resource-manger will accept console logs from the VM
>> + qcom,free-irq-start:
>> + $ref: /schemas/types.yaml#/definitions/uint32
>> + description: Set on ARM systems which use a GIC. First VIRQ number which is free
>> + for virtual interrupt use.
>> + required:
>> + - qcom,is-full-duplex
>> +
>> +
>> +required:
>> +- compatible
>> +- "#address-cells"
>> +- "#size-cells"
>> +
>> +examples:
>> + - |
>> + hypervisor {
>> + #address-cells = <2>;
>> + #size-cells = <0>;
>> + compatible = "qcom,gunyah-hypervisor-1.0", "qcom,gunyah-hypervisor", "simple-bus";
>> + name = "hypervisor";
>> +
>> + qcom,gunyah-vm {
>> + compatible = "qcom,gunyah-vm-id-1.0", "qcom,gunyah-vm-id";
>> + qcom,vendor = "Qualcomm Technologies, Inc.";
>> + qcom,vmid = <45>;
>> + qcom,owner-vmid = <3>;
>> + };
>> +
>> + qcom,resource-manager-rpc@0000000000000001 {
>> + compatible = "qcom,resource-manager-1-0", "qcom,resource-manager",
>> + "qcom,gunyah-message-queue", "qcom,gunyah-capability";
>> + interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
>> + <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
>> + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>;
>> + /* TX, RX cap ids */
>> + qcom,is-full-duplex;
>> + qcom,free-irq-start = <0>;
>> + qcom,tx-queue-depth = <8>;
>> + qcom,tx-message-size = <0xf0>;
>> + qcom,rx-queue-depth = <8>;
>> + qcom,rx-message-size = <0xf0>;
>> + };
>> + };
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index bed175adc4c3..6a918f653eac 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -8400,6 +8400,7 @@ M: Elliot Berman <quic_eberman@quicinc.com>
>> M: Murali Nalajala <quic_mnalajal@quicinc.com>
>> L: linux-arm-msm@vger.kernel.org
>> S: Maintained
>> +F: Documentation/devicetree/bindings/gunyah/
>> F: Documentation/virt/gunyah/
>>
>> H8/300 ARCHITECTURE
>> --
>> 2.25.1
>>
>>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI
2022-02-24 10:26 ` Marc Zyngier
@ 2022-04-13 17:09 ` Elliot Berman
0 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-04-13 17:09 UTC (permalink / raw)
To: Marc Zyngier, Mark Rutland
Cc: Bjorn Andersson, linux-kernel, Trilok Soni, Carl van Schaik,
Andy Gross, linux-arm-msm, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Murali Nalajala, Srivatsa Vaddagiri
On 2/24/2022 2:26 AM, Marc Zyngier wrote:
> Thanks Mark for roping me in.
>
> On Thu, 24 Feb 2022 10:10:02 +0000,
> Mark Rutland <mark.rutland@arm.com> wrote:
>>
>> Hi,
>>
>> As a general thing, this is the *only* patch from this series which has
>> been Cc'd to linux-arm-kernel, which makes it practically impossible to
>> understand the context for this, which is somewhat frustrating.
>>
>> Looking on lore.kernel.org I see that the entire series was Cc'd to
>> linux-arm-msm, but most people don't subscribe to that list. If you send
>> one patch in a series to a list, please send the *entire* series there. >>>> On Wed, Feb 23, 2022 at 03:37:21PM -0800, Elliot Berman wrote:
>>> Add initial support to perform Gunyah hypercalls. The arm64 ABI for
>>> Gunyah hypercalls generally follows the AAPCS64, and can be summarized:
>>> - Function identifier is passed through the imm operand
>>> - [r0,r7] are parameter and result registers
>>> - [r8-r18] are temporary and saved by the caller (VM)
>>> - [r19-r31] are preserved and saved by the hypervisor
>>>
>>> The preprocessor macors for creating the necessary HVC instruction
>>> roughly follows the SMCCC 1.1 implementation in
>>> include/linux/arm-smccc.h.
>>
>> I've added the SMCCC maintainers (myself, Lorenzo, and SUdeep) to Cc,
>> and also Marc who was involvedi n prior discussions in this area. Please
>> Cc us on any future patches adding HVC or SMCC interfaces (SMCCC or
>> otherwise).
>
> In general, please Cc all the interested parties with the whole
> series. Random patches randomly cc'd out of context are pretty useless
> and only lead to them being ignored.
>
I'll do this in the future, thanks for the recommendation here.
>>
>> We've previously said NO to any new hypercall mechanisms which do not
>> follow SMCCC. There is no reason to fragment this space further; please
>> use SMCCC (which your hypervisor must already implement in part if it
>> exposes PSCI to a guest).
>>
>> NAK to this non-SMCCC interface.
>
> Agreed. We pushed back on that for Hyper-V, and I don't see a reason
> for changing tack on that.
>
> The calling convention exists for a reason: portability. If this
> hypervisor is to be "independent of any high-level OS kernel" (as it
> is being advertised), then it must already implement SMCCC.
>
> What is the issue with properly supporting SMCCC for all interactions
> with the hypervisor and not reinventing a square wheel? >
We had discussion on this and will implement SMCCC calls for Gunyah's
hypercalls. I'll share updated patches soon once we have update the
hypervisor.
> Thanks,
>
> M.
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v2 00/11] Gunyah Hypervisor drivers
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
` (10 preceding siblings ...)
2022-02-23 23:37 ` [PATCH 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
` (10 more replies)
11 siblings, 11 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Gunyah is a Type-1 hypervisor independent of any
high-level OS kernel, and runs in a higher CPU privilege level. It does
not depend on any lower-privileged OS kernel/code for its core
functionality. This increases its security and can support a much smaller
trusted computing base than Type-2 hypervisors. This series adds the initial
support for Gunyah hypercalls, IPC via message queues, communication with the
Gunyah Resource Manager to enable Gunyah's paravirtualized console.
Gunyah is an open source hypervisor. The source repo is available at
https://github.com/quic/gunyah-hypervisor.
This series enables guest awareness of Gunyah and establishes the basic
architecture of Gunyah capabilities in the Linux kernel. As an end-to-end
use case, a TTY driver for the Gunyah-based console is added which can demonstrate
communication with the resource manager. In a future series, we intend to add
support for loading secondary VMs.
To self-test the console driver without other VMs, reads and writes to
/dev/ttyGH0 will be sent to /dev/ttyGH1 via the hypervisor. In a system with
multiple VMs, a secondary VM could use ttyGH0 as its boot console.
The primary VM would be able to access that secondary VM's console via a ttyGHx.
Changes in v2:
- DT bindings clean up
- Switch hypercalls to follow SMCCC
Elliot Berman (11):
docs: gunyah: Introduce Gunyah Hypervisor
dt-bindings: Add binding for gunyah hypervisor
arm64: gunyah: Add Gunyah hypercalls ABI
gunyah: Common types and error codes for Gunyah hypercalls
virt: gunyah: Add sysfs nodes
virt: gunyah: Add capabilities bus and devices
gunyah: msgq: Add Gunyah message queues
gunyah: rsc_mgr: Add resource manager RPC core
gunyah: rsc_mgr: Add auxiliary devices for console
gunyah: rsc_mgr: Add RPC for console services
gunyah: Add tty console driver for RM Console Serivces
.../ABI/testing/sysfs-hypervisor-gunyah | 37 +
.../bindings/firmware/gunyah-hypervisor.yaml | 84 +++
Documentation/virt/gunyah/index.rst | 99 +++
Documentation/virt/gunyah/message-queue.rst | 52 ++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 12 +
arch/arm64/include/asm/gunyah.h | 142 ++++
drivers/virt/Kconfig | 1 +
drivers/virt/Makefile | 1 +
drivers/virt/gunyah/Kconfig | 24 +
drivers/virt/gunyah/Makefile | 8 +
drivers/virt/gunyah/device.c | 108 +++
drivers/virt/gunyah/gunyah_private.h | 18 +
drivers/virt/gunyah/msgq.c | 223 ++++++
drivers/virt/gunyah/rsc_mgr.c | 682 ++++++++++++++++++
drivers/virt/gunyah/rsc_mgr.h | 56 ++
drivers/virt/gunyah/rsc_mgr_console.c | 405 +++++++++++
drivers/virt/gunyah/rsc_mgr_rpc.c | 151 ++++
drivers/virt/gunyah/sysfs.c | 176 +++++
include/linux/gunyah.h | 133 ++++
include/linux/gunyah_rsc_mgr.h | 45 ++
21 files changed, 2458 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah
create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
create mode 100644 Documentation/virt/gunyah/index.rst
create mode 100644 Documentation/virt/gunyah/message-queue.rst
create mode 100644 arch/arm64/include/asm/gunyah.h
create mode 100644 drivers/virt/gunyah/Kconfig
create mode 100644 drivers/virt/gunyah/Makefile
create mode 100644 drivers/virt/gunyah/device.c
create mode 100644 drivers/virt/gunyah/gunyah_private.h
create mode 100644 drivers/virt/gunyah/msgq.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.h
create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c
create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c
create mode 100644 drivers/virt/gunyah/sysfs.c
create mode 100644 include/linux/gunyah.h
create mode 100644 include/linux/gunyah_rsc_mgr.h
--
2.25.1
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
` (9 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel, linux-doc, Jonathan Corbet
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Will Deacon, Catalin Marinas, devicetree
Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It
does not depend on any lower-privileged OS/kernel code for its core
functionality. This increases its security and can support a smaller
trusted computing based when compared to Type-2 hypervisors.
Add documentation describing the Gunyah hypervisor and the main
components of the Gunyah hypervisor which are of interest to Linux
virtualization development.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
Documentation/virt/gunyah/index.rst | 92 +++++++++++++++++++++
Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 7 ++
4 files changed, 152 insertions(+)
create mode 100644 Documentation/virt/gunyah/index.rst
create mode 100644 Documentation/virt/gunyah/message-queue.rst
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
new file mode 100644
index 000000000000..e7bb2b14543e
--- /dev/null
+++ b/Documentation/virt/gunyah/index.rst
@@ -0,0 +1,92 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Gunyah Hypervisor
+=================
+
+.. toctree::
+ :maxdepth: 1
+
+ message-queue
+
+Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
+a higher CPU privilege level. It does not depend on any lower-privileged operating system
+for its core functionality. This increases its security and can support a much smaller
+trusted computing base than a Type-2 hypervisor.
+
+Gunyah is an open source hypervisor. The source repo is available at
+https://github.com/quic/gunyah-hypervisor.
+
+Gunyah provides these following features.
+
+- Scheduling:
+ A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing
+ of the CPUs.
+- Memory Management:
+ APIs handling memory, abstracted as objects, limiting direct use of physical
+ addresses. Memory ownership and usage tracking of all memory under its control.
+ Memory partitioning between VMs is a fundamental security feature.
+- Interrupt Virtualization:
+ Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled
+ in the hypervisor and routed to the assigned VM.
+- Inter-VM Communication:
+ There are several different mechanisms provided for communicating between VMs.
+- Virtual platform:
+ Architectural devices such as interrupt controllers and CPU timers are directly provided
+ by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI.
+- Device Virtualization:
+ Para-virtualization of devices is supported using inter-VM communication.
+
+Architectures supported
+=======================
+AArch64 with a GIC
+
+Resources and Capabilities
+==========================
+
+Some services or resources provided by the Gunyah hypervisor are described by capability IDs.
+For instance, inter-VM communication is performed with doorbells and message queues. The specific
+instance of a doorbell is described by a capability ID. These devices are described in Linux as a
+struct gunyah_device.
+
+High level management of these resources is performed by the resource manager VM. RM informs a
+guest VM about resources it can access through either the device tree or via guest-initiated RPC.
+
+Resource Manager
+================
+
+The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor.
+It provides policy enforcement aspects of the virtualization system. The resource manager can
+be treated as an extension of the Hypervisor but is separated to its own partition to ensure
+that the hypervisor layer itself remains small and secure and to maintain a separation of policy
+and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines.
+
+Communication with the resource manager from each guest VM happens with message-queue.rst. Details
+about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
+
+::
+ +-------+ +--------+ +--------+
+ | RM | | VM_A | | VM_B |
+ +-.-.-.-+ +---.----+ +---.----+
+ | | | |
+ +-.-.-----------.------------.----+
+ | | \==========/ | |
+ | \========================/ |
+ | Gunyah |
+ +---------------------------------+
+
+The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager.
+
+The resource manager provides the following features:
+
+- Generate device-tree overlay
+- VM creation and deletion
+- VM device-tree management
+- VM access control policy
+- Interrupt routing configuration
+
+When booting a virtual machine which uses a devicetree, resource manager overlays a
+/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM,
+how to communicate with resource manager, and basic description and capabilities of
+this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description
+of this node.
diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
new file mode 100644
index 000000000000..afd405f3a5e1
--- /dev/null
+++ b/Documentation/virt/gunyah/message-queue.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Message Queues
+==============
+Message queue is a simple low-capacity IPC channel between two VMs. It is
+intended for sending small control and configuration messages. Each message
+queue object is unidirectional, so a full-duplex IPC channel requires a pair of
+objects.
+
+Messages can be up to 240 bytes in length. Longer messages require a further
+protocol on top of the message queue messages themselves. For instance, communication
+with the resource manager adds a header field for sending longer messages via multiple
+message fragments.
+
+The diagram below shows how message queue works. A typical configuration involves
+2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message
+queue 2 allows VM_B to send messages to VM_A.
+
+1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall
+ with the message to inform the hypervisor to add the message to
+ message queue 1's queue.
+2. Gunyah raises the corresponding interrupt for VM_B when any of these happens:
+ a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case.
+ b. Explicility with gh_msgq_push command from VM_A.
+ c. Message queue has reached a threshold depth.
+3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer.
+
+For VM_B to send a message to VM_A, the process is identical, except that hypercalls
+reference message queue 2's capability ID.
+
+::
+
+ +---------------+ +-----------------+ +---------------+
+ | VM_A | |Gunyah hypervisor| | VM_B |
+ | | | | | |
+ | | | | | |
+ | | Tx | | | |
+ | |-------->| | Rx vIRQ | |
+ |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() |
+ | |<------- | | | |
+ | | | | | |
+ | Message Queue | | | | Message Queue |
+ | driver | | | | driver |
+ | | | | | |
+ | | | | | |
+ | | | | Tx | |
+ | | Rx vIRQ | |<--------| |
+ |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() |
+ | | | |-------->| |
+ | | | | | |
+ | | | | | |
+ +---------------+ +-----------------+ +---------------+
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index 492f0920b988..dd4e8ef284eb 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -14,6 +14,7 @@ Linux Virtualization Support
ne_overview
acrn/index
coco/sev-guest
+ gunyah/index
.. only:: html and subproject
diff --git a/MAINTAINERS b/MAINTAINERS
index f679152bdbad..0a8ea7c28b23 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8735,6 +8735,13 @@ L: linux-efi@vger.kernel.org
S: Maintained
F: block/partitions/efi.*
+GUNYAH HYPERVISOR DRIVER
+M: Elliot Berman <quic_eberman@quicinc.com>
+M: Murali Nalajala <quic_mnalajal@quicinc.com>
+L: linux-arm-msm@vger.kernel.org
+S: Maintained
+F: Documentation/virt/gunyah/
+
HABANALABS PCI DRIVER
M: Oded Gabbay <ogabbay@kernel.org>
S: Supported
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-07-14 21:29 ` [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
` (8 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel, Rob Herring, Krzysztof Kozlowski,
devicetree
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Jonathan Corbet, Will Deacon,
Catalin Marinas, linux-doc
When Linux is booted as a guest under the Gunyah hypervisor, Gunyah
applies a devicetree overlay describing the virtual platform
configuration of the guest VM, such as the message queue capability IDs
for communicating with the Resource Manager. Add the DT bindings that
Gunyah adheres for the hypervisor node and message queues.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
.../bindings/firmware/gunyah-hypervisor.yaml | 84 +++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 85 insertions(+)
create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
new file mode 100644
index 000000000000..e50d932e768c
--- /dev/null
+++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Gunyah Hypervisor
+
+maintainers:
+ - Murali Nalajala <quic_mnalajal@quicinc.com>
+ - Elliot Berman <quic_eberman@quicinc.com>
+
+description: |+
+ On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which
+ describes the basic configuration of the hypervisor. Virtual machines use this information for
+ initial discovery that they are running as a Gunyah guest VM.
+ See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: gunyah-hypervisor-1.0
+ - const: gunyah-hypervisor
+
+ "#address-cells":
+ description: Number of cells needed to represent 64-bit capability IDs.
+ const: 2
+ "#size-cells":
+ description: must be 0, because capability IDs are not memory address
+ ranges and do not have a size.
+ const: 0
+
+patternProperties:
+ "^gunyah-resource-mgr(@.*)?":
+ type: object
+ description:
+ Resource Manager node which is required to communicate to Resource
+ Manager VM using Gunyah Message Queues.
+
+ properties:
+ compatible:
+ oneOf:
+ - items:
+ - const: gunyah-resource-manager-1-0
+ - const: gunyah-resource-manager
+ reg:
+ items:
+ - description: Gunyah capability ID of the TX message queue
+ - description: Gunyah capability ID of the RX message queue
+ interrupts:
+ items:
+ - description: Interrupt for the TX message queue
+ - description: Interrupt for the RX message queue
+ additionalProperties: false
+ required:
+ - compatible
+ - reg
+ - interrupts
+
+additionalProperties: false
+
+required:
+ - compatible
+ - "#address-cells"
+ - "#size-cells"
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ hypervisor {
+ #address-cells = <2>;
+ #size-cells = <0>;
+ compatible = "gunyah-hypervisor-1.0", "gunyah-hypervisor";
+
+ gunyah-resource-mgr@1 {
+ compatible = "gunyah-resource-manager-1-0", "gunyah-resource-manager";
+ interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>, /* TX full IRQ */
+ <GIC_SPI 4 IRQ_TYPE_EDGE_RISING>; /* RX empty IRQ */
+ reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>;
+ /* TX, RX cap ids */
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 0a8ea7c28b23..b36bd47bcaaa 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8740,6 +8740,7 @@ M: Elliot Berman <quic_eberman@quicinc.com>
M: Murali Nalajala <quic_mnalajal@quicinc.com>
L: linux-arm-msm@vger.kernel.org
S: Maintained
+F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
F: Documentation/virt/gunyah/
HABANALABS PCI DRIVER
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-07-14 21:29 ` [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
2022-07-14 21:29 ` [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
` (7 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel, Lorenzo Pieralisi, Sudeep Holla,
Marc Zyngier
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Rob Herring,
Krzysztof Kozlowski, Jonathan Corbet, Will Deacon,
Catalin Marinas, devicetree, linux-doc
Add initial support to perform Gunyah hypercalls. The arm64 ABI for
Gunyah hypercalls generally follows the SMC Calling Convention.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 1 +
arch/arm64/include/asm/gunyah.h | 134 ++++++++++++++++++++++++++++++++
2 files changed, 135 insertions(+)
create mode 100644 arch/arm64/include/asm/gunyah.h
diff --git a/MAINTAINERS b/MAINTAINERS
index b36bd47bcaaa..1d098bdba5c9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8742,6 +8742,7 @@ L: linux-arm-msm@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
F: Documentation/virt/gunyah/
+F: arch/arm64/include/asm/gunyah.h
HABANALABS PCI DRIVER
M: Oded Gabbay <ogabbay@kernel.org>
diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h
new file mode 100644
index 000000000000..2dbef08d58d7
--- /dev/null
+++ b/arch/arm64/include/asm/gunyah.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef __ASM_GUNYAH_H
+#define __ASM_GUNYAH_H
+
+#include <linux/arm-smccc.h>
+#include <linux/types.h>
+
+#define GH_CALL_TYPE_PLATFORM_CALL 0
+#define GH_CALL_TYPE_HYPERCALL 2
+#define GH_CALL_TYPE_SERVICE 3
+#define GH_CALL_TYPE_SHIFT 14
+#define GH_CALL_FUNCTION_NUM_MASK 0x3fff
+
+#define GH_SERVICE(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \
+ ARM_SMCCC_OWNER_VENDOR_HYP, \
+ (GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \
+ | ((fn) & GH_CALL_FUNCTION_NUM_MASK))
+
+#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
+ ARM_SMCCC_OWNER_VENDOR_HYP, \
+ (GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \
+ | ((fn) & GH_CALL_FUNCTION_NUM_MASK))
+
+#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
+
+#define __gh_count_args(...) \
+ ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0)
+
+#define __gh_skip_0(...) __VA_ARGS__
+#define __gh_skip_1(a, ...) __VA_ARGS__
+#define __gh_skip_2(a, b, ...) __VA_ARGS__
+#define __gh_skip_3(a, b, c, ...) __VA_ARGS__
+#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__
+#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__
+#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__
+#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__
+#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__
+#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__)
+
+#define __gh_declare_arg_0(...)
+
+#define __gh_declare_arg_1(arg1, ...) \
+ .a1 = (arg1)
+
+#define __gh_declare_arg_2(arg1, arg2, ...) \
+ __gh_declare_arg_1(arg1), \
+ .a2 = (arg2)
+
+#define __gh_declare_arg_3(arg1, arg2, arg3, ...) \
+ __gh_declare_arg_2(arg1, arg2), \
+ .a3 = (arg3)
+
+#define __gh_declare_arg_4(arg1, arg2, arg3, arg4, ...) \
+ __gh_declare_arg_3(arg1, arg2, arg3), \
+ .a4 = (arg4)
+
+#define __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5, ...) \
+ __gh_declare_arg_4(arg1, arg2, arg3, arg4), \
+ .a5 = (arg5)
+
+#define __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6, ...) \
+ __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5), \
+ .a6 = (arg6)
+
+#define __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7, ...) \
+ __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6), \
+ .a7 = (arg7)
+
+#define __gh_declare_arg_8(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, ...) \
+ __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7), \
+ .a8 = (arg8)
+
+#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs
+#define __gh_declare_args(nargs) ___gh_declare_args(nargs)
+#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__)
+
+#define __gh_assign_res_0(...)
+
+#define __gh_assign_res_1(r1) \
+ r1 = res.a0
+
+#define __gh_assign_res_2(r1, r2) \
+ __gh_assign_res_1(r1); \
+ r2 = res.a1
+
+#define __gh_assign_res_3(r1, r2, r3) \
+ __gh_assign_res_2(r1, r2); \
+ r3 = res.a2
+
+#define __gh_assign_res_4(r1, r2, r3, r4) \
+ __gh_assign_res_3(r1, r2, r3); \
+ r4 = res.a3
+
+#define __gh_assign_res_5(r1, r2, r3, r4, r5) \
+ __gh_assign_res_4(r1, r2, r3, r4); \
+ r5 = res.a4
+
+#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \
+ __gh_assign_res_5(r1, r2, r3, r4, r5); \
+ r6 = res.a5
+
+#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \
+ __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \
+ r7 = res.a6
+
+#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \
+ __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \
+ r8 = res.a7
+
+#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs
+#define __gh_assign_res(nargs) ___gh_assign_res(nargs)
+#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__)
+
+/**
+ * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI
+ * @hcall_num: Hypercall function ID to invoke
+ * @nargs: Number of input arguments
+ * @...: First nargs are the input arguments. Remaining arguments are output variables.
+ */
+#define arch_gh_hypercall(hcall_num, nargs, ...) \
+ do { \
+ struct arm_smccc_1_2_regs res; \
+ struct arm_smccc_1_2_regs args = { \
+ .a0 = hcall_num, \
+ _gh_declare_args(nargs, __VA_ARGS__) \
+ }; \
+ arm_smccc_1_2_hvc(&args, &res); \
+ _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \
+ } while (0)
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (2 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
` (6 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Add architecture-independent standard error codes, types, and macros for
Gunyah hypercalls.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 1 +
include/linux/gunyah.h | 75 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 76 insertions(+)
create mode 100644 include/linux/gunyah.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 1d098bdba5c9..405a13e1f4d7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8743,6 +8743,7 @@ S: Maintained
F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah.h
+F: include/linux/gunyah.h
HABANALABS PCI DRIVER
M: Oded Gabbay <ogabbay@kernel.org>
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
new file mode 100644
index 000000000000..69931a0f5736
--- /dev/null
+++ b/include/linux/gunyah.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_H
+#define _GUNYAH_H
+
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <asm/gunyah.h>
+
+typedef u64 gh_capid_t;
+
+/* Common Gunyah macros */
+#define GH_CAPID_INVAL U64_MAX
+
+#define GH_ERROR_OK 0
+#define GH_ERROR_UNIMPLEMENTED -1
+#define GH_ERROR_RETRY -2
+
+#define GH_ERROR_ARG_INVAL 1
+#define GH_ERROR_ARG_SIZE 2
+#define GH_ERROR_ARG_ALIGN 3
+
+#define GH_ERROR_NOMEM 10
+
+#define GH_ERROR_ADDR_OVFL 20
+#define GH_ERROR_ADDR_UNFL 21
+#define GH_ERROR_ADDR_INVAL 22
+
+#define GH_ERROR_DENIED 30
+#define GH_ERROR_BUSY 31
+#define GH_ERROR_IDLE 32
+
+#define GH_ERROR_IRQ_BOUND 40
+#define GH_ERROR_IRQ_UNBOUND 41
+
+#define GH_ERROR_CSPACE_CAP_NULL 50
+#define GH_ERROR_CSPACE_CAP_REVOKED 51
+#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52
+#define GH_ERROR_CSPACE_INSUF_RIGHTS 53
+#define GH_ERROR_CSPACE_FULL 54
+
+#define GH_ERROR_MSGQUEUE_EMPTY 60
+#define GH_ERROR_MSGQUEUE_FULL 61
+
+static inline int gh_remap_error(int gh_error)
+{
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ return 0;
+ case GH_ERROR_NOMEM:
+ return -ENOMEM;
+ case GH_ERROR_DENIED:
+ case GH_ERROR_CSPACE_CAP_NULL:
+ case GH_ERROR_CSPACE_CAP_REVOKED:
+ case GH_ERROR_CSPACE_WRONG_OBJ_TYPE:
+ case GH_ERROR_CSPACE_INSUF_RIGHTS:
+ case GH_ERROR_CSPACE_FULL:
+ return -EACCES;
+ case GH_ERROR_BUSY:
+ case GH_ERROR_IDLE:
+ return -EBUSY;
+ case GH_ERROR_IRQ_BOUND:
+ case GH_ERROR_IRQ_UNBOUND:
+ case GH_ERROR_MSGQUEUE_FULL:
+ case GH_ERROR_MSGQUEUE_EMPTY:
+ return -EPERM;
+ default:
+ return -EINVAL;
+ }
+}
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 05/11] virt: gunyah: Add sysfs nodes
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (3 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 22:41 ` Randy Dunlap
2022-07-14 21:29 ` [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
` (5 subsequent siblings)
10 siblings, 1 reply; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Add /sys/hypervisor support when detecting that Linux is running in a
Gunyah environment. Export the version of Gunyah which is reported via
the hyp_identify hypercall.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
.../ABI/testing/sysfs-hypervisor-gunyah | 37 +++++
MAINTAINERS | 2 +
arch/arm64/include/asm/gunyah.h | 4 +
drivers/virt/Kconfig | 1 +
drivers/virt/Makefile | 1 +
drivers/virt/gunyah/Kconfig | 13 ++
drivers/virt/gunyah/Makefile | 4 +
drivers/virt/gunyah/sysfs.c | 139 ++++++++++++++++++
8 files changed, 201 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah
create mode 100644 drivers/virt/gunyah/Kconfig
create mode 100644 drivers/virt/gunyah/Makefile
create mode 100644 drivers/virt/gunyah/sysfs.c
diff --git a/Documentation/ABI/testing/sysfs-hypervisor-gunyah b/Documentation/ABI/testing/sysfs-hypervisor-gunyah
new file mode 100644
index 000000000000..ebbdd0aead7b
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-hypervisor-gunyah
@@ -0,0 +1,37 @@
+What: /sys/hypervisor/type
+Date: January 2022
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ Type of hypervisor:
+ "gunyah": Gunyah hypervisor
+
+What: /sys/hypervisor/features
+Date: January 2022
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ Space separated list of features supported by Linux and Gunyah:
+ "cspace": Gunyah devices
+ "doorbell": Sending/receiving virtual interrupts via Gunyah doorbells
+ "message-queue": Sending/receiving messages via Gunyah message queues
+ "vic": Interrupt lending
+ "vpm": Virtual platform management
+ "vcpu": Virtual CPU management
+ "memextent": Memory lending/management
+ "trace": Gunyah hypervisor tracing
+
+
+What: /sys/hypervisor/version/api
+Date: April 2020
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ The Gunyah API version.
+
+What: /sys/hypervisor/version/variant
+Date: April 2020
+KernelVersion: 5.17
+Contact: linux-arm-msm@vger.kernel.org
+Description: If running under Gunyah:
+ The Gunyah variant (build) version.
\ No newline at end of file
diff --git a/MAINTAINERS b/MAINTAINERS
index 405a13e1f4d7..5d46c1f91b92 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8740,9 +8740,11 @@ M: Elliot Berman <quic_eberman@quicinc.com>
M: Murali Nalajala <quic_mnalajal@quicinc.com>
L: linux-arm-msm@vger.kernel.org
S: Maintained
+F: Documentation/ABI/testing/sysfs-hypervisor-gunyah
F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah.h
+F: drivers/virt/gunyah/
F: include/linux/gunyah.h
HABANALABS PCI DRIVER
diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h
index 2dbef08d58d7..3eef2a9ea299 100644
--- a/arch/arm64/include/asm/gunyah.h
+++ b/arch/arm64/include/asm/gunyah.h
@@ -19,11 +19,15 @@
(GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \
| ((fn) & GH_CALL_FUNCTION_NUM_MASK))
+#define GH_HYPERCALL_CALL_UID GH_SERVICE(0x3f01)
+
#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
ARM_SMCCC_OWNER_VENDOR_HYP, \
(GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \
| ((fn) & GH_CALL_FUNCTION_NUM_MASK))
+#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000)
+
#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
#define __gh_count_args(...) \
diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig
index 87ef258cec64..259dc2be6cad 100644
--- a/drivers/virt/Kconfig
+++ b/drivers/virt/Kconfig
@@ -52,4 +52,5 @@ source "drivers/virt/coco/efi_secret/Kconfig"
source "drivers/virt/coco/sev-guest/Kconfig"
+source "drivers/virt/gunyah/Kconfig"
endif
diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile
index 093674e05c40..ec4fdfa8eef1 100644
--- a/drivers/virt/Makefile
+++ b/drivers/virt/Makefile
@@ -6,6 +6,7 @@
obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o
obj-$(CONFIG_VMGENID) += vmgenid.o
obj-y += vboxguest/
+obj-$(CONFIG_GUNYAH) += gunyah/
obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/
obj-$(CONFIG_ACRN_HSM) += acrn/
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
new file mode 100644
index 000000000000..e88289963518
--- /dev/null
+++ b/drivers/virt/gunyah/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config GUNYAH
+ tristate "Gunyah Virtualization drivers"
+ depends on ARM64
+ select SYS_HYPERVISOR
+ help
+ The Gunyah drivers are the helper interfaces that runs in a guest VM
+ such as basic inter-VM IPC and signaling mechanism,s and higher level
+ services such as memory/device sharing, IRQ sharing, and so on.
+
+ Say Y here to enable the drivers needed to interact in a Gunyah
+ virtual environment.
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
new file mode 100644
index 000000000000..0aa086f9149f
--- /dev/null
+++ b/drivers/virt/gunyah/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+gunyah-y += sysfs.o
+obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
new file mode 100644
index 000000000000..253433a939cf
--- /dev/null
+++ b/drivers/virt/gunyah/sysfs.c
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gunyah: " fmt
+
+#include <linux/kobject.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+#include <linux/of.h>
+
+#define QC_HYP_UID0 0x19bd54bd
+#define QC_HYP_UID1 0x0b37571b
+#define QC_HYP_UID2 0x946f609b
+#define QC_HYP_UID3 0x54539de6
+
+#define GUNYAH_UID0 0x673d5f14
+#define GUNYAH_UID1 0x9265ce36
+#define GUNYAH_UID2 0xa4535fdb
+#define GUNYAH_UID3 0xc1d58fcd
+
+#define gh_uid_matches(prefix, uid) \
+ ((uid)[0] == prefix ## _UID0 && (uid)[1] == prefix ## _UID1 && \
+ (uid)[2] == prefix ## _UID2 && (uid)[3] == prefix ## _UID3)
+
+#define GH_API_INFO_API_VERSION(x) (((x) >> 0) & 0x3fff)
+#define GH_API_INFO_BIG_ENDIAN(x) (((x) >> 14) & 1)
+#define GH_API_INFO_IS_64BIT(x) (((x) >> 15) & 1)
+#define GH_API_INFO_VARIANT(x) (((x) >> 56) & 0xff)
+
+#define GH_IDENTIFY_PARTITION_CSPACE(flags) (((flags)[0] >> 0) & 1)
+#define GH_IDENTIFY_DOORBELL(flags) (((flags)[0] >> 1) & 1)
+#define GH_IDENTIFY_MSGQUEUE(flags) (((flags)[0] >> 2) & 1)
+#define GH_IDENTIFY_VIC(flags) (((flags)[0] >> 3) & 1)
+#define GH_IDENTIFY_VPM(flags) (((flags)[0] >> 4) & 1)
+#define GH_IDENTIFY_VCPU(flags) (((flags)[0] >> 5) & 1)
+#define GH_IDENTIFY_MEMEXTENT(flags) (((flags)[0] >> 6) & 1)
+#define GH_IDENTIFY_TRACE_CTRL(flags) (((flags)[0] >> 7) & 1)
+
+struct gh_hypercall_hyp_identify_resp {
+ u64 api_info;
+ u64 flags[3];
+};
+
+static struct gh_hypercall_hyp_identify_resp gunyah_api;
+
+static ssize_t type_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "gunyah\n");
+}
+static struct kobj_attribute type_attr = __ATTR_RO(type);
+
+static ssize_t api_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_API_VERSION(gunyah_api.api_info));
+}
+static struct kobj_attribute api_attr = __ATTR_RO(api);
+
+static ssize_t variant_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_VARIANT(gunyah_api.api_info));
+}
+static struct kobj_attribute variant_attr = __ATTR_RO(variant);
+
+static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
+{
+ return sysfs_emit(buffer, "\n");
+}
+static struct kobj_attribute features_attr = __ATTR_RO(features);
+
+static struct attribute *version_attrs[] = {
+ &api_attr.attr,
+ &variant_attr.attr,
+ NULL
+};
+
+static const struct attribute_group version_group = {
+ .name = "version",
+ .attrs = version_attrs,
+};
+
+static int __init gh_sysfs_register(void)
+{
+ int ret;
+
+ ret = sysfs_create_file(hypervisor_kobj, &type_attr.attr);
+ if (ret)
+ return ret;
+
+ ret = sysfs_create_group(hypervisor_kobj, &version_group);
+ if (ret)
+ return ret;
+
+ return sysfs_create_file(hypervisor_kobj, &features_attr.attr);
+}
+
+static void gh_sysfs_unregister(void)
+{
+ sysfs_remove_file(hypervisor_kobj, &type_attr.attr);
+ sysfs_remove_group(hypervisor_kobj, &version_group);
+}
+
+static int __init gunyah_init(void)
+{
+ unsigned long uid[4];
+
+ arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]);
+
+ if (!(gh_uid_matches(GUNYAH, uid) || gh_uid_matches(QC_HYP, uid)))
+ return 0;
+
+ arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info,
+ gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]);
+
+ if (GH_API_INFO_API_VERSION(gunyah_api.api_info) != 1) {
+ pr_warn("Unrecognized gunyah version: %llu. Currently supported: 1\n",
+ GH_API_INFO_API_VERSION(gunyah_api.api_info));
+ return 0;
+ }
+
+ pr_notice("Running under Gunyah hypervisor v%lld/%llx\n",
+ GH_API_INFO_API_VERSION(gunyah_api.api_info),
+ GH_API_INFO_VARIANT(gunyah_api.api_info));
+
+ return gh_sysfs_register();
+}
+module_init(gunyah_init);
+
+static void __exit gunyah_exit(void)
+{
+ gh_sysfs_unregister();
+}
+module_exit(gunyah_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Gunyah Hypervisor Driver");
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (4 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
` (4 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Some resources provided by the Gunyah hypervisor are described as
objects. The objects are identified with a capability ID. For instance,
Inter-VM communication is performed with doorbells and message queues.
Each doorbell and message queue endpoint can be described consisely as a
Linux device.
These resources are discovered either on the devicetree or reported by
the Resource Manager. Devices on the Gunyah bus are matched with drivers
according to the type ID reported by resource manager. Most resources
will be discovered directly from the resource manager, so matching
directly on type ID seems like sensible design.
Each resource may also optionally have an interrupt associated with it
and a known partner VM (e.g. which VM is the receiver of a message
queue).
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/Makefile | 2 +-
drivers/virt/gunyah/device.c | 108 +++++++++++++++++++++++++++
drivers/virt/gunyah/gunyah_private.h | 12 +++
drivers/virt/gunyah/sysfs.c | 25 ++++++-
include/linux/gunyah.h | 45 +++++++++++
5 files changed, 189 insertions(+), 3 deletions(-)
create mode 100644 drivers/virt/gunyah/device.c
create mode 100644 drivers/virt/gunyah/gunyah_private.h
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 0aa086f9149f..3869fb7371df 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-gunyah-y += sysfs.o
+gunyah-y += sysfs.o device.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/device.c b/drivers/virt/gunyah/device.c
new file mode 100644
index 000000000000..93595f9a65b9
--- /dev/null
+++ b/drivers/virt/gunyah/device.c
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "ghdev: " fmt
+
+#include <linux/interrupt.h>
+#include <linux/gunyah.h>
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+
+#include "gunyah_private.h"
+
+static int gunyah_match(struct device *dev, struct device_driver *drv)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(drv);
+
+ return ghdev->type == ghdrv->type;
+}
+
+static int gunyah_probe(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver);
+
+ return ghdrv->probe ? ghdrv->probe(ghdev) : 0;
+}
+
+static void gunyah_remove(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+ struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver);
+
+ if (ghdrv->remove)
+ ghdrv->remove(ghdev);
+}
+
+static struct bus_type gunyah_bus = {
+ .name = "gunyah",
+ .match = gunyah_match,
+ .probe = gunyah_probe,
+ .remove = gunyah_remove,
+};
+
+int gunyah_register_driver(struct gunyah_driver *ghdrv)
+{
+ ghdrv->driver.bus = &gunyah_bus;
+ return driver_register(&ghdrv->driver);
+}
+
+void gunyah_unregister_driver(struct gunyah_driver *ghdrv)
+{
+ driver_unregister(&ghdrv->driver);
+}
+
+static void gunyah_device_release(struct device *dev)
+{
+ struct gunyah_device *ghdev = to_gunyah_device(dev);
+
+ kfree(ghdev);
+}
+
+struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type)
+{
+ struct gunyah_device *ghdev;
+
+ ghdev = kzalloc(sizeof(*ghdev), GFP_KERNEL);
+ if (!ghdev)
+ return NULL;
+
+ ghdev->capid = capid;
+ ghdev->type = type;
+ ghdev->irq = IRQ_NOTCONNECTED;
+ ghdev->dev.parent = parent;
+ ghdev->dev.release = gunyah_device_release;
+ ghdev->dev.bus = &gunyah_bus;
+ device_initialize(&ghdev->dev);
+ return ghdev;
+}
+
+int gunyah_device_add(struct gunyah_device *ghdev)
+{
+ int ret;
+
+ ret = dev_set_name(&ghdev->dev, "%u.%08llx", ghdev->type, ghdev->capid);
+ if (ret)
+ return ret;
+
+ return device_add(&ghdev->dev);
+}
+
+void gunyah_device_remove(struct gunyah_device *ghdev)
+{
+ device_unregister(&ghdev->dev);
+}
+
+int __init gunyah_bus_init(void)
+{
+ return bus_register(&gunyah_bus);
+}
+
+void gunyah_bus_exit(void)
+{
+ bus_unregister(&gunyah_bus);
+}
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
new file mode 100644
index 000000000000..5f3832608020
--- /dev/null
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_PRIVATE_H
+#define _GUNYAH_PRIVATE_H
+
+int __init gunyah_bus_init(void);
+void gunyah_bus_exit(void);
+
+#endif
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index 253433a939cf..220560cb3b1c 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -12,6 +12,8 @@
#include <linux/init.h>
#include <linux/of.h>
+#include "gunyah_private.h"
+
#define QC_HYP_UID0 0x19bd54bd
#define QC_HYP_UID1 0x0b37571b
#define QC_HYP_UID2 0x946f609b
@@ -67,7 +69,13 @@ static struct kobj_attribute variant_attr = __ATTR_RO(variant);
static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer)
{
- return sysfs_emit(buffer, "\n");
+ int len = 0;
+
+ if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags))
+ len += sysfs_emit_at(buffer, len, "cspace ");
+
+ len += sysfs_emit_at(buffer, len, "\n");
+ return len;
}
static struct kobj_attribute features_attr = __ATTR_RO(features);
@@ -105,6 +113,7 @@ static void gh_sysfs_unregister(void)
static int __init gunyah_init(void)
{
+ int ret;
unsigned long uid[4];
arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]);
@@ -125,12 +134,24 @@ static int __init gunyah_init(void)
GH_API_INFO_API_VERSION(gunyah_api.api_info),
GH_API_INFO_VARIANT(gunyah_api.api_info));
- return gh_sysfs_register();
+ ret = gh_sysfs_register();
+ if (ret)
+ return ret;
+
+ ret = gunyah_bus_init();
+ if (ret)
+ goto err_sysfs;
+
+ return ret;
+err_sysfs:
+ gh_sysfs_unregister();
+ return ret;
}
module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gunyah_bus_exit();
gh_sysfs_unregister();
}
module_exit(gunyah_exit);
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
index 69931a0f5736..ce35f4491773 100644
--- a/include/linux/gunyah.h
+++ b/include/linux/gunyah.h
@@ -6,6 +6,7 @@
#ifndef _GUNYAH_H
#define _GUNYAH_H
+#include <linux/device.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <asm/gunyah.h>
@@ -72,4 +73,48 @@ static inline int gh_remap_error(int gh_error)
}
}
+/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */
+#define GUNYAH_DEVICE_TYPE_BELL_TX 0
+#define GUNYAH_DEVICE_TYPE_BELL_RX 1
+#define GUNYAH_DEVICE_TYPE_MSGQ_TX 2
+#define GUNYAH_DEVICE_TYPE_MSGQ_RX 3
+#define GUNYAH_DEVICE_TYPE_VCPU 4
+
+struct gunyah_device {
+ u8 type;
+ gh_capid_t capid;
+ int irq;
+
+ struct device dev;
+};
+
+#define to_gunyah_device(dev) container_of(dev, struct gunyah_device, dev)
+
+static inline void *ghdev_get_drvdata(const struct gunyah_device *ghdev)
+{
+ return dev_get_drvdata(&ghdev->dev);
+}
+
+static inline void ghdev_set_drvdata(struct gunyah_device *ghdev, void *data)
+{
+ dev_set_drvdata(&ghdev->dev, data);
+}
+
+struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type);
+
+int gunyah_device_add(struct gunyah_device *ghdev);
+void gunyah_device_remove(struct gunyah_device *ghdev);
+
+struct gunyah_driver {
+ struct device_driver driver;
+ u8 type;
+ int (*probe)(struct gunyah_device *ghdev);
+ int (*remove)(struct gunyah_device *ghdev);
+};
+
+#define to_gunyah_driver(drv) container_of(drv, struct gunyah_driver, driver)
+
+int gunyah_register_driver(struct gunyah_driver *ghdrv);
+void gunyah_unregister_driver(struct gunyah_driver *ghdrv);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (5 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
` (3 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Gunyah message queues are unidirectional pipelines to communicate
between 2 virtual machines, but are typically paired to allow
bidirectional communication. The intended use case is for small control
messages between 2 VMs, as they support a maximum of 240 bytes.
Message queues can be discovered either by resource manager or on the
devicetree. To support discovery on the devicetree, client drivers can
use gh_msgq_platform_host_attach to allocate the tx and rx message
queues according to
Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
arch/arm64/include/asm/gunyah.h | 4 +
drivers/virt/gunyah/Makefile | 2 +-
drivers/virt/gunyah/gunyah_private.h | 3 +
drivers/virt/gunyah/msgq.c | 223 +++++++++++++++++++++++++++
drivers/virt/gunyah/sysfs.c | 9 ++
include/linux/gunyah.h | 13 ++
6 files changed, 253 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/gunyah/msgq.c
diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h
index 3eef2a9ea299..76c26bd78338 100644
--- a/arch/arm64/include/asm/gunyah.h
+++ b/arch/arm64/include/asm/gunyah.h
@@ -27,6 +27,10 @@
| ((fn) & GH_CALL_FUNCTION_NUM_MASK))
#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000)
+#define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x001B)
+#define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x001C)
+
+#define GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH BIT(0)
#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 3869fb7371df..94dc8e738911 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
-gunyah-y += sysfs.o device.o
+gunyah-y += sysfs.o device.o msgq.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
index 5f3832608020..2ade32bd9bdf 100644
--- a/drivers/virt/gunyah/gunyah_private.h
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -9,4 +9,7 @@
int __init gunyah_bus_init(void);
void gunyah_bus_exit(void);
+int __init gh_msgq_init(void);
+void gh_msgq_exit(void);
+
#endif
diff --git a/drivers/virt/gunyah/msgq.c b/drivers/virt/gunyah/msgq.c
new file mode 100644
index 000000000000..afc2572d3e7d
--- /dev/null
+++ b/drivers/virt/gunyah/msgq.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+
+#include "gunyah_private.h"
+
+struct gh_msgq {
+ bool ready;
+ wait_queue_head_t wq;
+ spinlock_t lock;
+};
+
+static irqreturn_t gh_msgq_irq_handler(int irq, void *dev)
+{
+ struct gh_msgq *msgq = dev;
+
+ spin_lock(&msgq->lock);
+ msgq->ready = true;
+ spin_unlock(&msgq->lock);
+ wake_up_interruptible_all(&msgq->wq);
+
+ return IRQ_HANDLED;
+}
+
+static int __gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, u64 tx_flags)
+{
+ unsigned long flags, gh_error;
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+ bool ready;
+
+ spin_lock_irqsave(&msgq->lock, flags);
+ arch_gh_hypercall(GH_HYPERCALL_MSGQ_SEND, 5,
+ ghdev->capid, size, (uintptr_t)buff, tx_flags, 0,
+ gh_error, ready);
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ ret = 0;
+ msgq->ready = ready;
+ break;
+ case GH_ERROR_MSGQUEUE_FULL:
+ ret = -EAGAIN;
+ msgq->ready = false;
+ break;
+ default:
+ ret = gh_remap_error(gh_error);
+ break;
+ }
+ spin_unlock_irqrestore(&msgq->lock, flags);
+
+ return ret;
+}
+
+/**
+ * gh_msgq_send() - Send a message to the client running on a different VM
+ * @client: The client descriptor that was obtained via gh_msgq_register()
+ * @buff: Pointer to the buffer where the received data must be placed
+ * @buff_size: The size of the buffer space available
+ * @flags: Optional flags to pass to receive the data. For the list of flags,
+ * see linux/gunyah/gh_msgq.h
+ *
+ * Returns: The number of bytes copied to buff. <0 if there was an error.
+ *
+ * Note: this function may sleep and should not be called from interrupt context
+ */
+ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size,
+ const unsigned long flags)
+{
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+ u64 tx_flags = 0;
+
+ if (flags & GH_MSGQ_TX_PUSH)
+ tx_flags |= GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH;
+
+ do {
+ ret = __gh_msgq_send(ghdev, buff, size, tx_flags);
+
+ if (ret == -EAGAIN) {
+ if (flags & GH_MSGQ_NONBLOCK)
+ goto out;
+ if (wait_event_interruptible(msgq->wq, msgq->ready))
+ ret = -ERESTARTSYS;
+ }
+ } while (ret == -EAGAIN);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_msgq_send);
+
+static ssize_t __gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size)
+{
+ unsigned long flags, gh_error;
+ size_t recv_size;
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+ bool ready;
+
+ spin_lock_irqsave(&msgq->lock, flags);
+
+ arch_gh_hypercall(GH_HYPERCALL_MSGQ_RECV, 4,
+ ghdev->capid, (uintptr_t)buff, size, 0,
+ gh_error, recv_size, ready);
+ switch (gh_error) {
+ case GH_ERROR_OK:
+ ret = recv_size;
+ msgq->ready = ready;
+ break;
+ case GH_ERROR_MSGQUEUE_EMPTY:
+ ret = -EAGAIN;
+ msgq->ready = false;
+ break;
+ default:
+ ret = gh_remap_error(gh_error);
+ break;
+ }
+ spin_unlock_irqrestore(&msgq->lock, flags);
+
+ return ret;
+}
+
+/**
+ * gh_msgq_recv() - Receive a message from the client running on a different VM
+ * @client: The client descriptor that was obtained via gh_msgq_register()
+ * @buff: Pointer to the buffer where the received data must be placed
+ * @buff_size: The size of the buffer space available
+ * @flags: Optional flags to pass to receive the data. For the list of flags,
+ * see linux/gunyah/gh_msgq.h
+ *
+ * Returns: The number of bytes copied to buff. <0 if there was an error.
+ *
+ * Note: this function may sleep and should not be called from interrupt context
+ */
+ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size,
+ const unsigned long flags)
+{
+ struct gh_msgq *msgq = ghdev_get_drvdata(ghdev);
+ ssize_t ret;
+
+ do {
+ ret = __gh_msgq_recv(ghdev, buff, size);
+
+ if (ret == -EAGAIN) {
+ if (flags & GH_MSGQ_NONBLOCK)
+ goto out;
+ if (wait_event_interruptible(msgq->wq, msgq->ready))
+ ret = -ERESTARTSYS;
+ }
+ } while (ret == -EAGAIN);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_msgq_recv);
+
+static int gh_msgq_probe(struct gunyah_device *ghdev)
+{
+ struct gh_msgq *msgq;
+
+ msgq = devm_kzalloc(&ghdev->dev, sizeof(*msgq), GFP_KERNEL);
+ if (!msgq)
+ return -ENOMEM;
+ ghdev_set_drvdata(ghdev, msgq);
+
+ msgq->ready = true; /* Assume we can use the message queue right away */
+ init_waitqueue_head(&msgq->wq);
+ spin_lock_init(&msgq->lock);
+
+ return devm_request_irq(&ghdev->dev, ghdev->irq, gh_msgq_irq_handler, 0,
+ dev_name(&ghdev->dev), msgq);
+}
+
+static struct gunyah_driver gh_msgq_tx_driver = {
+ .driver = {
+ .name = "gh_msgq_tx",
+ .owner = THIS_MODULE,
+ },
+ .type = GUNYAH_DEVICE_TYPE_MSGQ_TX,
+ .probe = gh_msgq_probe,
+};
+
+static struct gunyah_driver gh_msgq_rx_driver = {
+ .driver = {
+ .name = "gh_msgq_rx",
+ .owner = THIS_MODULE,
+ },
+ .type = GUNYAH_DEVICE_TYPE_MSGQ_RX,
+ .probe = gh_msgq_probe,
+};
+
+int __init gh_msgq_init(void)
+{
+ int ret;
+
+ ret = gunyah_register_driver(&gh_msgq_tx_driver);
+ if (ret)
+ return ret;
+
+ ret = gunyah_register_driver(&gh_msgq_rx_driver);
+ if (ret)
+ goto err_rx;
+
+ return ret;
+err_rx:
+ gunyah_unregister_driver(&gh_msgq_tx_driver);
+ return ret;
+}
+
+void gh_msgq_exit(void)
+{
+ gunyah_unregister_driver(&gh_msgq_rx_driver);
+ gunyah_unregister_driver(&gh_msgq_tx_driver);
+}
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index 220560cb3b1c..7589689e5e92 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -73,6 +73,8 @@ static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr,
if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags))
len += sysfs_emit_at(buffer, len, "cspace ");
+ if (GH_IDENTIFY_MSGQUEUE(gunyah_api.flags))
+ len += sysfs_emit_at(buffer, len, "message-queue ");
len += sysfs_emit_at(buffer, len, "\n");
return len;
@@ -142,7 +144,13 @@ static int __init gunyah_init(void)
if (ret)
goto err_sysfs;
+ ret = gh_msgq_init();
+ if (ret)
+ goto err_bus;
+
return ret;
+err_bus:
+ gunyah_bus_exit();
err_sysfs:
gh_sysfs_unregister();
return ret;
@@ -151,6 +159,7 @@ module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gh_msgq_exit();
gunyah_bus_exit();
gh_sysfs_unregister();
}
diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h
index ce35f4491773..099224f9d6d1 100644
--- a/include/linux/gunyah.h
+++ b/include/linux/gunyah.h
@@ -6,6 +6,7 @@
#ifndef _GUNYAH_H
#define _GUNYAH_H
+#include <linux/platform_device.h>
#include <linux/device.h>
#include <linux/types.h>
#include <linux/errno.h>
@@ -117,4 +118,16 @@ struct gunyah_driver {
int gunyah_register_driver(struct gunyah_driver *ghdrv);
void gunyah_unregister_driver(struct gunyah_driver *ghdrv);
+#define GH_MSGQ_MAX_MSG_SIZE 1024
+
+/* Possible flags to pass for Tx or Rx */
+#define GH_MSGQ_TX_PUSH BIT(0)
+#define GH_MSGQ_NONBLOCK BIT(32)
+
+ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size,
+ const unsigned long flags);
+ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size,
+ const unsigned long flags);
+
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (6 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
` (2 subsequent siblings)
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
The resource manager is a special virtual machine which is always
running on a Gunyah system. It provides APIs for creating and destroying
VMs, secure memory management, sharing/lending of memory between VMs,
and setup of inter-VM communication. Calls to the resource manager are
made via message queues.
This patch implements the basic probing and RPC mechanism to make those
API calls. Request/response calls can be made with gh_rm_call.
Drivers can also register to notifications pushed by RM via
gh_rm_register_notifier
Specific API calls that resource manager supports will be implemented in
subsequent patches.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
MAINTAINERS | 2 +-
drivers/virt/gunyah/Kconfig | 1 +
drivers/virt/gunyah/Makefile | 1 +
drivers/virt/gunyah/gunyah_private.h | 3 +
drivers/virt/gunyah/rsc_mgr.c | 623 +++++++++++++++++++++++++++
drivers/virt/gunyah/rsc_mgr.h | 34 ++
drivers/virt/gunyah/sysfs.c | 7 +
include/linux/gunyah_rsc_mgr.h | 29 ++
8 files changed, 699 insertions(+), 1 deletion(-)
create mode 100644 drivers/virt/gunyah/rsc_mgr.c
create mode 100644 drivers/virt/gunyah/rsc_mgr.h
create mode 100644 include/linux/gunyah_rsc_mgr.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 5d46c1f91b92..4bbec2ae4a6c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8745,7 +8745,7 @@ F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
F: Documentation/virt/gunyah/
F: arch/arm64/include/asm/gunyah.h
F: drivers/virt/gunyah/
-F: include/linux/gunyah.h
+F: include/linux/gunyah*.h
HABANALABS PCI DRIVER
M: Oded Gabbay <ogabbay@kernel.org>
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
index e88289963518..2ef4887e280d 100644
--- a/drivers/virt/gunyah/Kconfig
+++ b/drivers/virt/gunyah/Kconfig
@@ -4,6 +4,7 @@ config GUNYAH
tristate "Gunyah Virtualization drivers"
depends on ARM64
select SYS_HYPERVISOR
+ select AUXILIARY_BUS
help
The Gunyah drivers are the helper interfaces that runs in a guest VM
such as basic inter-VM IPC and signaling mechanism,s and higher level
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 94dc8e738911..86655bca8944 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
gunyah-y += sysfs.o device.o msgq.o
+gunyah-y += rsc_mgr.o
obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h
index 2ade32bd9bdf..6483ffa8c15d 100644
--- a/drivers/virt/gunyah/gunyah_private.h
+++ b/drivers/virt/gunyah/gunyah_private.h
@@ -12,4 +12,7 @@ void gunyah_bus_exit(void);
int __init gh_msgq_init(void);
void gh_msgq_exit(void);
+int __init gh_rsc_mgr_init(void);
+void gh_rsc_mgr_exit(void);
+
#endif
diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c
new file mode 100644
index 000000000000..b8268ee02fab
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr.c
@@ -0,0 +1,623 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr: " fmt
+
+#include <linux/of.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/gunyah.h>
+#include <linux/module.h>
+#include <linux/of_irq.h>
+#include <linux/kthread.h>
+#include <linux/notifier.h>
+#include <linux/irqdomain.h>
+#include <linux/workqueue.h>
+#include <linux/completion.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/gunyah_rsc_mgr.h>
+#include <linux/mod_devicetable.h>
+#include <linux/platform_device.h>
+
+#include "gunyah_private.h"
+#include "rsc_mgr.h"
+
+/* Resource Manager Header */
+struct gh_rm_rpc_hdr {
+ u8 version:4,
+ hdr_words:4;
+ u8 type:2,
+ fragments:6;
+ u16 seq;
+ u32 msg_id;
+} __packed;
+
+/* Standard reply header */
+struct gh_rm_rpc_reply_hdr {
+ struct gh_rm_rpc_hdr rpc_hdr;
+ u32 err_code;
+} __packed;
+
+/* RPC Header versions */
+#define GH_RM_RPC_HDR_VERSION_ONE 0x1
+
+/* RPC Header words */
+#define GH_RM_RPC_HDR_WORDS 0x2
+
+/* RPC Message types */
+#define GH_RM_RPC_TYPE_CONT 0x0
+#define GH_RM_RPC_TYPE_REQ 0x1
+#define GH_RM_RPC_TYPE_RPLY 0x2
+#define GH_RM_RPC_TYPE_NOTIF 0x3
+
+#define GH_RM_MAX_NUM_FRAGMENTS 62
+
+#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr))
+
+/**
+ * struct gh_rm_connection - Represents a complete message from resource manager
+ * @payload: Combined payload of all the fragments (i.e. msg headers stripped off).
+ * @size: Size of the payload.
+ * @ret: Linux return code, set in case there was an error processing the connection.
+ * @msg_id: Message ID from the header.
+ * @type: GH_RM_RPC_TYPE_RPLY or GH_RM_RPC_TYPE_NOTIF.
+ * @num_fragments: total number of fragments expected to be received for this connection.
+ * @fragments_recieved: fragments received so far.
+ * @rm_error: For request/reply sequences with standard replies.
+ * @seq: Sequence ID for the main message.
+ */
+struct gh_rm_connection {
+ void *payload;
+ size_t size;
+ int ret;
+ u32 msg_id;
+ u8 type;
+
+ u8 num_fragments;
+ u8 fragments_received;
+
+ /* only for req/reply sequence */
+ u32 rm_error;
+ u16 seq;
+ struct completion seq_done;
+};
+
+struct gh_rm_notif_complete {
+ struct gh_rm_connection *conn;
+ struct work_struct work;
+};
+
+struct gh_rsc_mgr {
+ struct task_struct *recv_task;
+ struct gunyah_device *msgq_tx, *msgq_rx;
+
+ struct idr call_idr;
+ struct mutex call_idr_lock;
+
+ struct mutex send_lock;
+};
+
+static struct gh_rsc_mgr *__rsc_mgr;
+SRCU_NOTIFIER_HEAD_STATIC(gh_rm_notifier);
+
+static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type)
+{
+ struct gh_rm_connection *connection;
+
+ connection = kzalloc(sizeof(*connection), GFP_KERNEL);
+ if (!connection)
+ return NULL;
+
+ connection->type = type;
+ connection->msg_id = msg_id;
+
+ return connection;
+}
+
+/**
+ * gh_rm_init_connection_payload() - Fills the first message for a connection.
+ */
+static int gh_rm_init_connection_payload(struct gh_rm_connection *connection, void *msg,
+ size_t hdr_size, size_t payload_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ size_t max_buf_size;
+
+ connection->num_fragments = hdr->fragments;
+ connection->fragments_received = 0;
+ connection->type = hdr->type;
+
+ /* There's not going to be any payload, no need to allocate buffer. */
+ if (!payload_size && !connection->num_fragments)
+ return 0;
+
+ /*
+ * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size
+ * and can received (hdr->fragments + 1) of those
+ */
+ max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (hdr->fragments + 1);
+
+ connection->payload = kzalloc(max_buf_size, GFP_KERNEL);
+ if (!connection->payload)
+ return -ENOMEM;
+
+ memcpy(connection->payload, msg + hdr_size, payload_size);
+ connection->size = payload_size;
+ return 0;
+}
+
+static void gh_rm_notif_work(struct work_struct *work)
+{
+ struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work);
+ struct gh_rm_connection *connection = notif->conn;
+ u32 notif_id = connection->msg_id;
+ struct gh_rm_notification notification = {
+ .buff = connection->payload,
+ .size = connection->size,
+ };
+
+ srcu_notifier_call_chain(&gh_rm_notifier, notif_id, ¬ification);
+
+ kfree(connection->payload);
+ kfree(connection);
+ kfree(notif);
+}
+
+static struct gh_rm_connection *gh_rm_process_notif(struct gh_rsc_mgr *rsc_mgr,
+ void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ struct gh_rm_connection *connection;
+
+ connection = gh_rm_alloc_connection(hdr->msg_id, hdr->type);
+ if (!connection) {
+ pr_err("Failed to alloc connection for notification, dropping.\n");
+ return NULL;
+ }
+
+ if (gh_rm_init_connection_payload(connection, msg, sizeof(*hdr), msg_size - sizeof(*hdr))) {
+ pr_err("Failed to alloc connection buffer for notification, dropping.\n");
+ kfree(connection);
+ return NULL;
+ }
+
+ return connection;
+}
+
+static struct gh_rm_connection *gh_rm_process_rply(struct gh_rsc_mgr *rsc_mgr,
+ void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_reply_hdr *reply_hdr = msg;
+ struct gh_rm_rpc_hdr *hdr = msg;
+ struct gh_rm_connection *connection;
+
+ if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock))
+ return ERR_PTR(-ERESTARTSYS);
+
+ connection = idr_find(&rsc_mgr->call_idr, hdr->seq);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ if (!connection) {
+ pr_err("Failed to find connection for sequence %u\n", hdr->seq);
+ return NULL;
+ }
+ if (connection->msg_id != hdr->msg_id) {
+ pr_err("Reply for sequence %u expected msg_id: %x but got %x\n", hdr->seq,
+ connection->msg_id, hdr->msg_id);
+ /*
+ * Don't complete connection and error the client, maybe resource manager will
+ * send us the expected reply sequence soon.
+ */
+ return NULL;
+ }
+
+ if (gh_rm_init_connection_payload(connection, msg, sizeof(*reply_hdr),
+ msg_size - sizeof(*reply_hdr))) {
+ pr_err("Failed to alloc connection buffer for sequence %d\n", hdr->seq);
+ /* Send connection complete and error the client. */
+ connection->ret = -ENOMEM;
+ complete(&connection->seq_done);
+ return NULL;
+ }
+
+ connection->rm_error = reply_hdr->err_code;
+ return connection;
+}
+
+static void gh_rm_process_cont(struct gh_rm_connection *connection, void *msg, size_t msg_size)
+{
+ struct gh_rm_rpc_hdr *hdr = msg;
+ size_t payload_size = msg_size - sizeof(*hdr);
+
+ /*
+ * hdr->fragments and hdr->msg_id preserves the value from first reply or notif message.
+ * For sake of sanity, check if it's still intact.
+ */
+ if (connection->msg_id != hdr->msg_id)
+ pr_warn("Appending mismatched continuation with id %d to connection with id %d\n",
+ hdr->msg_id, connection->msg_id);
+ if (connection->num_fragments != hdr->fragments)
+ pr_warn("Number of fragments mismatch for seq: %d\n", hdr->seq);
+
+ memcpy(connection->payload + connection->size, msg + sizeof(*hdr), payload_size);
+ connection->size += payload_size;
+ connection->fragments_received++;
+}
+
+static bool gh_rm_complete_connection(struct gh_rm_connection *connection)
+{
+ struct gh_rm_notif_complete *notif_work;
+
+ if (!connection)
+ return false;
+
+ if (connection->fragments_received != connection->num_fragments)
+ return false;
+
+ switch (connection->type) {
+ case GH_RM_RPC_TYPE_RPLY:
+ complete(&connection->seq_done);
+ break;
+ case GH_RM_RPC_TYPE_NOTIF:
+ notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL);
+ if (notif_work == NULL)
+ break;
+
+ notif_work->conn = connection;
+ INIT_WORK(¬if_work->work, gh_rm_notif_work);
+
+ schedule_work(¬if_work->work);
+ break;
+ default:
+ pr_err("Invalid message type (%d) received\n", connection->type);
+ break;
+ }
+
+ return true;
+}
+
+static void gh_rm_abort_connection(struct gh_rm_connection *connection)
+{
+ switch (connection->type) {
+ case GH_RM_RPC_TYPE_RPLY:
+ connection->ret = -EIO;
+ complete(&connection->seq_done);
+ break;
+ case GH_RM_RPC_TYPE_NOTIF:
+ fallthrough;
+ default:
+ kfree(connection->payload);
+ kfree(connection);
+ }
+}
+
+static int gh_rm_recv_task_fn(void *data)
+{
+ struct gh_rsc_mgr *rsc_mgr = data;
+ struct gh_rm_connection *connection = NULL;
+ struct gh_rm_rpc_hdr *hdr = NULL;
+ ssize_t msg_size;
+ void *msg;
+
+ msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
+ while (!kthread_should_stop()) {
+ /* Block until a new message is received */
+ msg_size = gh_msgq_recv(rsc_mgr->msgq_rx, msg, GH_MSGQ_MAX_MSG_SIZE, 0);
+ if (msg_size < 0) {
+ pr_err("Failed to receive the message: %ld\n", msg_size);
+ continue;
+ } else if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) {
+ pr_err("Invalid message size received: %ld is too small\n", msg_size);
+ continue;
+ }
+
+ hdr = msg;
+ switch (hdr->type) {
+ case GH_RM_RPC_TYPE_NOTIF:
+ if (connection) {
+ /* Not possible per protocol. Do something better than BUG_ON */
+ pr_warn("Received start of new notification without finishing existing message series.\n");
+ gh_rm_abort_connection(connection);
+ }
+ connection = gh_rm_process_notif(rsc_mgr, msg, msg_size);
+ break;
+ case GH_RM_RPC_TYPE_RPLY:
+ if (connection) {
+ /* Not possible per protocol. Do something better than BUG_ON */
+ pr_warn("Received start of new reply without finishing existing message series.\n");
+ gh_rm_abort_connection(connection);
+ }
+ connection = gh_rm_process_rply(rsc_mgr, msg, msg_size);
+ break;
+ case GH_RM_RPC_TYPE_CONT:
+ if (!connection) {
+ pr_warn("Received a continuation message without receiving initial message\n");
+ break;
+ }
+ gh_rm_process_cont(connection, msg, msg_size);
+ break;
+ default:
+ pr_err("Invalid message type (%d) received\n", hdr->type);
+ continue;
+ }
+
+ if (gh_rm_complete_connection(connection))
+ connection = NULL;
+ }
+
+ return 0;
+}
+
+static int gh_rm_send_request(struct gh_rsc_mgr *rsc_mgr, u32 message_id,
+ const void *req_buff, size_t req_buff_size,
+ struct gh_rm_connection *connection)
+{
+ size_t buff_size_remaining = req_buff_size;
+ const void *req_buff_curr = req_buff;
+ struct gh_rm_rpc_hdr *hdr;
+ unsigned long tx_flags;
+ u32 num_fragments = 0;
+ size_t payload_size;
+ void *msg;
+ int i, ret = 0;
+
+ if (req_buff_size > GH_RM_MAX_MSG_SIZE)
+ num_fragments = req_buff_size / GH_RM_MAX_MSG_SIZE;
+
+ if (WARN(num_fragments > GH_RM_MAX_NUM_FRAGMENTS,
+ "Limit exceeded for the number of fragments: %u\n", num_fragments))
+ return -E2BIG;
+
+ /*
+ * The above calculation also includes the count for the 'request' packet.
+ * Exclude it as the header needs to fill the num. of fragments to follow.
+ */
+ if (num_fragments)
+ num_fragments--;
+
+ if (mutex_lock_interruptible(&rsc_mgr->send_lock))
+ return -ERESTARTSYS;
+
+ msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!msg) {
+ mutex_unlock(&rsc_mgr->send_lock);
+ return -ENOMEM;
+ }
+
+ /* Consider also the 'request' packet for the loop count */
+ for (i = 0; i <= num_fragments; i++) {
+ if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) {
+ payload_size = GH_RM_MAX_MSG_SIZE;
+ buff_size_remaining -= payload_size;
+ } else {
+ payload_size = buff_size_remaining;
+ }
+
+ memset(msg, 0, GH_MSGQ_MAX_MSG_SIZE);
+
+ /* Fill header */
+ hdr = msg;
+ hdr->version = GH_RM_RPC_HDR_VERSION_ONE;
+ hdr->hdr_words = GH_RM_RPC_HDR_WORDS;
+ hdr->type = i == 0 ? GH_RM_RPC_TYPE_REQ : GH_RM_RPC_TYPE_CONT;
+ hdr->fragments = num_fragments;
+ hdr->seq = connection->seq;
+ hdr->msg_id = message_id;
+
+ /* Copy payload */
+ memcpy(msg + sizeof(*hdr), req_buff_curr, payload_size);
+ req_buff_curr += payload_size;
+
+ /* Force the last fragment to be sent immediately to the receiver */
+ tx_flags = (i == num_fragments) ? GH_MSGQ_TX_PUSH : 0;
+
+ ret = gh_msgq_send(rsc_mgr->msgq_tx, msg, sizeof(*hdr) + payload_size, tx_flags);
+
+ if (ret < 0)
+ break;
+ }
+
+ mutex_unlock(&rsc_mgr->send_lock);
+ return ret < 0 ? ret : 0;
+}
+
+/**
+ * gh_rm_call: Achieve request-response type communication with RPC
+ * @message_id: The RM RPC message-id
+ * @req_buff: Request buffer that contains the payload
+ * @req_buff_size: Total size of the payload
+ * @resp_buf: Pointer to a response buffer
+ * @resp_buff_size: Size of the response buffer
+ * @reply_err_code: Returns Gunyah standard error code for the response
+ *
+ * Make a request to the RM-VM and wait for reply back. For a successful
+ * response, the function returns the payload. The size of the payload is set in resp_buff_size.
+ * The resp_buf should be freed by the caller.
+ *
+ * Context: Process context. Will sleep waiting for reply.
+ * Return: >0 is standard reply error from RM. <0 on internal error.
+ */
+int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
+ void **resp_buf, size_t *resp_buff_size)
+{
+ struct gh_rm_connection *connection;
+ int ret;
+ struct gh_rsc_mgr *rsc_mgr = __rsc_mgr;
+
+ /* messaged_id 0 is reserved */
+ if (!message_id)
+ return -EINVAL;
+
+ if (!rsc_mgr)
+ return -EPROBE_DEFER;
+
+ connection = gh_rm_alloc_connection(message_id, GH_RM_RPC_TYPE_RPLY);
+ if (!connection)
+ return -ENOMEM;
+
+ init_completion(&connection->seq_done);
+
+ /* Allocate a new seq number for this connection */
+ if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) {
+ kfree(connection);
+ return -ERESTARTSYS;
+ }
+ connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ /* Send the request to the Resource Manager */
+ ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection);
+ if (ret < 0)
+ goto out;
+
+ /* Wait for response */
+ wait_for_completion(&connection->seq_done);
+
+ if (connection->ret) {
+ ret = connection->ret;
+ kfree(connection->payload);
+ goto out;
+ }
+
+ if (connection->rm_error) {
+ ret = connection->rm_error;
+ kfree(connection->payload);
+ goto out;
+ }
+
+ *resp_buf = connection->payload;
+ *resp_buff_size = connection->size;
+
+out:
+ mutex_lock(&rsc_mgr->call_idr_lock);
+ idr_remove(&rsc_mgr->call_idr, connection->seq);
+ mutex_unlock(&rsc_mgr->call_idr_lock);
+
+ kfree(connection);
+ return ret;
+}
+
+int gh_rm_register_notifier(struct notifier_block *nb)
+{
+ return srcu_notifier_chain_register(&gh_rm_notifier, nb);
+}
+EXPORT_SYMBOL_GPL(gh_rm_register_notifier);
+
+int gh_rm_unregister_notifier(struct notifier_block *nb)
+{
+ return srcu_notifier_chain_unregister(&gh_rm_notifier, nb);
+}
+EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier);
+
+static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev,
+ u8 gh_type, int idx)
+{
+ int irq, ret;
+ u64 capid;
+ struct device_node *node = pdev->dev.of_node;
+ struct gunyah_device *ghdev;
+
+ irq = platform_get_irq(pdev, idx);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, irq);
+ return ERR_PTR(irq);
+ }
+
+ ret = of_property_read_u64_index(node, "reg", idx, &capid);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret);
+ return ERR_PTR(ret);
+ }
+
+ ghdev = gunyah_device_alloc(&pdev->dev, capid, gh_type);
+ ghdev->irq = irq;
+ ret = gunyah_device_add(ghdev);
+ if (ret) {
+ kfree(ghdev);
+ return ERR_PTR(ret);
+ }
+
+ return ghdev;
+}
+
+static int gh_rm_drv_probe(struct platform_device *pdev)
+{
+ struct gh_rsc_mgr *rsc_mgr;
+ int ret;
+
+ rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL);
+ if (!rsc_mgr)
+ return -ENOMEM;
+ platform_set_drvdata(pdev, rsc_mgr);
+
+ mutex_init(&rsc_mgr->call_idr_lock);
+ idr_init(&rsc_mgr->call_idr);
+ mutex_init(&rsc_mgr->send_lock);
+
+ rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0);
+ if (IS_ERR(rsc_mgr->msgq_tx))
+ return PTR_ERR(rsc_mgr->msgq_tx);
+ rsc_mgr->msgq_rx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_RX, 1);
+ if (IS_ERR(rsc_mgr->msgq_rx)) {
+ ret = PTR_ERR(rsc_mgr->msgq_rx);
+ goto err_msgq_tx;
+ }
+
+ rsc_mgr->recv_task = kthread_run(gh_rm_recv_task_fn, rsc_mgr, "gh_rm_recv_task");
+ if (IS_ERR_OR_NULL(rsc_mgr->recv_task)) {
+ ret = PTR_ERR(rsc_mgr->recv_task);
+ goto err_msgq;
+ }
+
+ __rsc_mgr = rsc_mgr;
+
+ return 0;
+
+err_msgq:
+ gunyah_device_remove(rsc_mgr->msgq_rx);
+err_msgq_tx:
+ gunyah_device_remove(rsc_mgr->msgq_tx);
+ return ret;
+}
+
+static int gh_rm_drv_remove(struct platform_device *pdev)
+{
+ struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev);
+
+ gunyah_device_remove(rsc_mgr->msgq_tx);
+ gunyah_device_remove(rsc_mgr->msgq_rx);
+
+ return 0;
+}
+
+static const struct of_device_id gh_rm_of_match[] = {
+ { .compatible = "gunyah-resource-manager" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, gh_rm_of_match);
+
+static struct platform_driver gh_rsc_mgr_driver = {
+ .probe = gh_rm_drv_probe,
+ .remove = gh_rm_drv_remove,
+ .driver = {
+ .name = "gh_rsc_mgr",
+ .of_match_table = gh_rm_of_match,
+ },
+};
+
+int __init gh_rsc_mgr_init(void)
+{
+ return platform_driver_register(&gh_rsc_mgr_driver);
+}
+
+void gh_rsc_mgr_exit(void)
+{
+ platform_driver_unregister(&gh_rsc_mgr_driver);
+}
diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h
new file mode 100644
index 000000000000..e4f2499267bf
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef __GH_RSC_MGR_PRIV_H
+#define __GH_RSC_MGR_PRIV_H
+
+#include <linux/gunyah.h>
+
+/* RM Error codes */
+#define GH_RM_ERROR_OK 0x0
+#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF
+#define GH_RM_ERROR_NOMEM 0x1
+#define GH_RM_ERROR_NORESOURCE 0x2
+#define GH_RM_ERROR_DENIED 0x3
+#define GH_RM_ERROR_INVALID 0x4
+#define GH_RM_ERROR_BUSY 0x5
+#define GH_RM_ERROR_ARGUMENT_INVALID 0x6
+#define GH_RM_ERROR_HANDLE_INVALID 0x7
+#define GH_RM_ERROR_VALIDATE_FAILED 0x8
+#define GH_RM_ERROR_MAP_FAILED 0x9
+#define GH_RM_ERROR_MEM_INVALID 0xA
+#define GH_RM_ERROR_MEM_INUSE 0xB
+#define GH_RM_ERROR_MEM_RELEASED 0xC
+#define GH_RM_ERROR_VMID_INVALID 0xD
+#define GH_RM_ERROR_LOOKUP_FAILED 0xE
+#define GH_RM_ERROR_IRQ_INVALID 0xF
+#define GH_RM_ERROR_IRQ_INUSE 0x10
+#define GH_RM_ERROR_IRQ_RELEASED 0x11
+
+int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
+ void **resp_buf, size_t *resp_buff_size);
+
+#endif
diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c
index 7589689e5e92..7c0efc80f85e 100644
--- a/drivers/virt/gunyah/sysfs.c
+++ b/drivers/virt/gunyah/sysfs.c
@@ -148,7 +148,13 @@ static int __init gunyah_init(void)
if (ret)
goto err_bus;
+ ret = gh_rsc_mgr_init();
+ if (ret)
+ goto err_msgq;
+
return ret;
+err_msgq:
+ gh_msgq_exit();
err_bus:
gunyah_bus_exit();
err_sysfs:
@@ -159,6 +165,7 @@ module_init(gunyah_init);
static void __exit gunyah_exit(void)
{
+ gh_rsc_mgr_exit();
gh_msgq_exit();
gunyah_bus_exit();
gh_sysfs_unregister();
diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h
new file mode 100644
index 000000000000..015bd851e1a3
--- /dev/null
+++ b/include/linux/gunyah_rsc_mgr.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _GUNYAH_RSC_MGR_H
+#define _GUNYAH_RSC_MGR_H
+
+#include <linux/list.h>
+#include <linux/notifier.h>
+#include <linux/gunyah.h>
+
+typedef u16 gh_vmid_t;
+typedef u32 gh_virq_handle_t;
+
+#define GH_VMID_INVAL U16_MAX
+
+/* Gunyah recognizes VMID0 as an alias to the current VM's ID */
+#define GH_VMID_SELF 0
+
+struct gh_rm_notification {
+ const void *buff;
+ const size_t size;
+};
+
+int gh_rm_register_notifier(struct notifier_block *nb);
+int gh_rm_unregister_notifier(struct notifier_block *nb);
+
+#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (7 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
2022-07-14 21:29 ` [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Gunyah resource manager exposes a concrete functionalities which
complicate a single resource manager driver. Use auxiliary bus
to help split high level functions for the resource manager and keep the
primary resource manager driver focused on the RPC with RM itself.
Delegate Resource Manager's console functionality to the auxiliary bus.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/rsc_mgr.c | 61 ++++++++++++++++++++++++++++++++++-
1 file changed, 60 insertions(+), 1 deletion(-)
diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c
index b8268ee02fab..44b22cef7d44 100644
--- a/drivers/virt/gunyah/rsc_mgr.c
+++ b/drivers/virt/gunyah/rsc_mgr.c
@@ -91,6 +91,11 @@ struct gh_rm_notif_complete {
struct work_struct work;
};
+struct gh_rsc_mgr_adev {
+ struct auxiliary_device adev;
+ struct list_head list;
+};
+
struct gh_rsc_mgr {
struct task_struct *recv_task;
struct gunyah_device *msgq_tx, *msgq_rx;
@@ -99,6 +104,13 @@ struct gh_rsc_mgr {
struct mutex call_idr_lock;
struct mutex send_lock;
+
+ struct list_head adevs;
+};
+
+/* List of auxiliary devices which resource manager creates */
+static const char * const adev_names[] = {
+ "console",
};
static struct gh_rsc_mgr *__rsc_mgr;
@@ -516,6 +528,14 @@ int gh_rm_unregister_notifier(struct notifier_block *nb)
}
EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier);
+static void gh_rm_adev_release(struct device *dev)
+{
+ struct gh_rsc_mgr_adev *rm_adev = container_of(dev, struct gh_rsc_mgr_adev, adev.dev);
+
+ list_del(&rm_adev->list);
+ kfree(rm_adev);
+}
+
static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev,
u8 gh_type, int idx)
{
@@ -550,7 +570,9 @@ static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_de
static int gh_rm_drv_probe(struct platform_device *pdev)
{
struct gh_rsc_mgr *rsc_mgr;
- int ret;
+ struct gh_rsc_mgr_adev *rm_adev;
+ struct list_head *l, *n;
+ int ret, i;
rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL);
if (!rsc_mgr)
@@ -560,6 +582,7 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
mutex_init(&rsc_mgr->call_idr_lock);
idr_init(&rsc_mgr->call_idr);
mutex_init(&rsc_mgr->send_lock);
+ INIT_LIST_HEAD(&rsc_mgr->adevs);
rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0);
if (IS_ERR(rsc_mgr->msgq_tx))
@@ -576,10 +599,38 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
goto err_msgq;
}
+ for (i = 0; i < ARRAY_SIZE(adev_names); i++) {
+ rm_adev = kzalloc(sizeof(*rm_adev), GFP_KERNEL);
+
+ rm_adev->adev.dev.parent = &pdev->dev;
+ rm_adev->adev.dev.release = gh_rm_adev_release;
+ rm_adev->adev.name = adev_names[i];
+ ret = auxiliary_device_init(&rm_adev->adev);
+ if (ret) {
+ kfree(rm_adev);
+ goto err_adevs;
+ }
+
+ list_add(&rm_adev->list, &rsc_mgr->adevs);
+
+ ret = auxiliary_device_add(&rm_adev->adev);
+ if (ret) {
+ auxiliary_device_uninit(&rm_adev->adev);
+ goto err_adevs;
+ }
+ }
+
__rsc_mgr = rsc_mgr;
return 0;
+err_adevs:
+ list_for_each_safe(l, n, &rsc_mgr->adevs) {
+ rm_adev = container_of(l, struct gh_rsc_mgr_adev, list);
+ auxiliary_device_delete(&rm_adev->adev);
+ auxiliary_device_uninit(&rm_adev->adev);
+ }
+
err_msgq:
gunyah_device_remove(rsc_mgr->msgq_rx);
err_msgq_tx:
@@ -590,6 +641,14 @@ static int gh_rm_drv_probe(struct platform_device *pdev)
static int gh_rm_drv_remove(struct platform_device *pdev)
{
struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev);
+ struct gh_rsc_mgr_adev *rm_adev;
+ struct list_head *l, *n;
+
+ list_for_each_safe(l, n, &rsc_mgr->adevs) {
+ rm_adev = container_of(l, struct gh_rsc_mgr_adev, list);
+ auxiliary_device_delete(&rm_adev->adev);
+ auxiliary_device_uninit(&rm_adev->adev);
+ }
gunyah_device_remove(rsc_mgr->msgq_tx);
gunyah_device_remove(rsc_mgr->msgq_rx);
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (8 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
2022-07-14 21:29 ` [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Gunyah resource manager defines a simple API for virtual machine log
sharing with the console service. A VM's own log can be opened by using
GH_VMID_SELF. Another VM's log can be accessed via its VMID. Once
opened, characters can be written to the log with a write command.
Characters are received with resource manager notifications (using ID
GH_RM_NOTIF_VM_CONSOLE_CHARS).
These high level rpc calls are kept in
drivers/virt/gunyah/rsc_mgr_rpc.c. Future RPC calls, e.g. to launch a VM
will also be maintained in this file.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
drivers/virt/gunyah/Makefile | 4 +-
drivers/virt/gunyah/rsc_mgr.h | 22 +++++
drivers/virt/gunyah/rsc_mgr_rpc.c | 151 ++++++++++++++++++++++++++++++
include/linux/gunyah_rsc_mgr.h | 16 ++++
4 files changed, 191 insertions(+), 2 deletions(-)
create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index 86655bca8944..b3f15c052297 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
gunyah-y += sysfs.o device.o msgq.o
-gunyah-y += rsc_mgr.o
-obj-$(CONFIG_GUNYAH) += gunyah.o
\ No newline at end of file
+gunyah-y += rsc_mgr.o rsc_mgr_rpc.o
+obj-$(CONFIG_GUNYAH) += gunyah.o
diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h
index e4f2499267bf..c82a2e97ad49 100644
--- a/drivers/virt/gunyah/rsc_mgr.h
+++ b/drivers/virt/gunyah/rsc_mgr.h
@@ -28,6 +28,28 @@
#define GH_RM_ERROR_IRQ_INUSE 0x10
#define GH_RM_ERROR_IRQ_RELEASED 0x11
+/* Message IDs: VM Management */
+#define GH_RM_RPC_VM_GET_VMID 0x56000024
+
+/* Message IDs: VM Services */
+#define GH_RM_RPC_VM_CONSOLE_OPEN_ID 0x56000081
+#define GH_RM_RPC_VM_CONSOLE_CLOSE_ID 0x56000082
+#define GH_RM_RPC_VM_CONSOLE_WRITE_ID 0x56000083
+#define GH_RM_RPC_VM_CONSOLE_FLUSH_ID 0x56000084
+
+/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */
+struct gh_vm_console_common_req {
+ gh_vmid_t vmid;
+ u16 reserved0;
+} __packed;
+
+/* Call: CONSOLE_WRITE */
+struct gh_vm_console_write_req {
+ gh_vmid_t vmid;
+ u16 num_bytes;
+ u8 data[0];
+} __packed;
+
int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size,
void **resp_buf, size_t *resp_buff_size);
diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c
new file mode 100644
index 000000000000..be9317968537
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr_rpc.c
@@ -0,0 +1,151 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr: " fmt
+
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/printk.h>
+#include <linux/gunyah_rsc_mgr.h>
+
+#include "rsc_mgr.h"
+
+/**
+ * gh_rm_get_vmid: Retrieve VMID of this virtual machine
+ * @vmid: Filled with the VMID of this VM
+ */
+int gh_rm_get_vmid(gh_vmid_t *vmid)
+{
+ void *resp;
+ size_t resp_size;
+ int ret;
+ int payload = 0;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_GET_VMID, &payload, sizeof(payload), &resp, &resp_size);
+ if (ret)
+ return ret;
+
+ if (resp_size != sizeof(*vmid))
+ return -EIO;
+ *vmid = *(gh_vmid_t *)resp;
+ kfree(resp);
+
+ return ret;
+}
+
+/**
+ * gh_rm_console_open: Open a console with a VM
+ * @vmid: VMID of the other vmid whose console to open. If VMID is GH_VMID_SELF, the
+ * console associated with this VM is opened.
+ */
+int gh_rm_console_open(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_OPEN_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_OPEN: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_open);
+
+/**
+ * gh_rm_console_close: Close a console with a VM
+ * @vmid: The vmid of the vm whose console to close.
+ */
+int gh_rm_console_close(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_CLOSE_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_CLOSE: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_close);
+
+/**
+ * gh_rm_console_write: Write to a VM's console
+ * @vmid: The vmid of the vm whose console to write to.
+ * @buf: Buffer to write to the VM's console
+ * @size: Size of the buffer
+ */
+int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size)
+{
+ void *resp;
+ struct gh_vm_console_write_req *req_payload;
+ size_t resp_size;
+ int ret = 0;
+ size_t req_payload_size = sizeof(*req_payload) + size;
+
+ if (size < 1 || size > (U32_MAX - sizeof(*req_payload)))
+ return -EINVAL;
+
+ req_payload = kzalloc(req_payload_size, GFP_KERNEL);
+
+ if (!req_payload)
+ return -ENOMEM;
+
+ req_payload->vmid = vmid;
+ req_payload->num_bytes = size;
+ memcpy(req_payload->data, buf, size);
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_WRITE_ID,
+ req_payload, req_payload_size,
+ &resp, &resp_size);
+ kfree(req_payload);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_WRITE: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_write);
+
+/**
+ * gh_rm_console_flush: Flush a console with a VM
+ * @vmid: The vmid of the vm whose console to flush
+ */
+int gh_rm_console_flush(gh_vmid_t vmid)
+{
+ void *resp;
+ struct gh_vm_console_common_req req_payload = {0};
+ size_t resp_size;
+ int ret;
+
+ req_payload.vmid = vmid;
+
+ ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_FLUSH_ID,
+ &req_payload, sizeof(req_payload),
+ &resp, &resp_size);
+ kfree(resp);
+
+ if (!ret && resp_size)
+ pr_warn("Received unexpected payload for CONSOLE_FLUSH: %lu\n", resp_size);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gh_rm_console_flush);
diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h
index 015bd851e1a3..4211ac9a219c 100644
--- a/include/linux/gunyah_rsc_mgr.h
+++ b/include/linux/gunyah_rsc_mgr.h
@@ -26,4 +26,20 @@ struct gh_rm_notification {
int gh_rm_register_notifier(struct notifier_block *nb);
int gh_rm_unregister_notifier(struct notifier_block *nb);
+/* Notification type Message IDs */
+#define GH_RM_NOTIF_VM_CONSOLE_CHARS 0x56100080
+
+struct gh_rm_notif_vm_console_chars {
+ gh_vmid_t vmid;
+ u16 num_bytes;
+ u8 bytes[0];
+} __packed;
+
+/* RPC Calls */
+int gh_rm_get_vmid(gh_vmid_t *vmid);
+int gh_rm_console_open(gh_vmid_t vmid);
+int gh_rm_console_close(gh_vmid_t vmid);
+int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size);
+int gh_rm_console_flush(gh_vmid_t vmid);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
` (9 preceding siblings ...)
2022-07-14 21:29 ` [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
@ 2022-07-14 21:29 ` Elliot Berman
10 siblings, 0 replies; 30+ messages in thread
From: Elliot Berman @ 2022-07-14 21:29 UTC (permalink / raw)
To: Bjorn Andersson, linux-kernel
Cc: Elliot Berman, Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
Gunyah provides a console for each VM using the VM console resource
manager APIs. This driver allows console data from other
VMs to be accessed via a TTY device and exports a console device to dump
Linux's own logs to our console.
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
Documentation/virt/gunyah/index.rst | 7 +
drivers/virt/gunyah/Kconfig | 10 +
drivers/virt/gunyah/Makefile | 3 +
drivers/virt/gunyah/rsc_mgr_console.c | 405 ++++++++++++++++++++++++++
4 files changed, 425 insertions(+)
create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
index e7bb2b14543e..95ba9b71ab30 100644
--- a/Documentation/virt/gunyah/index.rst
+++ b/Documentation/virt/gunyah/index.rst
@@ -90,3 +90,10 @@ When booting a virtual machine which uses a devicetree, resource manager overlay
how to communicate with resource manager, and basic description and capabilities of
this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description
of this node.
+
+Resource Manager Consoles
+-------------------------
+RM provides infrastructure for virtual machines to share an interactive console. This can be used to
+interact with a VM which may not have access to a serial port. Linux will register a printk console:
+ttyGH0. That console and other VM's consoles can be accessed via ttyGHX.
+/sys/class/tty/ttyGHX/vmid will print the VM which is associated with that TTY.
diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
index 2ef4887e280d..f5a951ee2b4a 100644
--- a/drivers/virt/gunyah/Kconfig
+++ b/drivers/virt/gunyah/Kconfig
@@ -12,3 +12,13 @@ config GUNYAH
Say Y here to enable the drivers needed to interact in a Gunyah
virtual environment.
+
+if GUNYAH
+config GUNYAH_RESOURCE_MANAGER_CONSOLE
+ tristate "Gunyah Resource Manager Consoles"
+ depends on TTY
+ help
+ This enables support for console output using Gunyah's Resource Manager RPC.
+ This is normally used when a secondary VM which does not have exclusive access
+ to a real serial device and virtio-console is unavailable.
+endif
diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile
index b3f15c052297..001cf1630c03 100644
--- a/drivers/virt/gunyah/Makefile
+++ b/drivers/virt/gunyah/Makefile
@@ -3,3 +3,6 @@
gunyah-y += sysfs.o device.o msgq.o
gunyah-y += rsc_mgr.o rsc_mgr_rpc.o
obj-$(CONFIG_GUNYAH) += gunyah.o
+
+gunyah_console-y += rsc_mgr_console.o
+obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER_CONSOLE) += gunyah_console.o
diff --git a/drivers/virt/gunyah/rsc_mgr_console.c b/drivers/virt/gunyah/rsc_mgr_console.c
new file mode 100644
index 000000000000..eda25100314f
--- /dev/null
+++ b/drivers/virt/gunyah/rsc_mgr_console.c
@@ -0,0 +1,405 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#define pr_fmt(fmt) "gh_rsc_mgr_console: " fmt
+
+#include <linux/gunyah_rsc_mgr.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+#include <linux/tty_flip.h>
+#include <linux/console.h>
+#include <linux/module.h>
+#include <linux/kfifo.h>
+#include <linux/kref.h>
+#include <linux/slab.h>
+#include <linux/tty.h>
+#include <linux/of.h>
+
+/*
+ * The Linux TTY code does not support dynamic addition of tty derived devices so we need to know
+ * how many tty devices we might need when space is allocated for the tty device. Since VMs might be
+ * added/removed dynamically, we need to make sure we have enough allocated.
+ */
+#define RSC_MGR_TTY_ADAPTERS 16
+
+/* # of payload bytes that can fit in a 1-fragment CONSOLE_WRITE message */
+#define RM_CONS_WRITE_MSG_SIZE ((1 * (GH_MSGQ_MAX_MSG_SIZE - 8)) - 4)
+
+struct rm_cons_port {
+ struct tty_port port;
+ gh_vmid_t vmid;
+ bool open;
+ unsigned int index;
+
+ DECLARE_KFIFO(put_fifo, char, 1024);
+ spinlock_t fifo_lock;
+ struct work_struct put_work;
+
+ struct rm_cons_data *cons_data;
+};
+
+struct rm_cons_data {
+ struct tty_driver *tty_driver;
+ struct device *dev;
+
+ spinlock_t ports_lock;
+ struct rm_cons_port *ports[RSC_MGR_TTY_ADAPTERS];
+
+ struct notifier_block rsc_mgr_notif;
+ struct console console;
+};
+
+static void put_work_fn(struct work_struct *ws)
+{
+ char buf[RM_CONS_WRITE_MSG_SIZE];
+ int count, ret;
+ struct rm_cons_port *port = container_of(ws, struct rm_cons_port, put_work);
+
+ while (!kfifo_is_empty(&port->put_fifo)) {
+ count = kfifo_out_spinlocked(&port->put_fifo, buf, sizeof(buf), &port->fifo_lock);
+ if (count <= 0)
+ continue;
+
+ ret = gh_rm_console_write(port->vmid, buf, count);
+ if (ret) {
+ pr_warn_once("failed to send characters: %d\n", ret);
+ break;
+ }
+ }
+}
+
+static int rsc_mgr_console_notif(struct notifier_block *nb, unsigned long cmd, void *data)
+{
+ int count, i;
+ struct rm_cons_port *rm_port;
+ struct tty_port *tty_port = NULL;
+ struct rm_cons_data *cons_data = container_of(nb, struct rm_cons_data, rsc_mgr_notif);
+ const struct gh_rm_notification *notif = data;
+ struct gh_rm_notif_vm_console_chars const * const msg = notif->buff;
+
+ if (cmd != GH_RM_NOTIF_VM_CONSOLE_CHARS ||
+ notif->size < sizeof(*msg))
+ return NOTIFY_DONE;
+
+ spin_lock(&cons_data->ports_lock);
+ for (i = 0; i < RSC_MGR_TTY_ADAPTERS; i++) {
+ if (!cons_data->ports[i])
+ continue;
+ if (cons_data->ports[i]->vmid == msg->vmid) {
+ rm_port = cons_data->ports[i];
+ break;
+ }
+ }
+ if (rm_port)
+ tty_port = tty_port_get(&rm_port->port);
+ spin_unlock(&cons_data->ports_lock);
+
+ if (!rm_port)
+ pr_warn("Received unexpected console characters for VMID %u\n", msg->vmid);
+ if (!tty_port)
+ return NOTIFY_DONE;
+
+ count = tty_buffer_request_room(tty_port, msg->num_bytes);
+ tty_insert_flip_string(tty_port, msg->bytes, count);
+ tty_flip_buffer_push(tty_port);
+
+ tty_port_put(tty_port);
+ return NOTIFY_OK;
+}
+
+static ssize_t vmid_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(dev);
+
+ if (rm_port->vmid == GH_VMID_SELF)
+ return sysfs_emit(buf, "self\n");
+
+ return sysfs_emit(buf, "%u\n", rm_port->vmid);
+}
+
+static DEVICE_ATTR_RO(vmid);
+
+static struct attribute *rsc_mgr_tty_dev_attrs[] = {
+ &dev_attr_vmid.attr,
+ NULL
+};
+
+static const struct attribute_group rsc_mgr_tty_dev_attr_group = {
+ .attrs = rsc_mgr_tty_dev_attrs,
+};
+
+static const struct attribute_group *rsc_mgr_tty_dev_attr_groups[] = {
+ &rsc_mgr_tty_dev_attr_group,
+ NULL
+};
+
+static int rsc_mgr_tty_open(struct tty_struct *tty, struct file *filp)
+{
+ int ret;
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ if (!rm_port->open) {
+ ret = gh_rm_console_open(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = true;
+ }
+
+ return tty_port_open(&rm_port->port, tty, filp);
+}
+
+static void rsc_mgr_tty_close(struct tty_struct *tty, struct file *filp)
+{
+ int ret;
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ if (rm_port->open) {
+ if (rm_port->vmid != GH_VMID_SELF) {
+ ret = gh_rm_console_close(rm_port->vmid);
+ if (ret)
+ pr_warn("Failed to close RM console for vmid %d: %d\n",
+ rm_port->vmid, ret);
+ }
+ rm_port->open = false;
+
+ tty_port_close(&rm_port->port, tty, filp);
+ }
+
+}
+
+static int rsc_mgr_tty_write(struct tty_struct *tty, const unsigned char *buf, int count)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+ int ret;
+
+ ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock);
+ if (ret > 0)
+ schedule_work(&rm_port->put_work);
+
+ return ret;
+}
+
+static unsigned int rsc_mgr_mgr_tty_write_room(struct tty_struct *tty)
+{
+ struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev);
+
+ return kfifo_avail(&rm_port->put_fifo);
+}
+
+static void rsc_mgr_console_write(struct console *co, const char *buf, unsigned count)
+{
+ struct rm_cons_port *rm_port = co->data;
+ int ret;
+
+ ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock);
+ if (ret > 0)
+ schedule_work(&rm_port->put_work);
+}
+
+static struct tty_driver *rsc_mgr_console_device(struct console *co, int *index)
+{
+ struct rm_cons_port *rm_port = co->data;
+
+ *index = rm_port->index;
+ return rm_port->port.tty->driver;
+}
+
+static int rsc_mgr_console_setup(struct console *co, char *unused)
+{
+ int ret;
+ struct rm_cons_port *rm_port = co->data;
+
+ if (!rm_port->open) {
+ ret = gh_rm_console_open(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = true;
+ }
+
+ return 0;
+}
+
+static int rsc_mgr_console_exit(struct console *co)
+{
+ int ret;
+ struct rm_cons_port *rm_port = co->data;
+
+ if (rm_port->open) {
+ ret = gh_rm_console_close(rm_port->vmid);
+ if (ret) {
+ pr_err("Failed to close RM console for vmid %x: %d\n", rm_port->vmid, ret);
+ return ret;
+ }
+ rm_port->open = false;
+ }
+
+ return 0;
+}
+
+static const struct tty_operations rsc_mgr_tty_ops = {
+ .open = rsc_mgr_tty_open,
+ .close = rsc_mgr_tty_close,
+ .write = rsc_mgr_tty_write,
+ .write_room = rsc_mgr_mgr_tty_write_room,
+};
+
+static void rsc_mgr_port_destruct(struct tty_port *port)
+{
+ struct rm_cons_port *rm_port = container_of(port, struct rm_cons_port, port);
+ struct rm_cons_data *cons_data = rm_port->cons_data;
+
+ spin_lock(&cons_data->ports_lock);
+ WARN_ON(cons_data->ports[rm_port->index] != rm_port);
+ cons_data->ports[rm_port->index] = NULL;
+ spin_unlock(&cons_data->ports_lock);
+ kfree(rm_port);
+}
+
+static const struct tty_port_operations rsc_mgr_port_ops = {
+ .destruct = rsc_mgr_port_destruct,
+};
+
+static struct rm_cons_port *rsc_mgr_port_create(struct rm_cons_data *cons_data, gh_vmid_t vmid)
+{
+ struct rm_cons_port *rm_port;
+ struct device *ttydev;
+ unsigned int index;
+ int ret;
+
+ rm_port = kzalloc(sizeof(*rm_port), GFP_KERNEL);
+ rm_port->vmid = vmid;
+ INIT_KFIFO(rm_port->put_fifo);
+ spin_lock_init(&rm_port->fifo_lock);
+ INIT_WORK(&rm_port->put_work, put_work_fn);
+ tty_port_init(&rm_port->port);
+ rm_port->port.ops = &rsc_mgr_port_ops;
+
+ spin_lock(&cons_data->ports_lock);
+ for (index = 0; index < RSC_MGR_TTY_ADAPTERS; index++) {
+ if (!cons_data->ports[index]) {
+ cons_data->ports[index] = rm_port;
+ rm_port->index = index;
+ break;
+ }
+ }
+ spin_unlock(&cons_data->ports_lock);
+ if (index >= RSC_MGR_TTY_ADAPTERS) {
+ ret = -ENOSPC;
+ goto err_put_port;
+ }
+
+ ttydev = tty_port_register_device_attr(&rm_port->port, cons_data->tty_driver, index,
+ cons_data->dev, rm_port, rsc_mgr_tty_dev_attr_groups);
+ if (IS_ERR(ttydev)) {
+ ret = PTR_ERR(ttydev);
+ goto err_put_port;
+ }
+
+ return rm_port;
+err_put_port:
+ tty_port_put(&rm_port->port);
+ return ERR_PTR(ret);
+}
+
+static int rsc_mgr_console_probe(struct auxiliary_device *auxdev,
+ const struct auxiliary_device_id *aux_dev_id)
+{
+ struct rm_cons_data *cons_data;
+ struct rm_cons_port *rm_port;
+ int ret;
+ gh_vmid_t vmid;
+
+ cons_data = devm_kzalloc(&auxdev->dev, sizeof(*cons_data), GFP_KERNEL);
+ if (!cons_data)
+ return -ENOMEM;
+ dev_set_drvdata(&auxdev->dev, cons_data);
+ cons_data->dev = &auxdev->dev;
+
+ cons_data->tty_driver = tty_alloc_driver(RSC_MGR_TTY_ADAPTERS,
+ TTY_DRIVER_REAL_RAW | TTY_DRIVER_DYNAMIC_DEV);
+ if (IS_ERR(cons_data->tty_driver))
+ return PTR_ERR(cons_data->tty_driver);
+
+ cons_data->tty_driver->driver_name = "gh";
+ cons_data->tty_driver->name = "ttyGH";
+ cons_data->tty_driver->type = TTY_DRIVER_TYPE_SYSTEM;
+ cons_data->tty_driver->init_termios = tty_std_termios;
+ tty_set_operations(cons_data->tty_driver, &rsc_mgr_tty_ops);
+
+ ret = tty_register_driver(cons_data->tty_driver);
+ if (ret) {
+ dev_err(&auxdev->dev, "Could not register tty driver: %d\n", ret);
+ goto err_put_tty;
+ }
+
+ spin_lock_init(&cons_data->ports_lock);
+
+ cons_data->rsc_mgr_notif.notifier_call = rsc_mgr_console_notif;
+ ret = gh_rm_register_notifier(&cons_data->rsc_mgr_notif);
+ if (ret) {
+ dev_err(&auxdev->dev, "Could not register for resource manager notifications: %d\n",
+ ret);
+ goto err_put_tty;
+ }
+
+ rm_port = rsc_mgr_port_create(cons_data, GH_VMID_SELF);
+ if (IS_ERR(rm_port)) {
+ ret = PTR_ERR(rm_port);
+ dev_err(&auxdev->dev, "Could not create own console: %d\n", ret);
+ goto err_unreg_notif;
+ }
+
+ strncpy(cons_data->console.name, "ttyGH", sizeof(cons_data->console.name));
+ cons_data->console.write = rsc_mgr_console_write;
+ cons_data->console.device = rsc_mgr_console_device;
+ cons_data->console.setup = rsc_mgr_console_setup;
+ cons_data->console.exit = rsc_mgr_console_exit;
+ cons_data->console.index = rm_port->index;
+ cons_data->console.data = rm_port;
+ register_console(&cons_data->console);
+
+ ret = gh_rm_get_vmid(&vmid);
+ if (!ret)
+ rsc_mgr_port_create(cons_data, vmid);
+ else
+ pr_warn("Failed to get this VM's VMID: %d. Not creating loop-back console\n", ret);
+
+ return 0;
+err_unreg_notif:
+ gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif);
+err_put_tty:
+ tty_driver_kref_put(cons_data->tty_driver);
+ return ret;
+}
+
+static void rsc_mgr_console_remove(struct auxiliary_device *auxdev)
+{
+ struct rm_cons_data *cons_data = dev_get_drvdata(&auxdev->dev);
+
+ unregister_console(&cons_data->console);
+ gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif);
+ tty_driver_kref_put(cons_data->tty_driver);
+}
+
+static struct auxiliary_device_id rsc_mgr_console_ids[] = {
+ { .name = "gunyah.console" },
+ {}
+};
+MODULE_DEVICE_TABLE(auxiliary, rsc_mgr_console_ids);
+
+static struct auxiliary_driver rsc_mgr_console_drv = {
+ .probe = rsc_mgr_console_probe,
+ .remove = rsc_mgr_console_remove,
+ .id_table = rsc_mgr_console_ids,
+};
+module_auxiliary_driver(rsc_mgr_console_drv);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Gunyah Console");
--
2.25.1
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH v2 05/11] virt: gunyah: Add sysfs nodes
2022-07-14 21:29 ` [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
@ 2022-07-14 22:41 ` Randy Dunlap
0 siblings, 0 replies; 30+ messages in thread
From: Randy Dunlap @ 2022-07-14 22:41 UTC (permalink / raw)
To: Elliot Berman, Bjorn Andersson, linux-kernel
Cc: Trilok Soni, Murali Nalajala, Srivatsa Vaddagiri,
Carl van Schaik, Andy Gross, linux-arm-kernel, Lorenzo Pieralisi,
Sudeep Holla, Marc Zyngier, Rob Herring, Krzysztof Kozlowski,
Jonathan Corbet, Will Deacon, Catalin Marinas, devicetree,
linux-doc
On 7/14/22 14:29, Elliot Berman wrote:
> diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig
> new file mode 100644
> index 000000000000..e88289963518
> --- /dev/null
> +++ b/drivers/virt/gunyah/Kconfig
> @@ -0,0 +1,13 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +config GUNYAH
> + tristate "Gunyah Virtualization drivers"
> + depends on ARM64
> + select SYS_HYPERVISOR
> + help
> + The Gunyah drivers are the helper interfaces that runs in a guest VM
> + such as basic inter-VM IPC and signaling mechanism,s and higher level
Drop the "comma" ..................................^^^^
> + services such as memory/device sharing, IRQ sharing, and so on.
> +
> + Say Y here to enable the drivers needed to interact in a Gunyah
> + virtual environment.
--
~Randy
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2022-07-14 22:41 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-23 23:37 [PATCH 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-02-23 23:37 ` [PATCH 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
2022-02-23 23:37 ` [PATCH 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
2022-02-25 20:01 ` Rob Herring
2022-03-02 23:53 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
2022-02-24 10:10 ` Mark Rutland
2022-02-24 10:26 ` Marc Zyngier
2022-04-13 17:09 ` Elliot Berman
2022-02-23 23:37 ` [PATCH 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
2022-02-23 23:37 ` [PATCH 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
2022-02-23 23:37 ` [PATCH 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
2022-02-23 23:37 ` [PATCH 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
2022-02-23 23:37 ` [PATCH 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
2022-02-23 23:37 ` [PATCH 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
2022-02-23 23:37 ` [PATCH 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
2022-02-23 23:37 ` [PATCH 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
2022-07-14 21:29 ` [PATCH v2 00/11] Gunyah Hypervisor drivers Elliot Berman
2022-07-14 21:29 ` [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
2022-07-14 21:29 ` [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
2022-07-14 21:29 ` [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Elliot Berman
2022-07-14 21:29 ` [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
2022-07-14 21:29 ` [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Elliot Berman
2022-07-14 22:41 ` Randy Dunlap
2022-07-14 21:29 ` [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices Elliot Berman
2022-07-14 21:29 ` [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues Elliot Berman
2022-07-14 21:29 ` [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
2022-07-14 21:29 ` [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Elliot Berman
2022-07-14 21:29 ` [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services Elliot Berman
2022-07-14 21:29 ` [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces Elliot Berman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).