* [PATCH v4 0/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-10 9:40 ` Ricardo Cañuelo
0 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: virtualization, mst, jasowang, kernel, cohuck, bagasdotme
Basic documentation about virtio in the kernel and a small tutorial for
virtio drivers.
Tested on linux-next (next-20220802)
Changes in v4:
- reword the intro paragraphs
- grammar fixes
Changes in v3:
- fix commit message in patch 1
- minor additional fixes to virtio kerneldocs
- use proper Sphinx markup for links to references
Changes in v2:
- virtio spec links updated to v1.2
- simplify virtio.rst and remove most low level parts that are
already covered by the spec
- split the kerneldocs fixes to a separate patch
- remove :c:func: rst formatting
Ricardo Cañuelo (2):
virtio: kerneldocs fixes and enhancements
docs: driver-api: virtio: virtio on Linux
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/virtio/index.rst | 11 ++
Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
.../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
MAINTAINERS | 1 +
drivers/virtio/virtio_ring.c | 8 +
include/linux/virtio.h | 6 +-
include/linux/virtio_config.h | 6 +-
include/uapi/linux/virtio_ring.h | 16 +-
9 files changed, 368 insertions(+), 11 deletions(-)
create mode 100644 Documentation/driver-api/virtio/index.rst
create mode 100644 Documentation/driver-api/virtio/virtio.rst
create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
--
2.25.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v4 0/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-10 9:40 ` Ricardo Cañuelo
0 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: mst, cohuck, virtualization, bagasdotme, kernel
Basic documentation about virtio in the kernel and a small tutorial for
virtio drivers.
Tested on linux-next (next-20220802)
Changes in v4:
- reword the intro paragraphs
- grammar fixes
Changes in v3:
- fix commit message in patch 1
- minor additional fixes to virtio kerneldocs
- use proper Sphinx markup for links to references
Changes in v2:
- virtio spec links updated to v1.2
- simplify virtio.rst and remove most low level parts that are
already covered by the spec
- split the kerneldocs fixes to a separate patch
- remove :c:func: rst formatting
Ricardo Cañuelo (2):
virtio: kerneldocs fixes and enhancements
docs: driver-api: virtio: virtio on Linux
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/virtio/index.rst | 11 ++
Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
.../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
MAINTAINERS | 1 +
drivers/virtio/virtio_ring.c | 8 +
include/linux/virtio.h | 6 +-
include/linux/virtio_config.h | 6 +-
include/uapi/linux/virtio_ring.h | 16 +-
9 files changed, 368 insertions(+), 11 deletions(-)
create mode 100644 Documentation/driver-api/virtio/index.rst
create mode 100644 Documentation/driver-api/virtio/virtio.rst
create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
--
2.25.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v4 1/2] virtio: kerneldocs fixes and enhancements
2022-08-10 9:40 ` Ricardo Cañuelo
@ 2022-08-10 9:40 ` Ricardo Cañuelo
-1 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: virtualization, mst, jasowang, kernel, cohuck, bagasdotme
Fix variable names in some kerneldocs, naming in others.
Add kerneldocs for struct vring_desc and vring_interrupt.
Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
---
drivers/virtio/virtio_ring.c | 8 ++++++++
include/linux/virtio.h | 6 +++---
include/linux/virtio_config.h | 6 +++---
include/uapi/linux/virtio_ring.h | 16 +++++++++++-----
4 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index a5ec724c01d8..6e5daa19ca1b 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2147,6 +2147,14 @@ static inline bool more_used(const struct vring_virtqueue *vq)
return vq->packed_ring ? more_used_packed(vq) : more_used_split(vq);
}
+/**
+ * vring_interrupt - notify a virtqueue on an interrupt
+ * @irq: the IRQ number (ignored)
+ * @_vq: the struct virtqueue to notify
+ *
+ * Calls the callback function of @_vq to process the virtqueue
+ * notification.
+ */
irqreturn_t vring_interrupt(int irq, void *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index d8fdf170637c..fd8440e85933 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -11,7 +11,7 @@
#include <linux/gfp.h>
/**
- * virtqueue - a queue to register buffers for sending or receiving.
+ * struct virtqueue - a queue to register buffers for sending or receiving.
* @list: the chain of virtqueues for this device
* @callback: the function to call when buffers are consumed (can be NULL).
* @name: the name of this virtqueue (mainly for debugging)
@@ -90,7 +90,7 @@ dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
/**
- * virtio_device - representation of a device using virtio
+ * struct virtio_device - representation of a device using virtio
* @index: unique position on the virtio bus
* @failed: saved value for VIRTIO_CONFIG_S_FAILED bit (for restore)
* @config_enabled: configuration change reporting enabled
@@ -146,7 +146,7 @@ size_t virtio_max_dma_size(struct virtio_device *vdev);
list_for_each_entry(vq, &vdev->vqs, list)
/**
- * virtio_driver - operations for a virtio I/O driver
+ * struct virtio_driver - operations for a virtio I/O driver
* @driver: underlying device driver (populate name and owner).
* @id_table: the ids serviced by this driver.
* @feature_table: an array of feature numbers supported by this driver.
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index b47c2e7ed0ee..227a9ff5371e 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -225,7 +225,7 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
/**
* virtio_synchronize_cbs - synchronize with virtqueue callbacks
- * @vdev: the device
+ * @dev: the virtio device
*/
static inline
void virtio_synchronize_cbs(struct virtio_device *dev)
@@ -244,7 +244,7 @@ void virtio_synchronize_cbs(struct virtio_device *dev)
/**
* virtio_device_ready - enable vq use in probe function
- * @vdev: the device
+ * @dev: the virtio device
*
* Driver must call this to use vqs in the probe function.
*
@@ -292,7 +292,7 @@ const char *virtio_bus_name(struct virtio_device *vdev)
/**
* virtqueue_set_affinity - setting affinity for a virtqueue
* @vq: the virtqueue
- * @cpu: the cpu no.
+ * @cpu_mask: the cpu mask
*
* Pay attention the function are best-effort: the affinity hint may not be set
* due to config support, irq type and sharing.
diff --git a/include/uapi/linux/virtio_ring.h b/include/uapi/linux/virtio_ring.h
index 476d3e5c0fe7..f8c20d3de8da 100644
--- a/include/uapi/linux/virtio_ring.h
+++ b/include/uapi/linux/virtio_ring.h
@@ -93,15 +93,21 @@
#define VRING_USED_ALIGN_SIZE 4
#define VRING_DESC_ALIGN_SIZE 16
-/* Virtio ring descriptors: 16 bytes. These can chain together via "next". */
+/**
+ * struct vring_desc - Virtio ring descriptors,
+ * 16 bytes long. These can chain together via @next.
+ *
+ * @addr: buffer address (guest-physical)
+ * @len: buffer length
+ * @flags: descriptor flags
+ * @next: index of the next descriptor in the chain,
+ * if the VRING_DESC_F_NEXT flag is set. We chain unused
+ * descriptors via this, too.
+ */
struct vring_desc {
- /* Address (guest-physical). */
__virtio64 addr;
- /* Length. */
__virtio32 len;
- /* The flags as indicated above. */
__virtio16 flags;
- /* We chain unused descriptors via this, too */
__virtio16 next;
};
--
2.25.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 1/2] virtio: kerneldocs fixes and enhancements
@ 2022-08-10 9:40 ` Ricardo Cañuelo
0 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: mst, cohuck, virtualization, bagasdotme, kernel
Fix variable names in some kerneldocs, naming in others.
Add kerneldocs for struct vring_desc and vring_interrupt.
Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
---
drivers/virtio/virtio_ring.c | 8 ++++++++
include/linux/virtio.h | 6 +++---
include/linux/virtio_config.h | 6 +++---
include/uapi/linux/virtio_ring.h | 16 +++++++++++-----
4 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index a5ec724c01d8..6e5daa19ca1b 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -2147,6 +2147,14 @@ static inline bool more_used(const struct vring_virtqueue *vq)
return vq->packed_ring ? more_used_packed(vq) : more_used_split(vq);
}
+/**
+ * vring_interrupt - notify a virtqueue on an interrupt
+ * @irq: the IRQ number (ignored)
+ * @_vq: the struct virtqueue to notify
+ *
+ * Calls the callback function of @_vq to process the virtqueue
+ * notification.
+ */
irqreturn_t vring_interrupt(int irq, void *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index d8fdf170637c..fd8440e85933 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -11,7 +11,7 @@
#include <linux/gfp.h>
/**
- * virtqueue - a queue to register buffers for sending or receiving.
+ * struct virtqueue - a queue to register buffers for sending or receiving.
* @list: the chain of virtqueues for this device
* @callback: the function to call when buffers are consumed (can be NULL).
* @name: the name of this virtqueue (mainly for debugging)
@@ -90,7 +90,7 @@ dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
/**
- * virtio_device - representation of a device using virtio
+ * struct virtio_device - representation of a device using virtio
* @index: unique position on the virtio bus
* @failed: saved value for VIRTIO_CONFIG_S_FAILED bit (for restore)
* @config_enabled: configuration change reporting enabled
@@ -146,7 +146,7 @@ size_t virtio_max_dma_size(struct virtio_device *vdev);
list_for_each_entry(vq, &vdev->vqs, list)
/**
- * virtio_driver - operations for a virtio I/O driver
+ * struct virtio_driver - operations for a virtio I/O driver
* @driver: underlying device driver (populate name and owner).
* @id_table: the ids serviced by this driver.
* @feature_table: an array of feature numbers supported by this driver.
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index b47c2e7ed0ee..227a9ff5371e 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -225,7 +225,7 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs,
/**
* virtio_synchronize_cbs - synchronize with virtqueue callbacks
- * @vdev: the device
+ * @dev: the virtio device
*/
static inline
void virtio_synchronize_cbs(struct virtio_device *dev)
@@ -244,7 +244,7 @@ void virtio_synchronize_cbs(struct virtio_device *dev)
/**
* virtio_device_ready - enable vq use in probe function
- * @vdev: the device
+ * @dev: the virtio device
*
* Driver must call this to use vqs in the probe function.
*
@@ -292,7 +292,7 @@ const char *virtio_bus_name(struct virtio_device *vdev)
/**
* virtqueue_set_affinity - setting affinity for a virtqueue
* @vq: the virtqueue
- * @cpu: the cpu no.
+ * @cpu_mask: the cpu mask
*
* Pay attention the function are best-effort: the affinity hint may not be set
* due to config support, irq type and sharing.
diff --git a/include/uapi/linux/virtio_ring.h b/include/uapi/linux/virtio_ring.h
index 476d3e5c0fe7..f8c20d3de8da 100644
--- a/include/uapi/linux/virtio_ring.h
+++ b/include/uapi/linux/virtio_ring.h
@@ -93,15 +93,21 @@
#define VRING_USED_ALIGN_SIZE 4
#define VRING_DESC_ALIGN_SIZE 16
-/* Virtio ring descriptors: 16 bytes. These can chain together via "next". */
+/**
+ * struct vring_desc - Virtio ring descriptors,
+ * 16 bytes long. These can chain together via @next.
+ *
+ * @addr: buffer address (guest-physical)
+ * @len: buffer length
+ * @flags: descriptor flags
+ * @next: index of the next descriptor in the chain,
+ * if the VRING_DESC_F_NEXT flag is set. We chain unused
+ * descriptors via this, too.
+ */
struct vring_desc {
- /* Address (guest-physical). */
__virtio64 addr;
- /* Length. */
__virtio32 len;
- /* The flags as indicated above. */
__virtio16 flags;
- /* We chain unused descriptors via this, too */
__virtio16 next;
};
--
2.25.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
2022-08-10 9:40 ` Ricardo Cañuelo
@ 2022-08-10 9:40 ` Ricardo Cañuelo
-1 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: virtualization, mst, jasowang, kernel, cohuck, bagasdotme
Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
---
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/virtio/index.rst | 11 ++
Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
.../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
MAINTAINERS | 1 +
5 files changed, 343 insertions(+)
create mode 100644 Documentation/driver-api/virtio/index.rst
create mode 100644 Documentation/driver-api/virtio/virtio.rst
create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
index d3a58f77328e..30a3de452b1d 100644
--- a/Documentation/driver-api/index.rst
+++ b/Documentation/driver-api/index.rst
@@ -106,6 +106,7 @@ available subsections can be seen below.
vfio-mediated-device
vfio
vfio-pci-device-specific-driver-acceptance
+ virtio/index
xilinx/index
xillybus
zorro
diff --git a/Documentation/driver-api/virtio/index.rst b/Documentation/driver-api/virtio/index.rst
new file mode 100644
index 000000000000..528b14b291e3
--- /dev/null
+++ b/Documentation/driver-api/virtio/index.rst
@@ -0,0 +1,11 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======
+Virtio
+======
+
+.. toctree::
+ :maxdepth: 1
+
+ virtio
+ writing_virtio_drivers
diff --git a/Documentation/driver-api/virtio/virtio.rst b/Documentation/driver-api/virtio/virtio.rst
new file mode 100644
index 000000000000..921758fc3b23
--- /dev/null
+++ b/Documentation/driver-api/virtio/virtio.rst
@@ -0,0 +1,144 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _virtio:
+
+===============
+Virtio on Linux
+===============
+
+Introduction
+============
+
+Virtio is an open standard that defines a protocol for communication
+between drivers and devices of different types, see Chapter 5 ("Device
+Types") of the virtio spec `[1]`_. Originally developed as a standard
+for paravirtualized devices implemented by a hypervisor, it can be used
+to interface any compliant device (real or emulated) with a driver.
+
+For illustrative purposes, this document will focus on the common case
+of a Linux kernel running in a virtual machine and using paravirtualized
+devices provided by the hypervisor, which exposes them as virtio devices
+via standard mechanisms such as PCI.
+
+
+Device - Driver communication: virtqueues
+=========================================
+
+Although the virtio devices are really an abstraction layer in the
+hypervisor, they're exposed to the guest as if they are physical devices
+using a specific transport method -- PCI, MMIO or CCW -- that is
+orthogonal to the device itself. The virtio spec defines these transport
+methods in detail, including device discovery, capabilities and
+interrupt handling.
+
+The communication between the driver in the guest OS and the device in
+the hypervisor is done through shared memory (that's what makes virtio
+devices so efficient) using specialized data structures called
+virtqueues, which are actually ring buffers [#f1]_ of buffer descriptors
+similar to the ones used in a network device:
+
+.. kernel-doc:: include/uapi/linux/virtio_ring.h
+ :identifiers: struct vring_desc
+
+All the buffers the descriptors point to are allocated by the guest and
+used by the host either for reading or for writing but not for both.
+
+Refer to Chapter 2.5 ("Virtqueues") of the virtio spec `[1]`_ for the
+reference definitions of virtqueues and to `[2]`_ for an illustrated
+overview of how the host device and the guest driver communicate.
+
+The :c:type:`vring_virtqueue` struct models a virtqueue, including the
+ring buffers and management data. Embedded in this struct is the
+:c:type:`virtqueue` struct, which is the data structure that's
+ultimately used by virtio drivers:
+
+.. kernel-doc:: include/linux/virtio.h
+ :identifiers: struct virtqueue
+
+The callback function pointed by this struct is triggered when the
+device has consumed the buffers provided by the driver. More
+specifically, the trigger will be an interrupt issued by the hypervisor
+(see vring_interrupt()). Interrupt request handlers are registered for
+a virtqueue during the virtqueue setup process (transport-specific).
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: vring_interrupt
+
+
+Device discovery and probing
+============================
+
+In the kernel, the virtio core contains the virtio bus driver and
+transport-specific drivers like `virtio-pci` and `virtio-mmio`. Then
+there are individual virtio drivers for specific device types that are
+registered to the virtio bus driver.
+
+How a virtio device is found and configured by the kernel depends on how
+the hypervisor defines it. Taking the `QEMU virtio-console
+<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__
+device as an example. When using PCI as a transport method, the device
+will present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.)
+and device id 0x1003 (virtio console), as defined in the spec, so the
+kernel will detect it as it would do with any other PCI device.
+
+During the PCI enumeration process, if a device is found to match the
+virtio-pci driver (according to the virtio-pci device table, any PCI
+device with vendor id = 0x1af4)::
+
+ /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */
+ static const struct pci_device_id virtio_pci_id_table[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) },
+ { 0 }
+ };
+
+then the virtio-pci driver is probed and, if the probing goes well, the
+device is registered to the virtio bus::
+
+ static int virtio_pci_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *id)
+ {
+ ...
+
+ if (force_legacy) {
+ rc = virtio_pci_legacy_probe(vp_dev);
+ /* Also try modern mode if we can't map BAR0 (no IO space). */
+ if (rc == -ENODEV || rc == -ENOMEM)
+ rc = virtio_pci_modern_probe(vp_dev);
+ if (rc)
+ goto err_probe;
+ } else {
+ rc = virtio_pci_modern_probe(vp_dev);
+ if (rc == -ENODEV)
+ rc = virtio_pci_legacy_probe(vp_dev);
+ if (rc)
+ goto err_probe;
+ }
+
+ ...
+
+ rc = register_virtio_device(&vp_dev->vdev);
+
+When the device is registered to the virtio bus the kernel will look
+for a driver in the bus that can handle the device and call that
+driver's ``probe`` method.
+
+It's at this stage that the virtqueues will be allocated and configured
+by calling the appropriate ``virtio_find`` helper function, such as
+virtio_find_single_vq() or virtio_find_vqs(), which will end up
+calling a transport-specific ``find_vqs`` method.
+
+
+References
+==========
+
+_`[1]` Virtio Spec v1.2:
+https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
+
+Check for later versions of the spec as well.
+
+_`[2]` Virtqueues and virtio ring: How the data travels
+https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
+
+.. rubric:: Footnotes
+
+.. [#f1] that's why they may be also referred as virtrings.
diff --git a/Documentation/driver-api/virtio/writing_virtio_drivers.rst b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
new file mode 100644
index 000000000000..3e5e4648dd5e
--- /dev/null
+++ b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
@@ -0,0 +1,186 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _writing_virtio_drivers:
+
+======================
+Writing Virtio Drivers
+======================
+
+Introduction
+============
+
+This document serves as a basic guideline for driver programmers that
+need to hack a new virtio driver or understand the essentials of the
+existing ones. See :ref:`Virtio on Linux <virtio>` for a general
+overview of virtio.
+
+
+Driver boilerplate
+==================
+
+As a bare minimum, a virtio driver needs to register in the virtio bus
+and configure the virtqueues for the device according to its spec, the
+configuration of the virtqueues in the driver side must match the
+virtqueue definitions in the device. A basic driver skeleton could look
+like this::
+
+ #include <linux/virtio.h>
+ #include <linux/virtio_ids.h>
+ #include <linux/virtio_config.h>
+ #include <linux/module.h>
+
+ /* device private data (one per device) */
+ struct virtio_dummy_dev {
+ struct virtqueue *vq;
+ };
+
+ static void virtio_dummy_recv_cb(struct virtqueue *vq)
+ {
+ struct virtio_dummy_dev *dev = vq->vdev->priv;
+ char *buf;
+ unsigned int len;
+
+ buf = virtqueue_get_buf(dev->vq, &len);
+ /* spurious callback? */
+ if (!buf)
+ return;
+
+ /* Process the received data */
+ }
+
+ static int virtio_dummy_probe(struct virtio_device *vdev)
+ {
+ struct virtio_dummy_dev *dev = NULL;
+
+ /* initialize device data */
+ dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ /* the device has a single virtqueue */
+ dev->vq = virtio_find_single_vq(vdev, virtio_dummy_recv_cb, "input");
+ if (IS_ERR(dev->vq)) {
+ kfree(dev);
+ return PTR_ERR(dev->vq);
+
+ }
+ vdev->priv = dev;
+
+ return 0;
+ }
+
+ static void virtio_dummy_remove(struct virtio_device *vdev)
+ {
+ struct virtio_dummy_dev *dev = vdev->priv;
+
+ /*
+ * Disable vq interrupts: equivalent to
+ * vdev->config->reset(vdev)
+ */
+ virtio_reset_device(vdev);
+
+ /* remove virtqueues */
+ vdev->config->del_vqs(vdev);
+
+ kfree(dev);
+ }
+
+ static const struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_DUMMY, VIRTIO_DEV_ANY_ID },
+ { 0 },
+ };
+
+ static struct virtio_driver virtio_dummy_driver = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtio_dummy_probe,
+ .remove = virtio_dummy_remove,
+ };
+
+ module_virtio_driver(virtio_dummy_driver);
+ MODULE_DEVICE_TABLE(virtio, id_table);
+ MODULE_DESCRIPTION("Dummy virtio driver");
+ MODULE_LICENSE("GPL");
+
+The device id ``VIRTIO_ID_DUMMY`` here is a placeholder, virtio drivers
+should be added only for devices that are defined in the spec, see
+include/uapi/linux/virtio_ids.h. Device ids need to be at least reserved
+in the virtio spec before being added to that file.
+
+If your driver doesn't have to do anything special in its ``init`` and
+``exit`` methods, you can use the module_virtio_driver() helper to
+reduce the amount of boilerplate code.
+
+The ``probe`` method does the minimum driver setup in this case
+(memory allocation for the device data) and initializes the
+virtqueue. The virtqueues are automatically enabled after ``probe``
+returns, sending the appropriate "DRIVER_OK" status signal to the
+device. If the virtqueues need to be enabled before ``probe`` ends, they
+can be manually enabled by calling virtio_device_ready():
+
+.. kernel-doc:: include/linux/virtio_config.h
+ :identifiers: virtio_device_ready
+
+
+Sending and receiving data
+==========================
+
+The virtio_dummy_recv_cb() callback in the code above will be triggered
+when the device notifies the driver after it finishes processing a
+descriptor or descriptor chain, either for reading or writing. However,
+that's only the second half of the virtio device-driver communication
+process, as the communication is always started by the driver regardless
+of the direction of the data transfer.
+
+To configure a buffer transfer from the driver to the device, first you
+have to add the buffers -- packed as `scatterlists` -- to the
+appropriate virtqueue using any of the virtqueue_add_inbuf(),
+virtqueue_add_outbuf() or virtqueue_add_sgs(), depending on whether you
+need to add one input `scatterlist` (for the device to fill in), one
+output `scatterlist` (for the device to consume) or multiple
+`scatterlists`, respectively. Then, once the virtqueue is set up, a call
+to virtqueue_kick() sends a notification that will be serviced by the
+hypervisor that implements the device::
+
+ struct scatterlist sg[1];
+ sg_init_one(sg, buffer, BUFLEN);
+ virtqueue_add_inbuf(dev->vq, sg, 1, buffer, GFP_ATOMIC);
+ virtqueue_kick(dev->vq);
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_inbuf
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_outbuf
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_sgs
+
+Then, after the device has read or written the buffers prepared by the
+driver and notifies it back, the driver can call virtqueue_get_buf() to
+read the data produced by the device (if the virtqueue was set up with
+input buffers) or simply to reclaim the buffers if they were already
+consumed by the device:
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_get_buf_ctx
+
+The virtqueue callbacks can be disabled and re-enabled using the
+virtqueue_disable_cb() and the family of virtqueue_enable_cb() functions
+respectively. See drivers/virtio/virtio_ring.c for more details:
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_disable_cb
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_enable_cb
+
+
+References
+==========
+
+_`[1]` Virtio Spec v1.2:
+https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
+
+Check for later versions of the spec as well.
diff --git a/MAINTAINERS b/MAINTAINERS
index 779f599f9abf..6ecdddb89da4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21488,6 +21488,7 @@ S: Maintained
F: Documentation/ABI/testing/sysfs-bus-vdpa
F: Documentation/ABI/testing/sysfs-class-vduse
F: Documentation/devicetree/bindings/virtio/
+F: Documentation/driver-api/virtio/
F: drivers/block/virtio_blk.c
F: drivers/crypto/virtio/
F: drivers/net/virtio_net.c
--
2.25.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-10 9:40 ` Ricardo Cañuelo
0 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-10 9:40 UTC (permalink / raw)
To: linux-doc; +Cc: mst, cohuck, virtualization, bagasdotme, kernel
Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
---
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/virtio/index.rst | 11 ++
Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
.../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
MAINTAINERS | 1 +
5 files changed, 343 insertions(+)
create mode 100644 Documentation/driver-api/virtio/index.rst
create mode 100644 Documentation/driver-api/virtio/virtio.rst
create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
index d3a58f77328e..30a3de452b1d 100644
--- a/Documentation/driver-api/index.rst
+++ b/Documentation/driver-api/index.rst
@@ -106,6 +106,7 @@ available subsections can be seen below.
vfio-mediated-device
vfio
vfio-pci-device-specific-driver-acceptance
+ virtio/index
xilinx/index
xillybus
zorro
diff --git a/Documentation/driver-api/virtio/index.rst b/Documentation/driver-api/virtio/index.rst
new file mode 100644
index 000000000000..528b14b291e3
--- /dev/null
+++ b/Documentation/driver-api/virtio/index.rst
@@ -0,0 +1,11 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======
+Virtio
+======
+
+.. toctree::
+ :maxdepth: 1
+
+ virtio
+ writing_virtio_drivers
diff --git a/Documentation/driver-api/virtio/virtio.rst b/Documentation/driver-api/virtio/virtio.rst
new file mode 100644
index 000000000000..921758fc3b23
--- /dev/null
+++ b/Documentation/driver-api/virtio/virtio.rst
@@ -0,0 +1,144 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _virtio:
+
+===============
+Virtio on Linux
+===============
+
+Introduction
+============
+
+Virtio is an open standard that defines a protocol for communication
+between drivers and devices of different types, see Chapter 5 ("Device
+Types") of the virtio spec `[1]`_. Originally developed as a standard
+for paravirtualized devices implemented by a hypervisor, it can be used
+to interface any compliant device (real or emulated) with a driver.
+
+For illustrative purposes, this document will focus on the common case
+of a Linux kernel running in a virtual machine and using paravirtualized
+devices provided by the hypervisor, which exposes them as virtio devices
+via standard mechanisms such as PCI.
+
+
+Device - Driver communication: virtqueues
+=========================================
+
+Although the virtio devices are really an abstraction layer in the
+hypervisor, they're exposed to the guest as if they are physical devices
+using a specific transport method -- PCI, MMIO or CCW -- that is
+orthogonal to the device itself. The virtio spec defines these transport
+methods in detail, including device discovery, capabilities and
+interrupt handling.
+
+The communication between the driver in the guest OS and the device in
+the hypervisor is done through shared memory (that's what makes virtio
+devices so efficient) using specialized data structures called
+virtqueues, which are actually ring buffers [#f1]_ of buffer descriptors
+similar to the ones used in a network device:
+
+.. kernel-doc:: include/uapi/linux/virtio_ring.h
+ :identifiers: struct vring_desc
+
+All the buffers the descriptors point to are allocated by the guest and
+used by the host either for reading or for writing but not for both.
+
+Refer to Chapter 2.5 ("Virtqueues") of the virtio spec `[1]`_ for the
+reference definitions of virtqueues and to `[2]`_ for an illustrated
+overview of how the host device and the guest driver communicate.
+
+The :c:type:`vring_virtqueue` struct models a virtqueue, including the
+ring buffers and management data. Embedded in this struct is the
+:c:type:`virtqueue` struct, which is the data structure that's
+ultimately used by virtio drivers:
+
+.. kernel-doc:: include/linux/virtio.h
+ :identifiers: struct virtqueue
+
+The callback function pointed by this struct is triggered when the
+device has consumed the buffers provided by the driver. More
+specifically, the trigger will be an interrupt issued by the hypervisor
+(see vring_interrupt()). Interrupt request handlers are registered for
+a virtqueue during the virtqueue setup process (transport-specific).
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: vring_interrupt
+
+
+Device discovery and probing
+============================
+
+In the kernel, the virtio core contains the virtio bus driver and
+transport-specific drivers like `virtio-pci` and `virtio-mmio`. Then
+there are individual virtio drivers for specific device types that are
+registered to the virtio bus driver.
+
+How a virtio device is found and configured by the kernel depends on how
+the hypervisor defines it. Taking the `QEMU virtio-console
+<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__
+device as an example. When using PCI as a transport method, the device
+will present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.)
+and device id 0x1003 (virtio console), as defined in the spec, so the
+kernel will detect it as it would do with any other PCI device.
+
+During the PCI enumeration process, if a device is found to match the
+virtio-pci driver (according to the virtio-pci device table, any PCI
+device with vendor id = 0x1af4)::
+
+ /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */
+ static const struct pci_device_id virtio_pci_id_table[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) },
+ { 0 }
+ };
+
+then the virtio-pci driver is probed and, if the probing goes well, the
+device is registered to the virtio bus::
+
+ static int virtio_pci_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *id)
+ {
+ ...
+
+ if (force_legacy) {
+ rc = virtio_pci_legacy_probe(vp_dev);
+ /* Also try modern mode if we can't map BAR0 (no IO space). */
+ if (rc == -ENODEV || rc == -ENOMEM)
+ rc = virtio_pci_modern_probe(vp_dev);
+ if (rc)
+ goto err_probe;
+ } else {
+ rc = virtio_pci_modern_probe(vp_dev);
+ if (rc == -ENODEV)
+ rc = virtio_pci_legacy_probe(vp_dev);
+ if (rc)
+ goto err_probe;
+ }
+
+ ...
+
+ rc = register_virtio_device(&vp_dev->vdev);
+
+When the device is registered to the virtio bus the kernel will look
+for a driver in the bus that can handle the device and call that
+driver's ``probe`` method.
+
+It's at this stage that the virtqueues will be allocated and configured
+by calling the appropriate ``virtio_find`` helper function, such as
+virtio_find_single_vq() or virtio_find_vqs(), which will end up
+calling a transport-specific ``find_vqs`` method.
+
+
+References
+==========
+
+_`[1]` Virtio Spec v1.2:
+https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
+
+Check for later versions of the spec as well.
+
+_`[2]` Virtqueues and virtio ring: How the data travels
+https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
+
+.. rubric:: Footnotes
+
+.. [#f1] that's why they may be also referred as virtrings.
diff --git a/Documentation/driver-api/virtio/writing_virtio_drivers.rst b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
new file mode 100644
index 000000000000..3e5e4648dd5e
--- /dev/null
+++ b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
@@ -0,0 +1,186 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _writing_virtio_drivers:
+
+======================
+Writing Virtio Drivers
+======================
+
+Introduction
+============
+
+This document serves as a basic guideline for driver programmers that
+need to hack a new virtio driver or understand the essentials of the
+existing ones. See :ref:`Virtio on Linux <virtio>` for a general
+overview of virtio.
+
+
+Driver boilerplate
+==================
+
+As a bare minimum, a virtio driver needs to register in the virtio bus
+and configure the virtqueues for the device according to its spec, the
+configuration of the virtqueues in the driver side must match the
+virtqueue definitions in the device. A basic driver skeleton could look
+like this::
+
+ #include <linux/virtio.h>
+ #include <linux/virtio_ids.h>
+ #include <linux/virtio_config.h>
+ #include <linux/module.h>
+
+ /* device private data (one per device) */
+ struct virtio_dummy_dev {
+ struct virtqueue *vq;
+ };
+
+ static void virtio_dummy_recv_cb(struct virtqueue *vq)
+ {
+ struct virtio_dummy_dev *dev = vq->vdev->priv;
+ char *buf;
+ unsigned int len;
+
+ buf = virtqueue_get_buf(dev->vq, &len);
+ /* spurious callback? */
+ if (!buf)
+ return;
+
+ /* Process the received data */
+ }
+
+ static int virtio_dummy_probe(struct virtio_device *vdev)
+ {
+ struct virtio_dummy_dev *dev = NULL;
+
+ /* initialize device data */
+ dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ /* the device has a single virtqueue */
+ dev->vq = virtio_find_single_vq(vdev, virtio_dummy_recv_cb, "input");
+ if (IS_ERR(dev->vq)) {
+ kfree(dev);
+ return PTR_ERR(dev->vq);
+
+ }
+ vdev->priv = dev;
+
+ return 0;
+ }
+
+ static void virtio_dummy_remove(struct virtio_device *vdev)
+ {
+ struct virtio_dummy_dev *dev = vdev->priv;
+
+ /*
+ * Disable vq interrupts: equivalent to
+ * vdev->config->reset(vdev)
+ */
+ virtio_reset_device(vdev);
+
+ /* remove virtqueues */
+ vdev->config->del_vqs(vdev);
+
+ kfree(dev);
+ }
+
+ static const struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_DUMMY, VIRTIO_DEV_ANY_ID },
+ { 0 },
+ };
+
+ static struct virtio_driver virtio_dummy_driver = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtio_dummy_probe,
+ .remove = virtio_dummy_remove,
+ };
+
+ module_virtio_driver(virtio_dummy_driver);
+ MODULE_DEVICE_TABLE(virtio, id_table);
+ MODULE_DESCRIPTION("Dummy virtio driver");
+ MODULE_LICENSE("GPL");
+
+The device id ``VIRTIO_ID_DUMMY`` here is a placeholder, virtio drivers
+should be added only for devices that are defined in the spec, see
+include/uapi/linux/virtio_ids.h. Device ids need to be at least reserved
+in the virtio spec before being added to that file.
+
+If your driver doesn't have to do anything special in its ``init`` and
+``exit`` methods, you can use the module_virtio_driver() helper to
+reduce the amount of boilerplate code.
+
+The ``probe`` method does the minimum driver setup in this case
+(memory allocation for the device data) and initializes the
+virtqueue. The virtqueues are automatically enabled after ``probe``
+returns, sending the appropriate "DRIVER_OK" status signal to the
+device. If the virtqueues need to be enabled before ``probe`` ends, they
+can be manually enabled by calling virtio_device_ready():
+
+.. kernel-doc:: include/linux/virtio_config.h
+ :identifiers: virtio_device_ready
+
+
+Sending and receiving data
+==========================
+
+The virtio_dummy_recv_cb() callback in the code above will be triggered
+when the device notifies the driver after it finishes processing a
+descriptor or descriptor chain, either for reading or writing. However,
+that's only the second half of the virtio device-driver communication
+process, as the communication is always started by the driver regardless
+of the direction of the data transfer.
+
+To configure a buffer transfer from the driver to the device, first you
+have to add the buffers -- packed as `scatterlists` -- to the
+appropriate virtqueue using any of the virtqueue_add_inbuf(),
+virtqueue_add_outbuf() or virtqueue_add_sgs(), depending on whether you
+need to add one input `scatterlist` (for the device to fill in), one
+output `scatterlist` (for the device to consume) or multiple
+`scatterlists`, respectively. Then, once the virtqueue is set up, a call
+to virtqueue_kick() sends a notification that will be serviced by the
+hypervisor that implements the device::
+
+ struct scatterlist sg[1];
+ sg_init_one(sg, buffer, BUFLEN);
+ virtqueue_add_inbuf(dev->vq, sg, 1, buffer, GFP_ATOMIC);
+ virtqueue_kick(dev->vq);
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_inbuf
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_outbuf
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_add_sgs
+
+Then, after the device has read or written the buffers prepared by the
+driver and notifies it back, the driver can call virtqueue_get_buf() to
+read the data produced by the device (if the virtqueue was set up with
+input buffers) or simply to reclaim the buffers if they were already
+consumed by the device:
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_get_buf_ctx
+
+The virtqueue callbacks can be disabled and re-enabled using the
+virtqueue_disable_cb() and the family of virtqueue_enable_cb() functions
+respectively. See drivers/virtio/virtio_ring.c for more details:
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_disable_cb
+
+.. kernel-doc:: drivers/virtio/virtio_ring.c
+ :identifiers: virtqueue_enable_cb
+
+
+References
+==========
+
+_`[1]` Virtio Spec v1.2:
+https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
+
+Check for later versions of the spec as well.
diff --git a/MAINTAINERS b/MAINTAINERS
index 779f599f9abf..6ecdddb89da4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21488,6 +21488,7 @@ S: Maintained
F: Documentation/ABI/testing/sysfs-bus-vdpa
F: Documentation/ABI/testing/sysfs-class-vduse
F: Documentation/devicetree/bindings/virtio/
+F: Documentation/driver-api/virtio/
F: drivers/block/virtio_blk.c
F: drivers/crypto/virtio/
F: drivers/net/virtio_net.c
--
2.25.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v4 1/2] virtio: kerneldocs fixes and enhancements
2022-08-10 9:40 ` Ricardo Cañuelo
@ 2022-08-11 13:19 ` Cornelia Huck
-1 siblings, 0 replies; 16+ messages in thread
From: Cornelia Huck @ 2022-08-11 13:19 UTC (permalink / raw)
To: Ricardo Cañuelo, linux-doc
Cc: virtualization, mst, jasowang, kernel, bagasdotme
On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
> Fix variable names in some kerneldocs, naming in others.
> Add kerneldocs for struct vring_desc and vring_interrupt.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> ---
> drivers/virtio/virtio_ring.c | 8 ++++++++
> include/linux/virtio.h | 6 +++---
> include/linux/virtio_config.h | 6 +++---
> include/uapi/linux/virtio_ring.h | 16 +++++++++++-----
> 4 files changed, 25 insertions(+), 11 deletions(-)
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 1/2] virtio: kerneldocs fixes and enhancements
@ 2022-08-11 13:19 ` Cornelia Huck
0 siblings, 0 replies; 16+ messages in thread
From: Cornelia Huck @ 2022-08-11 13:19 UTC (permalink / raw)
To: Ricardo Cañuelo, linux-doc; +Cc: mst, kernel, bagasdotme, virtualization
On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
> Fix variable names in some kerneldocs, naming in others.
> Add kerneldocs for struct vring_desc and vring_interrupt.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> ---
> drivers/virtio/virtio_ring.c | 8 ++++++++
> include/linux/virtio.h | 6 +++---
> include/linux/virtio_config.h | 6 +++---
> include/uapi/linux/virtio_ring.h | 16 +++++++++++-----
> 4 files changed, 25 insertions(+), 11 deletions(-)
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
2022-08-10 9:40 ` Ricardo Cañuelo
@ 2022-08-11 13:32 ` Michael S. Tsirkin
-1 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 13:32 UTC (permalink / raw)
To: Ricardo Cañuelo
Cc: linux-doc, virtualization, jasowang, kernel, cohuck, bagasdotme
On Wed, Aug 10, 2022 at 11:40:04AM +0200, Ricardo Cañuelo wrote:
> Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
which tree is this for?
Applying: docs: driver-api: virtio: virtio on Linux
error: sha1 information is lacking or useless (MAINTAINERS).
error: could not build fake ancestor
Patch failed at 0001 docs: driver-api: virtio: virtio on Linux
> ---
> Documentation/driver-api/index.rst | 1 +
> Documentation/driver-api/virtio/index.rst | 11 ++
> Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> MAINTAINERS | 1 +
> 5 files changed, 343 insertions(+)
> create mode 100644 Documentation/driver-api/virtio/index.rst
> create mode 100644 Documentation/driver-api/virtio/virtio.rst
> create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
>
> diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
> index d3a58f77328e..30a3de452b1d 100644
> --- a/Documentation/driver-api/index.rst
> +++ b/Documentation/driver-api/index.rst
> @@ -106,6 +106,7 @@ available subsections can be seen below.
> vfio-mediated-device
> vfio
> vfio-pci-device-specific-driver-acceptance
> + virtio/index
> xilinx/index
> xillybus
> zorro
> diff --git a/Documentation/driver-api/virtio/index.rst b/Documentation/driver-api/virtio/index.rst
> new file mode 100644
> index 000000000000..528b14b291e3
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/index.rst
> @@ -0,0 +1,11 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======
> +Virtio
> +======
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + virtio
> + writing_virtio_drivers
> diff --git a/Documentation/driver-api/virtio/virtio.rst b/Documentation/driver-api/virtio/virtio.rst
> new file mode 100644
> index 000000000000..921758fc3b23
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/virtio.rst
> @@ -0,0 +1,144 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _virtio:
> +
> +===============
> +Virtio on Linux
> +===============
> +
> +Introduction
> +============
> +
> +Virtio is an open standard that defines a protocol for communication
> +between drivers and devices of different types, see Chapter 5 ("Device
> +Types") of the virtio spec `[1]`_. Originally developed as a standard
> +for paravirtualized devices implemented by a hypervisor, it can be used
> +to interface any compliant device (real or emulated) with a driver.
> +
> +For illustrative purposes, this document will focus on the common case
> +of a Linux kernel running in a virtual machine and using paravirtualized
> +devices provided by the hypervisor, which exposes them as virtio devices
> +via standard mechanisms such as PCI.
> +
> +
> +Device - Driver communication: virtqueues
> +=========================================
> +
> +Although the virtio devices are really an abstraction layer in the
> +hypervisor, they're exposed to the guest as if they are physical devices
> +using a specific transport method -- PCI, MMIO or CCW -- that is
> +orthogonal to the device itself. The virtio spec defines these transport
> +methods in detail, including device discovery, capabilities and
> +interrupt handling.
> +
> +The communication between the driver in the guest OS and the device in
> +the hypervisor is done through shared memory (that's what makes virtio
> +devices so efficient) using specialized data structures called
> +virtqueues, which are actually ring buffers [#f1]_ of buffer descriptors
> +similar to the ones used in a network device:
> +
> +.. kernel-doc:: include/uapi/linux/virtio_ring.h
> + :identifiers: struct vring_desc
> +
> +All the buffers the descriptors point to are allocated by the guest and
> +used by the host either for reading or for writing but not for both.
> +
> +Refer to Chapter 2.5 ("Virtqueues") of the virtio spec `[1]`_ for the
> +reference definitions of virtqueues and to `[2]`_ for an illustrated
> +overview of how the host device and the guest driver communicate.
> +
> +The :c:type:`vring_virtqueue` struct models a virtqueue, including the
> +ring buffers and management data. Embedded in this struct is the
> +:c:type:`virtqueue` struct, which is the data structure that's
> +ultimately used by virtio drivers:
> +
> +.. kernel-doc:: include/linux/virtio.h
> + :identifiers: struct virtqueue
> +
> +The callback function pointed by this struct is triggered when the
> +device has consumed the buffers provided by the driver. More
> +specifically, the trigger will be an interrupt issued by the hypervisor
> +(see vring_interrupt()). Interrupt request handlers are registered for
> +a virtqueue during the virtqueue setup process (transport-specific).
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: vring_interrupt
> +
> +
> +Device discovery and probing
> +============================
> +
> +In the kernel, the virtio core contains the virtio bus driver and
> +transport-specific drivers like `virtio-pci` and `virtio-mmio`. Then
> +there are individual virtio drivers for specific device types that are
> +registered to the virtio bus driver.
> +
> +How a virtio device is found and configured by the kernel depends on how
> +the hypervisor defines it. Taking the `QEMU virtio-console
> +<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__
> +device as an example. When using PCI as a transport method, the device
> +will present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.)
> +and device id 0x1003 (virtio console), as defined in the spec, so the
> +kernel will detect it as it would do with any other PCI device.
> +
> +During the PCI enumeration process, if a device is found to match the
> +virtio-pci driver (according to the virtio-pci device table, any PCI
> +device with vendor id = 0x1af4)::
> +
> + /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */
> + static const struct pci_device_id virtio_pci_id_table[] = {
> + { PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) },
> + { 0 }
> + };
> +
> +then the virtio-pci driver is probed and, if the probing goes well, the
> +device is registered to the virtio bus::
> +
> + static int virtio_pci_probe(struct pci_dev *pci_dev,
> + const struct pci_device_id *id)
> + {
> + ...
> +
> + if (force_legacy) {
> + rc = virtio_pci_legacy_probe(vp_dev);
> + /* Also try modern mode if we can't map BAR0 (no IO space). */
> + if (rc == -ENODEV || rc == -ENOMEM)
> + rc = virtio_pci_modern_probe(vp_dev);
> + if (rc)
> + goto err_probe;
> + } else {
> + rc = virtio_pci_modern_probe(vp_dev);
> + if (rc == -ENODEV)
> + rc = virtio_pci_legacy_probe(vp_dev);
> + if (rc)
> + goto err_probe;
> + }
> +
> + ...
> +
> + rc = register_virtio_device(&vp_dev->vdev);
> +
> +When the device is registered to the virtio bus the kernel will look
> +for a driver in the bus that can handle the device and call that
> +driver's ``probe`` method.
> +
> +It's at this stage that the virtqueues will be allocated and configured
> +by calling the appropriate ``virtio_find`` helper function, such as
> +virtio_find_single_vq() or virtio_find_vqs(), which will end up
> +calling a transport-specific ``find_vqs`` method.
> +
> +
> +References
> +==========
> +
> +_`[1]` Virtio Spec v1.2:
> +https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
> +
> +Check for later versions of the spec as well.
> +
> +_`[2]` Virtqueues and virtio ring: How the data travels
> +https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
> +
> +.. rubric:: Footnotes
> +
> +.. [#f1] that's why they may be also referred as virtrings.
> diff --git a/Documentation/driver-api/virtio/writing_virtio_drivers.rst b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
> new file mode 100644
> index 000000000000..3e5e4648dd5e
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
> @@ -0,0 +1,186 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _writing_virtio_drivers:
> +
> +======================
> +Writing Virtio Drivers
> +======================
> +
> +Introduction
> +============
> +
> +This document serves as a basic guideline for driver programmers that
> +need to hack a new virtio driver or understand the essentials of the
> +existing ones. See :ref:`Virtio on Linux <virtio>` for a general
> +overview of virtio.
> +
> +
> +Driver boilerplate
> +==================
> +
> +As a bare minimum, a virtio driver needs to register in the virtio bus
> +and configure the virtqueues for the device according to its spec, the
> +configuration of the virtqueues in the driver side must match the
> +virtqueue definitions in the device. A basic driver skeleton could look
> +like this::
> +
> + #include <linux/virtio.h>
> + #include <linux/virtio_ids.h>
> + #include <linux/virtio_config.h>
> + #include <linux/module.h>
> +
> + /* device private data (one per device) */
> + struct virtio_dummy_dev {
> + struct virtqueue *vq;
> + };
> +
> + static void virtio_dummy_recv_cb(struct virtqueue *vq)
> + {
> + struct virtio_dummy_dev *dev = vq->vdev->priv;
> + char *buf;
> + unsigned int len;
> +
> + buf = virtqueue_get_buf(dev->vq, &len);
> + /* spurious callback? */
> + if (!buf)
> + return;
most drivers need to do this in a loop, this code is only valid if
there's just 1 buf in flight - unusual.
> +
> + /* Process the received data */
> + }
> +
> + static int virtio_dummy_probe(struct virtio_device *vdev)
> + {
> + struct virtio_dummy_dev *dev = NULL;
> +
> + /* initialize device data */
> + dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
I dislike how we set dev to NULL and immediately to a different value
just below.
> + if (!dev)
> + return -ENOMEM;
> +
> + /* the device has a single virtqueue */
> + dev->vq = virtio_find_single_vq(vdev, virtio_dummy_recv_cb, "input");
> + if (IS_ERR(dev->vq)) {
> + kfree(dev);
> + return PTR_ERR(dev->vq);
> +
> + }
> + vdev->priv = dev;
> +
> + return 0;
what is missing here is registration with Linux core.
depending on device you might need a call to device_ready, too.
> + }
> +
> + static void virtio_dummy_remove(struct virtio_device *vdev)
> + {
> + struct virtio_dummy_dev *dev = vdev->priv;
> +
> + /*
> + * Disable vq interrupts: equivalent to
> + * vdev->config->reset(vdev)
> + */
> + virtio_reset_device(vdev);
> +
you highly likely need to detach unused buffers from vqs here.
> + /* remove virtqueues */
> + vdev->config->del_vqs(vdev);
> +
> + kfree(dev);
> + }
> +
> + static const struct virtio_device_id id_table[] = {
> + { VIRTIO_ID_DUMMY, VIRTIO_DEV_ANY_ID },
> + { 0 },
> + };
> +
> + static struct virtio_driver virtio_dummy_driver = {
> + .driver.name = KBUILD_MODNAME,
> + .driver.owner = THIS_MODULE,
> + .id_table = id_table,
> + .probe = virtio_dummy_probe,
> + .remove = virtio_dummy_remove,
> + };
> +
> + module_virtio_driver(virtio_dummy_driver);
> + MODULE_DEVICE_TABLE(virtio, id_table);
> + MODULE_DESCRIPTION("Dummy virtio driver");
> + MODULE_LICENSE("GPL");
> +
> +The device id ``VIRTIO_ID_DUMMY`` here is a placeholder, virtio drivers
> +should be added only for devices that are defined in the spec, see
> +include/uapi/linux/virtio_ids.h. Device ids need to be at least reserved
> +in the virtio spec before being added to that file.
> +
> +If your driver doesn't have to do anything special in its ``init`` and
> +``exit`` methods, you can use the module_virtio_driver() helper to
> +reduce the amount of boilerplate code.
> +
> +The ``probe`` method does the minimum driver setup in this case
> +(memory allocation for the device data) and initializes the
> +virtqueue. The virtqueues are automatically enabled after ``probe``
> +returns, sending the appropriate "DRIVER_OK" status signal to the
> +device. If the virtqueues need to be enabled before ``probe`` ends, they
> +can be manually enabled by calling virtio_device_ready():
let's be a bit clearer here that they must be enabled before add_buf
triggers.
> +
> +.. kernel-doc:: include/linux/virtio_config.h
> + :identifiers: virtio_device_ready
> +
> +
> +Sending and receiving data
> +==========================
> +
> +The virtio_dummy_recv_cb() callback in the code above will be triggered
> +when the device notifies the driver after it finishes processing a
> +descriptor or descriptor chain, either for reading or writing. However,
> +that's only the second half of the virtio device-driver communication
> +process, as the communication is always started by the driver regardless
> +of the direction of the data transfer.
> +
> +To configure a buffer transfer from the driver to the device, first you
> +have to add the buffers -- packed as `scatterlists` -- to the
> +appropriate virtqueue using any of the virtqueue_add_inbuf(),
> +virtqueue_add_outbuf() or virtqueue_add_sgs(), depending on whether you
> +need to add one input `scatterlist` (for the device to fill in), one
> +output `scatterlist` (for the device to consume) or multiple
> +`scatterlists`, respectively. Then, once the virtqueue is set up, a call
> +to virtqueue_kick() sends a notification that will be serviced by the
> +hypervisor that implements the device::
> +
> + struct scatterlist sg[1];
> + sg_init_one(sg, buffer, BUFLEN);
> + virtqueue_add_inbuf(dev->vq, sg, 1, buffer, GFP_ATOMIC);
> + virtqueue_kick(dev->vq);
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_inbuf
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_outbuf
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_sgs
> +
> +Then, after the device has read or written the buffers prepared by the
> +driver and notifies it back, the driver can call virtqueue_get_buf() to
> +read the data produced by the device (if the virtqueue was set up with
> +input buffers) or simply to reclaim the buffers if they were already
> +consumed by the device:
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_get_buf_ctx
> +
> +The virtqueue callbacks can be disabled and re-enabled using the
> +virtqueue_disable_cb() and the family of virtqueue_enable_cb() functions
> +respectively. See drivers/virtio/virtio_ring.c for more details:
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_disable_cb
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_enable_cb
> +
maybe clarify that they can still trigger even if enabled.
if you want to disable reliable you have to reset the device
or the vq.
> +References
> +==========
> +
> +_`[1]` Virtio Spec v1.2:
> +https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
> +
> +Check for later versions of the spec as well.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 779f599f9abf..6ecdddb89da4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -21488,6 +21488,7 @@ S: Maintained
> F: Documentation/ABI/testing/sysfs-bus-vdpa
> F: Documentation/ABI/testing/sysfs-class-vduse
> F: Documentation/devicetree/bindings/virtio/
> +F: Documentation/driver-api/virtio/
> F: drivers/block/virtio_blk.c
> F: drivers/crypto/virtio/
> F: drivers/net/virtio_net.c
> --
> 2.25.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-11 13:32 ` Michael S. Tsirkin
0 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 13:32 UTC (permalink / raw)
To: Ricardo Cañuelo
Cc: linux-doc, cohuck, virtualization, bagasdotme, kernel
On Wed, Aug 10, 2022 at 11:40:04AM +0200, Ricardo Cañuelo wrote:
> Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
which tree is this for?
Applying: docs: driver-api: virtio: virtio on Linux
error: sha1 information is lacking or useless (MAINTAINERS).
error: could not build fake ancestor
Patch failed at 0001 docs: driver-api: virtio: virtio on Linux
> ---
> Documentation/driver-api/index.rst | 1 +
> Documentation/driver-api/virtio/index.rst | 11 ++
> Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> MAINTAINERS | 1 +
> 5 files changed, 343 insertions(+)
> create mode 100644 Documentation/driver-api/virtio/index.rst
> create mode 100644 Documentation/driver-api/virtio/virtio.rst
> create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
>
> diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
> index d3a58f77328e..30a3de452b1d 100644
> --- a/Documentation/driver-api/index.rst
> +++ b/Documentation/driver-api/index.rst
> @@ -106,6 +106,7 @@ available subsections can be seen below.
> vfio-mediated-device
> vfio
> vfio-pci-device-specific-driver-acceptance
> + virtio/index
> xilinx/index
> xillybus
> zorro
> diff --git a/Documentation/driver-api/virtio/index.rst b/Documentation/driver-api/virtio/index.rst
> new file mode 100644
> index 000000000000..528b14b291e3
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/index.rst
> @@ -0,0 +1,11 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======
> +Virtio
> +======
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + virtio
> + writing_virtio_drivers
> diff --git a/Documentation/driver-api/virtio/virtio.rst b/Documentation/driver-api/virtio/virtio.rst
> new file mode 100644
> index 000000000000..921758fc3b23
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/virtio.rst
> @@ -0,0 +1,144 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _virtio:
> +
> +===============
> +Virtio on Linux
> +===============
> +
> +Introduction
> +============
> +
> +Virtio is an open standard that defines a protocol for communication
> +between drivers and devices of different types, see Chapter 5 ("Device
> +Types") of the virtio spec `[1]`_. Originally developed as a standard
> +for paravirtualized devices implemented by a hypervisor, it can be used
> +to interface any compliant device (real or emulated) with a driver.
> +
> +For illustrative purposes, this document will focus on the common case
> +of a Linux kernel running in a virtual machine and using paravirtualized
> +devices provided by the hypervisor, which exposes them as virtio devices
> +via standard mechanisms such as PCI.
> +
> +
> +Device - Driver communication: virtqueues
> +=========================================
> +
> +Although the virtio devices are really an abstraction layer in the
> +hypervisor, they're exposed to the guest as if they are physical devices
> +using a specific transport method -- PCI, MMIO or CCW -- that is
> +orthogonal to the device itself. The virtio spec defines these transport
> +methods in detail, including device discovery, capabilities and
> +interrupt handling.
> +
> +The communication between the driver in the guest OS and the device in
> +the hypervisor is done through shared memory (that's what makes virtio
> +devices so efficient) using specialized data structures called
> +virtqueues, which are actually ring buffers [#f1]_ of buffer descriptors
> +similar to the ones used in a network device:
> +
> +.. kernel-doc:: include/uapi/linux/virtio_ring.h
> + :identifiers: struct vring_desc
> +
> +All the buffers the descriptors point to are allocated by the guest and
> +used by the host either for reading or for writing but not for both.
> +
> +Refer to Chapter 2.5 ("Virtqueues") of the virtio spec `[1]`_ for the
> +reference definitions of virtqueues and to `[2]`_ for an illustrated
> +overview of how the host device and the guest driver communicate.
> +
> +The :c:type:`vring_virtqueue` struct models a virtqueue, including the
> +ring buffers and management data. Embedded in this struct is the
> +:c:type:`virtqueue` struct, which is the data structure that's
> +ultimately used by virtio drivers:
> +
> +.. kernel-doc:: include/linux/virtio.h
> + :identifiers: struct virtqueue
> +
> +The callback function pointed by this struct is triggered when the
> +device has consumed the buffers provided by the driver. More
> +specifically, the trigger will be an interrupt issued by the hypervisor
> +(see vring_interrupt()). Interrupt request handlers are registered for
> +a virtqueue during the virtqueue setup process (transport-specific).
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: vring_interrupt
> +
> +
> +Device discovery and probing
> +============================
> +
> +In the kernel, the virtio core contains the virtio bus driver and
> +transport-specific drivers like `virtio-pci` and `virtio-mmio`. Then
> +there are individual virtio drivers for specific device types that are
> +registered to the virtio bus driver.
> +
> +How a virtio device is found and configured by the kernel depends on how
> +the hypervisor defines it. Taking the `QEMU virtio-console
> +<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__
> +device as an example. When using PCI as a transport method, the device
> +will present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.)
> +and device id 0x1003 (virtio console), as defined in the spec, so the
> +kernel will detect it as it would do with any other PCI device.
> +
> +During the PCI enumeration process, if a device is found to match the
> +virtio-pci driver (according to the virtio-pci device table, any PCI
> +device with vendor id = 0x1af4)::
> +
> + /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */
> + static const struct pci_device_id virtio_pci_id_table[] = {
> + { PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) },
> + { 0 }
> + };
> +
> +then the virtio-pci driver is probed and, if the probing goes well, the
> +device is registered to the virtio bus::
> +
> + static int virtio_pci_probe(struct pci_dev *pci_dev,
> + const struct pci_device_id *id)
> + {
> + ...
> +
> + if (force_legacy) {
> + rc = virtio_pci_legacy_probe(vp_dev);
> + /* Also try modern mode if we can't map BAR0 (no IO space). */
> + if (rc == -ENODEV || rc == -ENOMEM)
> + rc = virtio_pci_modern_probe(vp_dev);
> + if (rc)
> + goto err_probe;
> + } else {
> + rc = virtio_pci_modern_probe(vp_dev);
> + if (rc == -ENODEV)
> + rc = virtio_pci_legacy_probe(vp_dev);
> + if (rc)
> + goto err_probe;
> + }
> +
> + ...
> +
> + rc = register_virtio_device(&vp_dev->vdev);
> +
> +When the device is registered to the virtio bus the kernel will look
> +for a driver in the bus that can handle the device and call that
> +driver's ``probe`` method.
> +
> +It's at this stage that the virtqueues will be allocated and configured
> +by calling the appropriate ``virtio_find`` helper function, such as
> +virtio_find_single_vq() or virtio_find_vqs(), which will end up
> +calling a transport-specific ``find_vqs`` method.
> +
> +
> +References
> +==========
> +
> +_`[1]` Virtio Spec v1.2:
> +https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
> +
> +Check for later versions of the spec as well.
> +
> +_`[2]` Virtqueues and virtio ring: How the data travels
> +https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels
> +
> +.. rubric:: Footnotes
> +
> +.. [#f1] that's why they may be also referred as virtrings.
> diff --git a/Documentation/driver-api/virtio/writing_virtio_drivers.rst b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
> new file mode 100644
> index 000000000000..3e5e4648dd5e
> --- /dev/null
> +++ b/Documentation/driver-api/virtio/writing_virtio_drivers.rst
> @@ -0,0 +1,186 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _writing_virtio_drivers:
> +
> +======================
> +Writing Virtio Drivers
> +======================
> +
> +Introduction
> +============
> +
> +This document serves as a basic guideline for driver programmers that
> +need to hack a new virtio driver or understand the essentials of the
> +existing ones. See :ref:`Virtio on Linux <virtio>` for a general
> +overview of virtio.
> +
> +
> +Driver boilerplate
> +==================
> +
> +As a bare minimum, a virtio driver needs to register in the virtio bus
> +and configure the virtqueues for the device according to its spec, the
> +configuration of the virtqueues in the driver side must match the
> +virtqueue definitions in the device. A basic driver skeleton could look
> +like this::
> +
> + #include <linux/virtio.h>
> + #include <linux/virtio_ids.h>
> + #include <linux/virtio_config.h>
> + #include <linux/module.h>
> +
> + /* device private data (one per device) */
> + struct virtio_dummy_dev {
> + struct virtqueue *vq;
> + };
> +
> + static void virtio_dummy_recv_cb(struct virtqueue *vq)
> + {
> + struct virtio_dummy_dev *dev = vq->vdev->priv;
> + char *buf;
> + unsigned int len;
> +
> + buf = virtqueue_get_buf(dev->vq, &len);
> + /* spurious callback? */
> + if (!buf)
> + return;
most drivers need to do this in a loop, this code is only valid if
there's just 1 buf in flight - unusual.
> +
> + /* Process the received data */
> + }
> +
> + static int virtio_dummy_probe(struct virtio_device *vdev)
> + {
> + struct virtio_dummy_dev *dev = NULL;
> +
> + /* initialize device data */
> + dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
I dislike how we set dev to NULL and immediately to a different value
just below.
> + if (!dev)
> + return -ENOMEM;
> +
> + /* the device has a single virtqueue */
> + dev->vq = virtio_find_single_vq(vdev, virtio_dummy_recv_cb, "input");
> + if (IS_ERR(dev->vq)) {
> + kfree(dev);
> + return PTR_ERR(dev->vq);
> +
> + }
> + vdev->priv = dev;
> +
> + return 0;
what is missing here is registration with Linux core.
depending on device you might need a call to device_ready, too.
> + }
> +
> + static void virtio_dummy_remove(struct virtio_device *vdev)
> + {
> + struct virtio_dummy_dev *dev = vdev->priv;
> +
> + /*
> + * Disable vq interrupts: equivalent to
> + * vdev->config->reset(vdev)
> + */
> + virtio_reset_device(vdev);
> +
you highly likely need to detach unused buffers from vqs here.
> + /* remove virtqueues */
> + vdev->config->del_vqs(vdev);
> +
> + kfree(dev);
> + }
> +
> + static const struct virtio_device_id id_table[] = {
> + { VIRTIO_ID_DUMMY, VIRTIO_DEV_ANY_ID },
> + { 0 },
> + };
> +
> + static struct virtio_driver virtio_dummy_driver = {
> + .driver.name = KBUILD_MODNAME,
> + .driver.owner = THIS_MODULE,
> + .id_table = id_table,
> + .probe = virtio_dummy_probe,
> + .remove = virtio_dummy_remove,
> + };
> +
> + module_virtio_driver(virtio_dummy_driver);
> + MODULE_DEVICE_TABLE(virtio, id_table);
> + MODULE_DESCRIPTION("Dummy virtio driver");
> + MODULE_LICENSE("GPL");
> +
> +The device id ``VIRTIO_ID_DUMMY`` here is a placeholder, virtio drivers
> +should be added only for devices that are defined in the spec, see
> +include/uapi/linux/virtio_ids.h. Device ids need to be at least reserved
> +in the virtio spec before being added to that file.
> +
> +If your driver doesn't have to do anything special in its ``init`` and
> +``exit`` methods, you can use the module_virtio_driver() helper to
> +reduce the amount of boilerplate code.
> +
> +The ``probe`` method does the minimum driver setup in this case
> +(memory allocation for the device data) and initializes the
> +virtqueue. The virtqueues are automatically enabled after ``probe``
> +returns, sending the appropriate "DRIVER_OK" status signal to the
> +device. If the virtqueues need to be enabled before ``probe`` ends, they
> +can be manually enabled by calling virtio_device_ready():
let's be a bit clearer here that they must be enabled before add_buf
triggers.
> +
> +.. kernel-doc:: include/linux/virtio_config.h
> + :identifiers: virtio_device_ready
> +
> +
> +Sending and receiving data
> +==========================
> +
> +The virtio_dummy_recv_cb() callback in the code above will be triggered
> +when the device notifies the driver after it finishes processing a
> +descriptor or descriptor chain, either for reading or writing. However,
> +that's only the second half of the virtio device-driver communication
> +process, as the communication is always started by the driver regardless
> +of the direction of the data transfer.
> +
> +To configure a buffer transfer from the driver to the device, first you
> +have to add the buffers -- packed as `scatterlists` -- to the
> +appropriate virtqueue using any of the virtqueue_add_inbuf(),
> +virtqueue_add_outbuf() or virtqueue_add_sgs(), depending on whether you
> +need to add one input `scatterlist` (for the device to fill in), one
> +output `scatterlist` (for the device to consume) or multiple
> +`scatterlists`, respectively. Then, once the virtqueue is set up, a call
> +to virtqueue_kick() sends a notification that will be serviced by the
> +hypervisor that implements the device::
> +
> + struct scatterlist sg[1];
> + sg_init_one(sg, buffer, BUFLEN);
> + virtqueue_add_inbuf(dev->vq, sg, 1, buffer, GFP_ATOMIC);
> + virtqueue_kick(dev->vq);
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_inbuf
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_outbuf
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_add_sgs
> +
> +Then, after the device has read or written the buffers prepared by the
> +driver and notifies it back, the driver can call virtqueue_get_buf() to
> +read the data produced by the device (if the virtqueue was set up with
> +input buffers) or simply to reclaim the buffers if they were already
> +consumed by the device:
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_get_buf_ctx
> +
> +The virtqueue callbacks can be disabled and re-enabled using the
> +virtqueue_disable_cb() and the family of virtqueue_enable_cb() functions
> +respectively. See drivers/virtio/virtio_ring.c for more details:
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_disable_cb
> +
> +.. kernel-doc:: drivers/virtio/virtio_ring.c
> + :identifiers: virtqueue_enable_cb
> +
maybe clarify that they can still trigger even if enabled.
if you want to disable reliable you have to reset the device
or the vq.
> +References
> +==========
> +
> +_`[1]` Virtio Spec v1.2:
> +https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html
> +
> +Check for later versions of the spec as well.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 779f599f9abf..6ecdddb89da4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -21488,6 +21488,7 @@ S: Maintained
> F: Documentation/ABI/testing/sysfs-bus-vdpa
> F: Documentation/ABI/testing/sysfs-class-vduse
> F: Documentation/devicetree/bindings/virtio/
> +F: Documentation/driver-api/virtio/
> F: drivers/block/virtio_blk.c
> F: drivers/crypto/virtio/
> F: drivers/net/virtio_net.c
> --
> 2.25.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
2022-08-10 9:40 ` Ricardo Cañuelo
@ 2022-08-11 13:46 ` Cornelia Huck
-1 siblings, 0 replies; 16+ messages in thread
From: Cornelia Huck @ 2022-08-11 13:46 UTC (permalink / raw)
To: Ricardo Cañuelo, linux-doc
Cc: virtualization, mst, jasowang, kernel, bagasdotme
On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
> Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> ---
> Documentation/driver-api/index.rst | 1 +
> Documentation/driver-api/virtio/index.rst | 11 ++
> Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> MAINTAINERS | 1 +
> 5 files changed, 343 insertions(+)
> create mode 100644 Documentation/driver-api/virtio/index.rst
> create mode 100644 Documentation/driver-api/virtio/virtio.rst
> create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
>
(...)
> +.. rubric:: Footnotes
> +
> +.. [#f1] that's why they may be also referred as virtrings.
"referred to"
(...)
> +The ``probe`` method does the minimum driver setup in this case
> +(memory allocation for the device data) and initializes the
> +virtqueue. The virtqueues are automatically enabled after ``probe``
> +returns, sending the appropriate "DRIVER_OK" status signal to the
> +device. If the virtqueues need to be enabled before ``probe`` ends, they
> +can be manually enabled by calling virtio_device_ready():
> +
> +.. kernel-doc:: include/linux/virtio_config.h
> + :identifiers: virtio_device_ready
Hm, not quite sure what the actual expectations are here: Should the
driver set DRIVER_OK in its probe function, and the core only set it as
a fallback? Michael, Jason?
(...)
LGTM in general.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-11 13:46 ` Cornelia Huck
0 siblings, 0 replies; 16+ messages in thread
From: Cornelia Huck @ 2022-08-11 13:46 UTC (permalink / raw)
To: Ricardo Cañuelo, linux-doc; +Cc: mst, kernel, bagasdotme, virtualization
On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
> Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
>
> Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> ---
> Documentation/driver-api/index.rst | 1 +
> Documentation/driver-api/virtio/index.rst | 11 ++
> Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> MAINTAINERS | 1 +
> 5 files changed, 343 insertions(+)
> create mode 100644 Documentation/driver-api/virtio/index.rst
> create mode 100644 Documentation/driver-api/virtio/virtio.rst
> create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
>
(...)
> +.. rubric:: Footnotes
> +
> +.. [#f1] that's why they may be also referred as virtrings.
"referred to"
(...)
> +The ``probe`` method does the minimum driver setup in this case
> +(memory allocation for the device data) and initializes the
> +virtqueue. The virtqueues are automatically enabled after ``probe``
> +returns, sending the appropriate "DRIVER_OK" status signal to the
> +device. If the virtqueues need to be enabled before ``probe`` ends, they
> +can be manually enabled by calling virtio_device_ready():
> +
> +.. kernel-doc:: include/linux/virtio_config.h
> + :identifiers: virtio_device_ready
Hm, not quite sure what the actual expectations are here: Should the
driver set DRIVER_OK in its probe function, and the core only set it as
a fallback? Michael, Jason?
(...)
LGTM in general.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
2022-08-11 13:46 ` Cornelia Huck
@ 2022-08-11 13:57 ` Michael S. Tsirkin
-1 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 13:57 UTC (permalink / raw)
To: Cornelia Huck
Cc: Ricardo Cañuelo, linux-doc, virtualization, jasowang,
kernel, bagasdotme
On Thu, Aug 11, 2022 at 03:46:46PM +0200, Cornelia Huck wrote:
> On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
>
> > Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
> >
> > Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> > ---
> > Documentation/driver-api/index.rst | 1 +
> > Documentation/driver-api/virtio/index.rst | 11 ++
> > Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> > .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> > MAINTAINERS | 1 +
> > 5 files changed, 343 insertions(+)
> > create mode 100644 Documentation/driver-api/virtio/index.rst
> > create mode 100644 Documentation/driver-api/virtio/virtio.rst
> > create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
> >
>
> (...)
>
> > +.. rubric:: Footnotes
> > +
> > +.. [#f1] that's why they may be also referred as virtrings.
>
> "referred to"
>
> (...)
>
> > +The ``probe`` method does the minimum driver setup in this case
> > +(memory allocation for the device data) and initializes the
> > +virtqueue. The virtqueues are automatically enabled after ``probe``
> > +returns, sending the appropriate "DRIVER_OK" status signal to the
> > +device. If the virtqueues need to be enabled before ``probe`` ends, they
> > +can be manually enabled by calling virtio_device_ready():
> > +
> > +.. kernel-doc:: include/linux/virtio_config.h
> > + :identifiers: virtio_device_ready
>
> Hm, not quite sure what the actual expectations are here: Should the
> driver set DRIVER_OK in its probe function, and the core only set it as
> a fallback? Michael, Jason?
>
> (...)
>
> LGTM in general.
Generally driver should either call device_ready
or defer adding device to linux in a scan callback.
Doing neither is likely a bug since you might
then be asked to add buffers before DRIVER_OK is set,
though e.g. if you are interrupt driven things might work
correctly.
--
MST
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-11 13:57 ` Michael S. Tsirkin
0 siblings, 0 replies; 16+ messages in thread
From: Michael S. Tsirkin @ 2022-08-11 13:57 UTC (permalink / raw)
To: Cornelia Huck; +Cc: linux-doc, virtualization, bagasdotme, kernel
On Thu, Aug 11, 2022 at 03:46:46PM +0200, Cornelia Huck wrote:
> On Wed, Aug 10 2022, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote:
>
> > Basic doc about Virtio on Linux and a short tutorial on Virtio drivers.
> >
> > Signed-off-by: Ricardo Cañuelo <ricardo.canuelo@collabora.com>
> > ---
> > Documentation/driver-api/index.rst | 1 +
> > Documentation/driver-api/virtio/index.rst | 11 ++
> > Documentation/driver-api/virtio/virtio.rst | 144 ++++++++++++++
> > .../virtio/writing_virtio_drivers.rst | 186 ++++++++++++++++++
> > MAINTAINERS | 1 +
> > 5 files changed, 343 insertions(+)
> > create mode 100644 Documentation/driver-api/virtio/index.rst
> > create mode 100644 Documentation/driver-api/virtio/virtio.rst
> > create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
> >
>
> (...)
>
> > +.. rubric:: Footnotes
> > +
> > +.. [#f1] that's why they may be also referred as virtrings.
>
> "referred to"
>
> (...)
>
> > +The ``probe`` method does the minimum driver setup in this case
> > +(memory allocation for the device data) and initializes the
> > +virtqueue. The virtqueues are automatically enabled after ``probe``
> > +returns, sending the appropriate "DRIVER_OK" status signal to the
> > +device. If the virtqueues need to be enabled before ``probe`` ends, they
> > +can be manually enabled by calling virtio_device_ready():
> > +
> > +.. kernel-doc:: include/linux/virtio_config.h
> > + :identifiers: virtio_device_ready
>
> Hm, not quite sure what the actual expectations are here: Should the
> driver set DRIVER_OK in its probe function, and the core only set it as
> a fallback? Michael, Jason?
>
> (...)
>
> LGTM in general.
Generally driver should either call device_ready
or defer adding device to linux in a scan callback.
Doing neither is likely a bug since you might
then be asked to add buffers before DRIVER_OK is set,
though e.g. if you are interrupt driven things might work
correctly.
--
MST
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
2022-08-11 13:32 ` Michael S. Tsirkin
@ 2022-08-11 14:54 ` Ricardo Cañuelo
-1 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-11 14:54 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: linux-doc, virtualization, jasowang, kernel, cohuck, bagasdotme
Hi Michael, thanks for reviewing:
On jue, ago 11 2022 at 09:32:05, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> which tree is this for?
>
> Applying: docs: driver-api: virtio: virtio on Linux
> error: sha1 information is lacking or useless (MAINTAINERS).
> error: could not build fake ancestor
> Patch failed at 0001 docs: driver-api: virtio: virtio on Linux
linux-next, as stated in the cover letter:
Tested on linux-next (next-20220802)
I just verified that the patch also applies to the vhost tree
(linux-next branch). Where did you test it?
>> + buf = virtqueue_get_buf(dev->vq, &len);
>> + /* spurious callback? */
>> + if (!buf)
>> + return;
>
> most drivers need to do this in a loop, this code is only valid if
> there's just 1 buf in flight - unusual.
That's a driver-specific consideration and this is supposed to be the
simplest possible driver skeleton, but ok.
>> + static int virtio_dummy_probe(struct virtio_device *vdev)
>> + {
>> + struct virtio_dummy_dev *dev = NULL;
>> +
>> + /* initialize device data */
>> + dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
>
>
> I dislike how we set dev to NULL and immediately to a different value
> just below.
This is a matter of style, I think. There are plenty of examples of this
in the kernel code, but I don't mind changing it.
> what is missing here is registration with Linux core.
Isn't that supposed to be done by module_virtio_driver()?
> depending on device you might need a call to device_ready, too.
I already explained below that this is done by default after probe()
(see virtio_dev_probe()). virtio_device_ready() is supposed to be useful
only if you need to use the vqs in the probe function.
>> + /*
>> + * Disable vq interrupts: equivalent to
>> + * vdev->config->reset(vdev)
>> + */
>> + virtio_reset_device(vdev);
>> +
>
> you highly likely need to detach unused buffers from vqs here.
Ack.
> let's be a bit clearer here that they must be enabled before add_buf
> triggers.
Ok.
> maybe clarify that they can still trigger even if enabled.
> if you want to disable reliable you have to reset the device
> or the vq.
I'll check that out, thanks.
Cheers,
Ricardo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux
@ 2022-08-11 14:54 ` Ricardo Cañuelo
0 siblings, 0 replies; 16+ messages in thread
From: Ricardo Cañuelo @ 2022-08-11 14:54 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: linux-doc, cohuck, virtualization, bagasdotme, kernel
Hi Michael, thanks for reviewing:
On jue, ago 11 2022 at 09:32:05, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> which tree is this for?
>
> Applying: docs: driver-api: virtio: virtio on Linux
> error: sha1 information is lacking or useless (MAINTAINERS).
> error: could not build fake ancestor
> Patch failed at 0001 docs: driver-api: virtio: virtio on Linux
linux-next, as stated in the cover letter:
Tested on linux-next (next-20220802)
I just verified that the patch also applies to the vhost tree
(linux-next branch). Where did you test it?
>> + buf = virtqueue_get_buf(dev->vq, &len);
>> + /* spurious callback? */
>> + if (!buf)
>> + return;
>
> most drivers need to do this in a loop, this code is only valid if
> there's just 1 buf in flight - unusual.
That's a driver-specific consideration and this is supposed to be the
simplest possible driver skeleton, but ok.
>> + static int virtio_dummy_probe(struct virtio_device *vdev)
>> + {
>> + struct virtio_dummy_dev *dev = NULL;
>> +
>> + /* initialize device data */
>> + dev = kzalloc(sizeof(struct virtio_dummy_dev), GFP_KERNEL);
>
>
> I dislike how we set dev to NULL and immediately to a different value
> just below.
This is a matter of style, I think. There are plenty of examples of this
in the kernel code, but I don't mind changing it.
> what is missing here is registration with Linux core.
Isn't that supposed to be done by module_virtio_driver()?
> depending on device you might need a call to device_ready, too.
I already explained below that this is done by default after probe()
(see virtio_dev_probe()). virtio_device_ready() is supposed to be useful
only if you need to use the vqs in the probe function.
>> + /*
>> + * Disable vq interrupts: equivalent to
>> + * vdev->config->reset(vdev)
>> + */
>> + virtio_reset_device(vdev);
>> +
>
> you highly likely need to detach unused buffers from vqs here.
Ack.
> let's be a bit clearer here that they must be enabled before add_buf
> triggers.
Ok.
> maybe clarify that they can still trigger even if enabled.
> if you want to disable reliable you have to reset the device
> or the vq.
I'll check that out, thanks.
Cheers,
Ricardo
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2022-08-11 14:54 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-10 9:40 [PATCH v4 0/2] docs: driver-api: virtio: virtio on Linux Ricardo Cañuelo
2022-08-10 9:40 ` Ricardo Cañuelo
2022-08-10 9:40 ` [PATCH v4 1/2] virtio: kerneldocs fixes and enhancements Ricardo Cañuelo
2022-08-10 9:40 ` Ricardo Cañuelo
2022-08-11 13:19 ` Cornelia Huck
2022-08-11 13:19 ` Cornelia Huck
2022-08-10 9:40 ` [PATCH v4 2/2] docs: driver-api: virtio: virtio on Linux Ricardo Cañuelo
2022-08-10 9:40 ` Ricardo Cañuelo
2022-08-11 13:32 ` Michael S. Tsirkin
2022-08-11 13:32 ` Michael S. Tsirkin
2022-08-11 14:54 ` Ricardo Cañuelo
2022-08-11 14:54 ` Ricardo Cañuelo
2022-08-11 13:46 ` Cornelia Huck
2022-08-11 13:46 ` Cornelia Huck
2022-08-11 13:57 ` Michael S. Tsirkin
2022-08-11 13:57 ` Michael S. Tsirkin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.