[virtio-dev] [PATCH] virtio-mmio: Introduce virtio_mmio hotplug

[virtio-dev] [PATCH] virtio-mmio: Introduce virtio_mmio hotplug

[Igor Skalkin]

From: Igor Skalkin <igor.skalkin@opensynergy.com>

While the virtio device is not yet running, the virtual machine manager
advertises the device with device_id set to 0.
During virtio mmio probing, the device_id is checked, and if it is 0,
the rest of the probing function is deferred until the interrupt arrives.

Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com>
---
In our setup, we have a Linux host running virtio devices and virtualised
Linux/Android Guest[s] running virtio drivers.
Situation "the guest OS calls the probe() function for the virtio driver,
but the virtio device has not yet started in the host OS." keeps happening.
Also, some devices need to be hot-plugged later instead of starting during
system sturtup.

Probing of the guest virtio drivers should be deferred until the host device
has started.
---
 drivers/virtio/virtio_mmio.c | 58 ++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 083ff1eb743d..c2e28a8faaaa 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -91,6 +91,8 @@ struct virtio_mmio_device {
        /* a list of queues so we can dispatch IRQs */
        spinlock_t lock;
        struct list_head virtqueues;
+
+       struct work_struct hotplug_work;
 };
  struct virtio_mmio_vq_info {
@@ -592,6 +594,43 @@ static void virtio_mmio_release_dev(struct device *_d)
  /* Platform device */
 +static irqreturn_t hotplug_interrupt(int irq, void *opaque)
+{
+       struct virtio_mmio_device *vm_dev = opaque;
+
+       if (readl(vm_dev->base + VIRTIO_MMIO_DEVICE_ID))
+               schedule_work(&vm_dev->hotplug_work);
+
+       return IRQ_HANDLED;
+}
+
+static int virtio_mmio_request_irq(irq_handler_t handler,
+                                  struct virtio_mmio_device *vm_dev)
+{
+       int err;
+
+       err = request_irq(platform_get_irq(vm_dev->pdev, 0), handler,
+                         IRQF_SHARED, dev_name(&vm_dev->pdev->dev), vm_dev);
+       if (err)
+               dev_err(&vm_dev->pdev->dev, "request_irq(%s) returns %d\n",
+                       dev_name(&vm_dev->pdev->dev), err);
+
+       return err;
+}
+
+static int finish_probe(struct virtio_mmio_device *vm_dev);
+static void virtio_mmio_hotplug_work(struct work_struct *hotplug_work)
+{
+       struct virtio_mmio_device *vm_dev =
+               container_of(hotplug_work, struct virtio_mmio_device,
+                           hotplug_work);
+
+       free_irq(platform_get_irq(vm_dev->pdev, 0), vm_dev);
+
+       if (finish_probe(vm_dev))
+               virtio_mmio_request_irq(hotplug_interrupt, vm_dev);
+}
+
 static int virtio_mmio_probe(struct platform_device *pdev)
 {
        struct virtio_mmio_device *vm_dev;
@@ -628,6 +667,25 @@ static int virtio_mmio_probe(struct platform_device
*pdev)
                return -ENXIO;
        }
 +      vm_dev->vdev.id.device = readl(vm_dev->base + VIRTIO_MMIO_DEVICE_ID);
+       if (!vm_dev->vdev.id.device) {
+               rc = virtio_mmio_request_irq(hotplug_interrupt, vm_dev);
+               if (rc)
+                       return rc;
+
+               INIT_WORK(&vm_dev->hotplug_work, virtio_mmio_hotplug_work);
+
+               return 0;
+       }
+
+       return finish_probe(vm_dev);
+}
+
+static int finish_probe(struct virtio_mmio_device *vm_dev)
+{
+       struct platform_device *pdev = vm_dev->pdev;
+       int rc;
+
        vm_dev->vdev.id.device = readl(vm_dev->base + VIRTIO_MMIO_DEVICE_ID);
        if (vm_dev->vdev.id.device == 0) {
                /*
--
2.37.1



Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] [PATCH] virtio-mmio: Introduce virtio_mmio hotplug

[Igor Skalkin]

As long as the virtual device is not yet running, virtual machine
manager presents the device with DeviceID set to 0.
If the driver reads zero from the DeviceID field, it should defer
the rest of the initialization process and wait for an interrupt.

As soon as the device starts, the virtual machine manager sets the
correct device ID and sends an interrupt.
The driver, when it receives an interrupt, must reread the DeviceID
and continue initialization if it is non-zero.
---
In our setup, we have a type 1 hypervisor, a Linux host with virtio
devices, and virtualised Linux/Android guests with virtio drivers.
Both the driver VM and the guest VM start at the same time, so the
"guest OS initializes virtio driver, but the virtio device is not
yet running in the host OS" situation happens all the time.
We also need the ability to start some devices manually after the
system starts.

Linux kernel patch with the implementation example:
https://lists.oasis-open.org/archives/virtio-dev/202208/msg00075.html

Signed-off-by: Igor Skalkin <Igor.Skalkin@opensynergy.com>
---
 content.tex | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/content.tex b/content.tex
index e863709..b9cef85 100644
--- a/content.tex
+++ b/content.tex
@@ -1804,7 +1804,8 @@ \subsection{MMIO Device Register Layout}\label{sec:Virtio Transport Options / Vi
     Value zero (0x0) is used to
     define a system memory map with placeholder devices at static,
     well known addresses, assigning functions to them depending
-    on user's needs.
+    on user's needs (one of the option - placeholder for the hot-plugged
+    device).
   }
   \hline
   \mmioreg{VendorID}{Virtio Subsystem Vendor ID}{0x00c}{R}{}
@@ -2031,7 +2032,7 @@ \subsection{MMIO Device Register Layout}\label{sec:Virtio Transport Options / Vi
 The driver MUST ignore a device with \field{Version} which is not 0x2,
 although it MAY report an error.

-The driver MUST ignore a device with \field{DeviceID} 0x0,
+The driver MUST ignore a device, as long as reading from \field{DeviceID} returns 0x0,
 but MUST NOT report any error.

 Before reading from \field{DeviceFeatures}, the driver MUST write a value to \field{DeviceFeaturesSel}.
@@ -2071,9 +2072,23 @@ \subsubsection{Device Initialization}\label{sec:Virtio Transport Options / Virti

 The driver MUST start the device initialization by reading and
 checking values from \field{MagicValue} and \field{Version}.
-If both values are valid, it MUST read \field{DeviceID}
-and if its value is zero (0x0) MUST abort initialization and
-MUST NOT access any other register.
+If both values are valid, it MUST read \field{DeviceID}.
+
+If the value of \field{DeviceID} is zero (0x0):
+\begin{itemize}
+\item The driver MUST postpone initialisation until an interrupt arrives,
+and MUST NOT access any other register.
+
+\item When an interrupt occurs, the driver should reread \field{DeviceID}, and
+if it is non-zero, continue with further initialization.
+\begin{note}
+At this stage, (the device has not even been reset yet) driver MUST NOT access
+any other registers, including \field{InterruptStatus} and \field{InterruptACK},
+so it can not proceed with the interrupt status check and acknowledgement as described in
+\ref{sec:Virtio Transport Options / Virtio Over MMIO / MMIO-specific Initialization And Device Operation / Notifications From The Device}~\nameref{sec:Virtio Transport Options / Virtio Over MMIO / MMIO-specific Initialization And Device Operation / Notifications From The Device},
+it can only check if a hotplug device has already started by reading \field{DeviceID}.
+\end{note}
+\end{itemize}

 Drivers not expecting shared memory MUST NOT use the shared
 memory registers.
--
2.37.2


Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org