[virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Nikos Dragazis]

Hi everyone,

this PATCH presents an updated version of the RFC virtio device spec for
the virtio-vhost-user device. The initial RFC implementation can be
found here: [1].

This PATCH is split into four parts:

1. the first commit is just a refactored version of the initial RFC
implementation [1]. I just moved the device spec into a separate .tex
file and changed the device id from #24 to #28 since the ids #24-#27 are
reserved.

2. the second commit fixes some minor issues with the device spec. This
is quite straightforward.

3. the third commit enhances the notification capability with some
device/driver requirements. This makes sense because the notification
capability behaves similarly to the MSI-X capability, thereby having
some requirements that need to be mentioned.

4. the fourth commit synchronizes the shared memory capability with a
recent patch [2] that attempts to standardize a standalone
SHARED_MEMORY_CFG virtio capability.

v3 changes:
 * Device Requirements for the Notification Capability: point out the
   difference between the MSI-X Table Size stored in the Message Control
   register of the MSI-X capability structure and the actual MSI-X Table
   Size

v2 changes:
 * Change device id from #25 to #28

Looking forward to your comments.

Thanks,
Nikos

[1] https://lists.oasis-open.org/archives/virtio-dev/201801/msg00110.html
[2] https://lists.oasis-open.org/archives/virtio-dev/201902/msg00142.html

Nikos Dragazis (3):
  vhost-user: minor fixes
  vhost-user: add requirements for the notification capability
  vhost-user: update shared memory capability

Stefan Hajnoczi (1):
  vhost-user: add vhost-user device type

 content.tex           |   3 +
 introduction.tex      |   1 +
 virtio-vhost-user.tex | 349 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 353 insertions(+)
 create mode 100644 virtio-vhost-user.tex

-- 
2.7.4


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] [PATCH v3 1/4] vhost-user: add vhost-user device type

[Nikos Dragazis]

From: Stefan Hajnoczi <stefanha@redhat.com>

The vhost-user device backend facilitates vhost-user device emulation
through vhost-user protocol exchanges and access to shared memory.
Software-defined networking, storage, and other I/O appliances can
provide services through this device.

This device is based on Wei Wang's vhost-pci work.  The virtio
vhost-user device differs from vhost-pci because it is a single virtio
device type that exposes the vhost-user protocol instead of a family of
new virtio device types, one for each vhost-user device type.

This device supports vhost-user slave and vhost-user master
reconnection.  It also contains a UUID so that vhost-user slave programs
can identify a specific device among many without using bus addresses.

It is somewhat unconventional for a virtio device because it makes use
of additional resources called doorbells, notifications, and shared
memory.  A mapping of these resources to the virtio PCI transport is
provided.  Other transports, such as CCW may not be able to support
this device.

Cc: Wei Wang <wei.w.wang@intel.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
v2:
 * Change device id from #25 to #28

 content.tex           |   3 +
 introduction.tex      |   1 +
 virtio-vhost-user.tex | 292 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 296 insertions(+)
 create mode 100644 virtio-vhost-user.tex

diff --git a/content.tex b/content.tex
index 193b6e1..cb7c545 100644
--- a/content.tex
+++ b/content.tex
@@ -2634,6 +2634,8 @@ \chapter{Device Types}\label{sec:Device Types}
 \hline
 24         &   Memory device \\
 \hline
+28         &   vhost-user device backend \\
+\hline
 \end{tabular}
 
 Some of the devices above are unspecified by this document,
@@ -5594,6 +5596,7 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 \input{virtio-input.tex}
 \input{virtio-crypto.tex}
 \input{virtio-vsock.tex}
+\input{virtio-vhost-user.tex}
 
 \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
 
diff --git a/introduction.tex b/introduction.tex
index c96acf9..f4bb047 100644
--- a/introduction.tex
+++ b/introduction.tex
@@ -60,6 +60,7 @@ \section{Normative References}\label{sec:Normative References}
 	\phantomsection\label{intro:SCSI MMC}\textbf{[SCSI MMC]} &
         SCSI Multimedia Commands,
         \newline\url{http://www.t10.org/cgi-bin/ac.pl?t=f&f=mmc6r00.pdf}\\
+	\phantomsection\label{intro:Vhost-user Protocol}\textbf{[Vhost-user Protocol]} & Vhost-user Protocol, \newline\url{https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/interop/vhost-user.txt;hb=HEAD}, and any future revisions\\
 
 \end{longtable}
 
diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
new file mode 100644
index 0000000..ac96dc2
--- /dev/null
+++ b/virtio-vhost-user.tex
@@ -0,0 +1,292 @@
+\section{Vhost-user Device Backend}\label{sec:Device Types / Vhost-user Device Backend}
+
+The vhost-user device backend facilitates vhost-user device emulation through
+vhost-user protocol exchanges and access to shared memory.  Software-defined
+networking, storage, and other I/O appliances can provide services through this
+device.
+
+This section relies on definitions from the \hyperref[intro:Vhost-user
+Protocol]{Vhost-user Protocol}.  Knowledge of the vhost-user protocol is a
+prerequisite for understanding this device.
+
+The \hyperref[intro:Vhost-user Protocol]{Vhost-user Protocol} was originally
+designed for processes on a single system communicating over UNIX domain
+sockets.  The virtio vhost-user device backend allows the vhost-user slave to
+communicate with the vhost-user master over the device instead of a UNIX domain
+socket.  This allows the slave and master to run on two separate systems such
+as a virtual machine and a hypervisor.
+
+The vhost-user slave program exchanges vhost-user protocol messages with the
+vhost-user master through this device.  How the device implementation
+communicates with the vhost-user master is beyond the scope of this
+specification.  One possible device implementation uses a UNIX domain socket to
+relay messages to a vhost-user master process running on the same host.
+
+Existing vhost-user slave programs that communicate over UNIX domain sockets
+can support the virtio vhost-user device backend without invasive changes
+because the pre-existing vhost-user wire protocol is used.
+
+\subsection{Device ID}\label{sec:Device Types / Vhost-user Device Backend / Device ID}
+  28
+
+\subsection{Virtqueues}\label{sec:Device Types / Vhost-user Device Backend / Virtqueues}
+
+\begin{description}
+\item[0] rxq (device-to-driver vhost-user protocol messages)
+\item[1] txq (driver-to-device vhost-user protocol messages)
+\end{description}
+
+\subsection{Feature bits}\label{sec:Device Types / Vhost-user Device Backend / Feature bits}
+
+No feature bits are defined at this time.
+
+\subsection{Device configuration layout}\label{sec:Device Types / Vhost-user Device Backend / Device configuration layout}
+
+  All fields of this configuration are always available.
+
+\begin{lstlisting}
+struct virtio_vhost_user_config {
+        le32 status;
+#define VIRTIO_VHOST_USER_STATUS_SLAVE_UP 0
+#define VIRTIO_VHOST_USER_STATUS_MASTER_UP 1
+        le32 max_vhost_queues;
+        u8 uuid[16];
+};
+\end{lstlisting}
+
+\begin{description}
+\item[\field{status}] contains the vhost-user operational status.  The default
+    value of this field is 0.
+
+    The driver sets VIRTIO_VHOST_USER_STATUS_SLAVE_UP to indicate readiness for
+    the vhost-user master to connect.  The vhost-user master cannot connect
+    unless the driver has set this bit first.
+
+    When the driver clears VIRTIO_VHOST_USER_SLAVE_UP while the vhost-user
+    master is connected, the vhost-user master is disconnected.
+
+    When the vhost-user master disconnects, both
+    VIRTIO_VHOST_USER_STATUS_SLAVE_UP and VIRTIO_VHOST_USER_STATUS_MASTER_UP
+    are cleared by the device.  Communication can be restarted by the driver
+    setting VIRTIO_VHOST_USER_STATUS_SLAVE_UP again.
+
+    A configuration change notification is sent when the device changes
+    this field unless a write to the field by the driver caused the change.
+
+\item[\field{max_vhost_queues}] is the maximum number of vhost-user queues
+    supported by this device.  This field is always greater than 0.
+
+\item[\field{uuid}] is the Universally Unique Identifier (UUID) for this
+    device.  If the device has no UUID then this field contains the nil
+    UUID (all zeroes).  The UUID allows vhost-user slave programs to identify a
+    specific vhost-user device backend among many without relying on bus
+    addresses.
+\end{description}
+
+\drivernormative{\subsubsection}{Device configuration layout}{Device Types / Vhost-user Device Backend / Device configuration layout}
+
+The driver MUST NOT write to device configuration fields other than
+\field{status}.
+
+The driver MUST NOT set undefined bits in the \field{status} configuration field.
+
+\devicenormative{\subsection}{Device Initialization}{Device Types / Vhost-user Device Backend / Device Initialization}
+
+The driver SHOULD check the \field{max_vhost_queues} configuration field to
+determine how many queues the vhost-user slave will be able to support.
+
+The driver SHOULD fetch the \field{uuid} configuration field to allow
+vhost-user slave programs to identify a specific device among many.
+
+The driver SHOULD place at least one buffer in rxq before setting the
+VIRTIO_VHOST_USER_SLAVE_UP bit in the \field{status} configuration field.
+
+The driver MUST handle rxq virtqueue notifications that occur before the
+configuration change notification.  It is possible that a vhost-user protocol
+message from the vhost-user master arrives before the driver has seen the
+configuration change notification for the VIRTIO_VHOST_USER_STATUS_MASTER_UP
+\field{status} change.
+
+\subsection{Device Operation}\label{sec:Device Types / Vhost-user Device Backend / Device Operation}
+
+Device operation consists of operating request queues and response queues.
+
+\subsubsection{Device Operation: Request Queues}\label{sec:Device Types / Vhost-user Device Backend / Device Operation / Device Operation: Request Queues}
+
+The driver receives vhost-user protocol messages from the vhost-user master on
+rxq.  The driver sends responses to the vhost-user master on txq.
+
+The driver sends slave-initiated requests on txq.  The driver receives
+responses from the vhost-user master on rxq.
+
+All virtqueues offer in-order guaranteed delivery semantics for vhost-user
+protocol messages.
+
+Each buffer is a vhost-user protocol message as defined by the
+\hyperref[intro:Vhost-user Protocol]{Vhost-user Protocol}.  In order to enable
+cross-endian communication, all message fields are little-endian instead of the
+native byte order normally used by the protocol.
+
+The appropriate size of rxq buffers is at least as large as the largest message
+defined by the \hyperref[intro:Vhost-user Protocol]{Vhost-user Protocol}
+standard version that the driver supports.  If the vhost-user master sends a
+message that is too large for an rxq buffer then DEVICE_NEEDS_RESET is set and
+the driver must reset the device.
+
+File descriptor passing is handled differently by the vhost-user device
+backend.  When a message is received that carries one or more file descriptors
+according to the vhost-user protocol, additional device resources become
+available to the driver.
+
+\subsection{Additional Device Resources over PCI}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI}
+
+The vhost-user device backend contains additional device resources beyond
+configuration space and virtqueues.  The nature of these resources is
+transport-specific and therefore only virtio transports that provide these
+resources support the vhost-user device backend.
+
+The following additional resources exist:
+\begin{description}
+  \item[Doorbells] The driver signals the vhost-user master through doorbells.  The signal does not carry any data, it is purely an event.
+  \item[Notifications] The vhost-user master signals the driver for events besides virtqueue activity and configuration changes by sending notifications.
+  \item[Shared memory] The vhost-user master gives access to memory that can be mapped by the driver.
+\end{description}
+
+\subsubsection{Doorbell Numbering}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Doorbell Numbering}
+
+Doorbells are laid out as follows:
+
+\begin{description}
+\item[0] Vring call for vhost-user queue 0
+\item[\ldots]
+\item[N] Vring err for vhost-user queue 0
+\item[\ldots]
+\item[2N] Log
+\end{description}
+
+\subsubsection{Notifications}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Notifications}
+
+Notifications are laid out as follows:
+
+\begin{description}
+\item[0] Vring kick for vhost-user queue 0
+\item[\ldots]
+\item[N-1] Vring kick for vhost-user queue N-1
+\end{description}
+
+\subsubsection{Shared Memory Layout}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory Layout}
+
+Shared memory is laid out as follows:
+
+\begin{description}
+\item[0] Vhost memory region 0
+\item[SIZE0] Vhost memory region 1
+\item[\ldots]
+\item[SIZE0 + SIZE1 + \ldots] Log
+\end{description}
+
+The size of vhost memory region 0 is \field{SIZE0}, the size of vhost memory
+region 1 is \field{SIZE1}, and so on.
+
+\subsubsection{Availability of Additional Resources}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Availability of Additional Resources}
+
+The following vhost-user protocol messages convey access to additional device
+resources:
+
+\begin{description}
+\item[VHOST_USER_SET_MEM_TABLE] Contents of vhost memory regions are available to the driver in shared memory.  Region contents are laid out in the same order as the vhost memory region list.
+\item[VHOST_USER_SET_LOG_BASE] Contents of the log are available to the driver in shared memory.
+\item[VHOST_USER_SET_LOG_FD] The log doorbell is available to the driver.  Writes to the log doorbell before this message is received produce no effect.
+\item[VHOST_USER_SET_VRING_KICK] The vring kick notification for this queue is available to the driver.  The first notification may occur before the driver has processed this message.
+\item[VHOST_USER_SET_VRING_CALL] The vring call doorbell for this queue is available to the driver.  Writes to the vring call doorbell before this message is received produce no effect.
+\item[VHOST_USER_SET_VRING_ERR] The vring err doorbell for this queue is available to the driver.  Writes to the vring err doorbell before this message is received produce no effect.
+\item[VHOST_USER_SET_SLAVE_REQ_FD] The driver may send vhost-user protocol slave messages on txq.  Buffers put onto txq before this message is received are discarded by the device.
+\end{description}
+
+Additional resources are configured on the virtio PCI transport by the following \field{struct virtio_pci_cap.cfg_type} values:
+
+\begin{lstlisting}
+#define VIRTIO_PCI_CAP_DOORBELL_CFG 6
+#define VIRTIO_PCI_CAP_NOTIFICATION_CFG 7
+#define VIRTIO_PCI_CAP_SHARED_MEMORY_CFG 8
+\end{lstlisting}
+
+\subsubsection{Doorbell structure layout}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Doorbell capability}
+
+The doorbell location is found using the VIRTIO_PCI_CAP_DOORBELL_CFG
+capability.  This capability is immediately followed by an additional
+field, like so:
+
+\begin{lstlisting}
+struct virtio_pci_doorbell_cap {
+        struct virtio_pci_cap cap;
+        le32 doorbell_off_multiplier;
+};
+\end{lstlisting}
+
+The doorbell address within a BAR is calculated as follows:
+
+\begin{lstlisting}
+        cap.offset + doorbell_idx * doorbell_off_multiplier
+\end{lstlisting}
+
+The \field{cap.offset} and \field{doorbell_off_multiplier} are taken from the
+notification capability structure above, and the \field{doorbell_idx} is the
+doorbell number.
+
+\devicenormative{\paragraph}{Doorbell capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Doorbell capability}
+The device MUST present at least one doorbell capability.
+
+The \field{cap.offset} MUST be 2-byte aligned.
+
+The device MUST either present \field{doorbell_off_multiplier} as an even power of 2,
+or present \field{doorbell_off_multiplier} as 0.
+
+The value \field{cap.length} presented by the device MUST be at least 2
+and MUST be large enough to support doorbell offsets for all supported
+doorbells in all possible configurations.
+
+The value \field{cap.length} presented by the device MUST satisfy:
+\begin{lstlisting}
+cap.length >= num_doorbells * doorbell_off_multiplier + 2
+\end{lstlisting}
+
+The number of doorbells is \field{num_doorbells} and is dependent on the
+device.
+
+\subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Notification capability}
+
+The notification structure allows MSI-X vectors to be configured for
+notification interrupts.  If MSI-X is not available, bit 2 of the ISR status
+indicates that a notification occurred.
+
+The notification structure is found using the VIRTIO_PCI_CAP_DOORBELL_CFG
+capability.
+
+\begin{lstlisting}
+struct virtio_pci_notification_cfg {
+        le16 notification_select;              /* read-write */
+        le16 notification_msix_vector;         /* read-write */
+};
+\end{lstlisting}
+
+The driver indicates which notification is of interest by writing the
+\field{notification_select} field.  The driver then writes the MSI-X vector or
+\field{VIRTIO_MSI_NO_VECTOR} to \field{notification_msix_vector} to change the
+MSI-X vector for that notification.
+
+\subsubsection{Shared memory capability}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
+
+The shared memory location is found using the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG
+capability.
+
+\devicenormative{\paragraph}{Shared Memory capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
+The device MUST present exactly one shared memory capability.
+
+The device MUST locate shared memory in a Memory Space BAR.
+
+The device SHOULD locate shared memory in a Prefetchable BAR.
+
+The \field{cap.offset} MUST be 4096-byte aligned.
+
+The value \field{cap.length} presented by the device MUST be non-zero and 4096-byte aligned.
-- 
2.7.4


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] [PATCH v3 2/4] vhost-user: minor fixes

[Nikos Dragazis]

Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 virtio-vhost-user.tex | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
index ac96dc2..13ecd41 100644
--- a/virtio-vhost-user.tex
+++ b/virtio-vhost-user.tex
@@ -231,7 +231,7 @@ \subsubsection{Doorbell structure layout}\label{sec:Device Types / Vhost-user De
 \end{lstlisting}
 
 The \field{cap.offset} and \field{doorbell_off_multiplier} are taken from the
-notification capability structure above, and the \field{doorbell_idx} is the
+doorbell capability structure above, and the \field{doorbell_idx} is the
 doorbell number.
 
 \devicenormative{\paragraph}{Doorbell capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Doorbell capability}
@@ -260,7 +260,7 @@ \subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-use
 notification interrupts.  If MSI-X is not available, bit 2 of the ISR status
 indicates that a notification occurred.
 
-The notification structure is found using the VIRTIO_PCI_CAP_DOORBELL_CFG
+The notification structure is found using the VIRTIO_PCI_CAP_NOTIFICATION_CFG
 capability.
 
 \begin{lstlisting}
-- 
2.7.4


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] [PATCH v3 3/4] vhost-user: add requirements for the notification capability

[Nikos Dragazis]

The configuration structure for the notification capability has certain
device/driver requirements, similar to those of the MSI-X vector
configuration. Add these requirements to the device spec.

Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
---
v3:
 * Point out the difference between the MSI-X Table Size stored in the
   Message Control Register of the MSI-X Capability structure and the
   actual Table Size

 virtio-vhost-user.tex | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
index 13ecd41..7a8cd9c 100644
--- a/virtio-vhost-user.tex
+++ b/virtio-vhost-user.tex
@@ -275,6 +275,48 @@ \subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-use
 \field{VIRTIO_MSI_NO_VECTOR} to \field{notification_msix_vector} to change the
 MSI-X vector for that notification.
 
+\devicenormative{\paragraph}{Notification capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Notification capability}
+
+If MSI-X is available, device MUST support mapping any master queue
+event to any valid vector 0 to MSI-X \field{Table Size}. Here
+\field{Table Size} is the \field{N-1} encoded Table Size stored in the
+Message Control register of the MSI-X capability structure according to
+\hyperref[intro:PCI]{[PCI]}, where \field{N} is the actual MSI-X Table
+Size.
+
+Device MUST support unmapping any master queue event.
+
+The device MUST return vector mapped to a given master queue event,
+(\field{NO_VECTOR} if unmapped) on read of
+\field{notification_msix_vector}. The device MUST have all master queue
+events unmapped upon reset.
+
+Devices SHOULD NOT cause mapping an event to vector to fail unless it is
+impossible for the device to satisfy the mapping request. Devices MUST
+report mapping failures by returning the \field{NO_VECTOR} value when
+the relevant \field{notification_msix_vector} field is read.
+
+\drivernormative{\paragraph}{Notification capability}{Device Types /
+Vhost-user Device Backend / Additional Device Resources over PCI /
+Noification capability}
+
+Driver MUST support device with any MSI-X Table Size 0 to 0x7FF. Driver
+MAY fall back on using INT\#x interrupts for a device which only
+supports one MSI-X vector (MSI-X Table Size = 0).
+
+Driver MAY intepret the \field{Table Size} as a hint from the device for
+the suggested number of MSI-X vectors to use.
+
+Driver MUST NOT attempt to map an event to a vector outside the MSI-X
+Table supported by the device, as reported by \field{Table Size} in the
+MSI-X Capability.
+
+After mapping an event to vector, the driver MUST verify success by
+reading the Vector field value: on success, the previously written value
+is returned, and on failure, \field{NO_VECTOR} is returned. If a mapping
+failure is detected, the driver MAY retry mapping with fewer vectors,
+disable MSI-X or report device failure.
+
 \subsubsection{Shared memory capability}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
 
 The shared memory location is found using the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG
-- 
2.7.4


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] [PATCH v3 4/4] vhost-user: update shared memory capability

[Nikos Dragazis]

The virtio vhost-user device has shared memory resources. Therefore, it
uses the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capability to standardize
those resources through the PCI Configuration Space. This patch
synchronizes the virtio vhost-user device spec with the following shared
memory resources PATCH:

https://lists.oasis-open.org/archives/virtio-dev/201902/msg00142.html

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
---
 virtio-vhost-user.tex | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
index 7a8cd9c..d1305dd 100644
--- a/virtio-vhost-user.tex
+++ b/virtio-vhost-user.tex
@@ -320,7 +320,22 @@ \subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-use
 \subsubsection{Shared memory capability}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
 
 The shared memory location is found using the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG
-capability.
+capability. Using the additional \field{offset_hi} and \field{length_hi}
+fields following the SHARED_MEMORY_CFG capability structure, the shared
+memory region's 64-bit BAR offset is calculated as follows:
+
+\begin{lstlisting}
+        offset_hi << 32 + cap.offset
+\end{lstlisting}
+
+and the shared memory region's 64-bit length is calculated as follows:
+
+\begin{lstlisting}
+        length_hi << 32 + cap.length
+\end{lstlisting}
+
+The \field{cap.offset} and \field{cap.length} fields are taken from the
+SHARED_MEMORY_CFG capability structure.
 
 \devicenormative{\paragraph}{Shared Memory capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
 The device MUST present exactly one shared memory capability.
-- 
2.7.4


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Nikos Dragazis]

Hi all,

This is a kind reminder to solicit comments on this patchset, and ask
for next steps. What can I do to help with the process of merging this?

Best regards,
Nikos

On 11/5/19 9:47 μ.μ., Nikos Dragazis wrote:
> Hi everyone,
>
> this PATCH presents an updated version of the RFC virtio device spec for
> the virtio-vhost-user device. The initial RFC implementation can be
> found here: [1].
>
> This PATCH is split into four parts:
>
> 1. the first commit is just a refactored version of the initial RFC
> implementation [1]. I just moved the device spec into a separate .tex
> file and changed the device id from #24 to #28 since the ids #24-#27 are
> reserved.
>
> 2. the second commit fixes some minor issues with the device spec. This
> is quite straightforward.
>
> 3. the third commit enhances the notification capability with some
> device/driver requirements. This makes sense because the notification
> capability behaves similarly to the MSI-X capability, thereby having
> some requirements that need to be mentioned.
>
> 4. the fourth commit synchronizes the shared memory capability with a
> recent patch [2] that attempts to standardize a standalone
> SHARED_MEMORY_CFG virtio capability.
>
> v3 changes:
>  * Device Requirements for the Notification Capability: point out the
>    difference between the MSI-X Table Size stored in the Message Control
>    register of the MSI-X capability structure and the actual MSI-X Table
>    Size
>
> v2 changes:
>  * Change device id from #25 to #28
>
> Looking forward to your comments.
>
> Thanks,
> Nikos
>
> [1] https://lists.oasis-open.org/archives/virtio-dev/201801/msg00110.html
> [2] https://lists.oasis-open.org/archives/virtio-dev/201902/msg00142.html
>
> Nikos Dragazis (3):
>   vhost-user: minor fixes
>   vhost-user: add requirements for the notification capability
>   vhost-user: update shared memory capability
>
> Stefan Hajnoczi (1):
>   vhost-user: add vhost-user device type
>
>  content.tex           |   3 +
>  introduction.tex      |   1 +
>  virtio-vhost-user.tex | 349 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 353 insertions(+)
>  create mode 100644 virtio-vhost-user.tex
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [PATCH v3 4/4] vhost-user: update shared memory capability

[Dr. David Alan Gilbert]

* Nikos Dragazis (ndragazis@arrikto.com) wrote:
> The virtio vhost-user device has shared memory resources. Therefore, it
> uses the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capability to standardize
> those resources through the PCI Configuration Space. This patch
> synchronizes the virtio vhost-user device spec with the following shared
> memory resources PATCH:
> 
> https://lists.oasis-open.org/archives/virtio-dev/201902/msg00142.html
> 
> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
> ---
>  virtio-vhost-user.tex | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
> 
> diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
> index 7a8cd9c..d1305dd 100644
> --- a/virtio-vhost-user.tex
> +++ b/virtio-vhost-user.tex
> @@ -320,7 +320,22 @@ \subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-use
>  \subsubsection{Shared memory capability}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
>  
>  The shared memory location is found using the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG
> -capability.
> +capability. Using the additional \field{offset_hi} and \field{length_hi}
> +fields following the SHARED_MEMORY_CFG capability structure, the shared
> +memory region's 64-bit BAR offset is calculated as follows:
> +
> +\begin{lstlisting}
> +        offset_hi << 32 + cap.offset
> +\end{lstlisting}
> +
> +and the shared memory region's 64-bit length is calculated as follows:
> +
> +\begin{lstlisting}
> +        length_hi << 32 + cap.length
> +\end{lstlisting}
> +
> +The \field{cap.offset} and \field{cap.length} fields are taken from the
> +SHARED_MEMORY_CFG capability structure.

Yes, or this could just reference the:
{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}

section once it's merged.

Do you actually need to specify that it's found by PCI though?
Could you just define it as using Shared Memory Regions,
reference {sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
and then it would be defined to work with PCI, MMIO or CCW if they
define the shared memory region support.

Dave


>  \devicenormative{\paragraph}{Shared Memory capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
>  The device MUST present exactly one shared memory capability.
> -- 
> 2.7.4
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Re: [PATCH v3 4/4] vhost-user: update shared memory capability

[Nikos Dragazis]

On 31/5/19 7:59 μ.μ., Dr. David Alan Gilbert wrote:
> * Nikos Dragazis (ndragazis@arrikto.com) wrote:
>> The virtio vhost-user device has shared memory resources. Therefore, it
>> uses the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG capability to standardize
>> those resources through the PCI Configuration Space. This patch
>> synchronizes the virtio vhost-user device spec with the following shared
>> memory resources PATCH:
>>
>> https://lists.oasis-open.org/archives/virtio-dev/201902/msg00142.html
>>
>> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
>> Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
>> ---
>>  virtio-vhost-user.tex | 17 ++++++++++++++++-
>>  1 file changed, 16 insertions(+), 1 deletion(-)
>>
>> diff --git a/virtio-vhost-user.tex b/virtio-vhost-user.tex
>> index 7a8cd9c..d1305dd 100644
>> --- a/virtio-vhost-user.tex
>> +++ b/virtio-vhost-user.tex
>> @@ -320,7 +320,22 @@ \subsubsection{Notification structure layout}\label{sec:Device Types / Vhost-use
>>  \subsubsection{Shared memory capability}\label{sec:Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
>>  
>>  The shared memory location is found using the VIRTIO_PCI_CAP_SHARED_MEMORY_CFG
>> -capability.
>> +capability. Using the additional \field{offset_hi} and \field{length_hi}
>> +fields following the SHARED_MEMORY_CFG capability structure, the shared
>> +memory region's 64-bit BAR offset is calculated as follows:
>> +
>> +\begin{lstlisting}
>> +        offset_hi << 32 + cap.offset
>> +\end{lstlisting}
>> +
>> +and the shared memory region's 64-bit length is calculated as follows:
>> +
>> +\begin{lstlisting}
>> +        length_hi << 32 + cap.length
>> +\end{lstlisting}
>> +
>> +The \field{cap.offset} and \field{cap.length} fields are taken from the
>> +SHARED_MEMORY_CFG capability structure.
> Yes, or this could just reference the:
> {sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Shared memory capability}
>
> section once it's merged.

That makes sense.

>
> Do you actually need to specify that it's found by PCI though?
> Could you just define it as using Shared Memory Regions,
> reference {sec:Basic Facilities of a Virtio Device / Shared Memory Regions}
> and then it would be defined to work with PCI, MMIO or CCW if they
> define the shared memory region support.

As you may have noticed, the virtio-vhost-user device, in contrast to
the other devices, needs some additional resources (doorbells,
notifications, shared memory), including of course the shared memory
regions which you are attempting to standardize. So, this device can
only be supported by transports that can somehow export all of these
resources, not just the shared memory resource.

In contrast to the shared memory region, the doorbells and notifications
resources are not standardized. Therefore, this patchset offers a
straightforward mapping of all these resources to the PCI transport via
virtio PCI capabilities. I am not sure about how the other virtio
transports could possibly support these resources.

Hope this makes sense :)
Any suggestions are welcome.

--
Nikos

>
> Dave
>
>
>>  \devicenormative{\paragraph}{Shared Memory capability}{Device Types / Vhost-user Device Backend / Additional Device Resources over PCI / Shared Memory capability}
>>  The device MUST present exactly one shared memory capability.
>> -- 
>> 2.7.4
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
> Hi everyone,
> 
> this PATCH presents an updated version of the RFC virtio device spec for
> the virtio-vhost-user device. The initial RFC implementation can be
> found here: [1].
> 
> This PATCH is split into four parts:
> 
> 1. the first commit is just a refactored version of the initial RFC
> implementation [1]. I just moved the device spec into a separate .tex
> file and changed the device id from #24 to #28 since the ids #24-#27 are
> reserved.
> 
> 2. the second commit fixes some minor issues with the device spec. This
> is quite straightforward.
> 
> 3. the third commit enhances the notification capability with some
> device/driver requirements. This makes sense because the notification
> capability behaves similarly to the MSI-X capability, thereby having
> some requirements that need to be mentioned.
> 
> 4. the fourth commit synchronizes the shared memory capability with a
> recent patch [2] that attempts to standardize a standalone
> SHARED_MEMORY_CFG virtio capability.
> 
> v3 changes:
>  * Device Requirements for the Notification Capability: point out the
>    difference between the MSI-X Table Size stored in the Message Control
>    register of the MSI-X capability structure and the actual MSI-X Table
>    Size
> 
> v2 changes:
>  * Change device id from #25 to #28
> 
> Looking forward to your comments.

I'm happy with v3.

Any other comments or shall we proceed to a vote?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Michael S. Tsirkin]

On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
> > Hi everyone,
> > 
> > this PATCH presents an updated version of the RFC virtio device spec for
> > the virtio-vhost-user device. The initial RFC implementation can be
> > found here: [1].
> > 
> > This PATCH is split into four parts:
> > 
> > 1. the first commit is just a refactored version of the initial RFC
> > implementation [1]. I just moved the device spec into a separate .tex
> > file and changed the device id from #24 to #28 since the ids #24-#27 are
> > reserved.
> > 
> > 2. the second commit fixes some minor issues with the device spec. This
> > is quite straightforward.
> > 
> > 3. the third commit enhances the notification capability with some
> > device/driver requirements. This makes sense because the notification
> > capability behaves similarly to the MSI-X capability, thereby having
> > some requirements that need to be mentioned.
> > 
> > 4. the fourth commit synchronizes the shared memory capability with a
> > recent patch [2] that attempts to standardize a standalone
> > SHARED_MEMORY_CFG virtio capability.
> > 
> > v3 changes:
> >  * Device Requirements for the Notification Capability: point out the
> >    difference between the MSI-X Table Size stored in the Message Control
> >    register of the MSI-X capability structure and the actual MSI-X Table
> >    Size
> > 
> > v2 changes:
> >  * Change device id from #25 to #28
> > 
> > Looking forward to your comments.
> 
> I'm happy with v3.
> 
> Any other comments or shall we proceed to a vote?
> 
> Stefan

I think we need to wait for shared memory part to be finalized, right?

-- 
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Nikos Dragazis]

On 21/6/19 11:43 μ.μ., Michael S. Tsirkin wrote:
> On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
>> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
>>> Hi everyone,
>>>
>>> this PATCH presents an updated version of the RFC virtio device spec for
>>> the virtio-vhost-user device. The initial RFC implementation can be
>>> found here: [1].
>>>
>>> This PATCH is split into four parts:
>>>
>>> 1. the first commit is just a refactored version of the initial RFC
>>> implementation [1]. I just moved the device spec into a separate .tex
>>> file and changed the device id from #24 to #28 since the ids #24-#27 are
>>> reserved.
>>>
>>> 2. the second commit fixes some minor issues with the device spec. This
>>> is quite straightforward.
>>>
>>> 3. the third commit enhances the notification capability with some
>>> device/driver requirements. This makes sense because the notification
>>> capability behaves similarly to the MSI-X capability, thereby having
>>> some requirements that need to be mentioned.
>>>
>>> 4. the fourth commit synchronizes the shared memory capability with a
>>> recent patch [2] that attempts to standardize a standalone
>>> SHARED_MEMORY_CFG virtio capability.
>>>
>>> v3 changes:
>>>  * Device Requirements for the Notification Capability: point out the
>>>    difference between the MSI-X Table Size stored in the Message Control
>>>    register of the MSI-X capability structure and the actual MSI-X Table
>>>    Size
>>>
>>> v2 changes:
>>>  * Change device id from #25 to #28
>>>
>>> Looking forward to your comments.
>> I'm happy with v3.
>>
>> Any other comments or shall we proceed to a vote?
>>
>> Stefan
> I think we need to wait for shared memory part to be finalized, right?
>

Hi Michael,

judging from the mailing list, I see that there is still some ongoing
activity on the shared memory patchset. So, yes, waiting for this
patchset to be finalized would be more proper.

Please, let me know when you are done with that.

Thanks,
Nikos

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Nikos Dragazis]

On 21/6/19 11:43 μ.μ., Michael S. Tsirkin wrote:
> On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
>> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
>>> Hi everyone,
>>>
>>> this PATCH presents an updated version of the RFC virtio device spec for
>>> the virtio-vhost-user device. The initial RFC implementation can be
>>> found here: [1].
>>>
>>> This PATCH is split into four parts:
>>>
>>> 1. the first commit is just a refactored version of the initial RFC
>>> implementation [1]. I just moved the device spec into a separate .tex
>>> file and changed the device id from #24 to #28 since the ids #24-#27 are
>>> reserved.
>>>
>>> 2. the second commit fixes some minor issues with the device spec. This
>>> is quite straightforward.
>>>
>>> 3. the third commit enhances the notification capability with some
>>> device/driver requirements. This makes sense because the notification
>>> capability behaves similarly to the MSI-X capability, thereby having
>>> some requirements that need to be mentioned.
>>>
>>> 4. the fourth commit synchronizes the shared memory capability with a
>>> recent patch [2] that attempts to standardize a standalone
>>> SHARED_MEMORY_CFG virtio capability.
>>>
>>> v3 changes:
>>>  * Device Requirements for the Notification Capability: point out the
>>>    difference between the MSI-X Table Size stored in the Message Control
>>>    register of the MSI-X capability structure and the actual MSI-X Table
>>>    Size
>>>
>>> v2 changes:
>>>  * Change device id from #25 to #28
>>>
>>> Looking forward to your comments.
>> I'm happy with v3.
>>
>> Any other comments or shall we proceed to a vote?
>>
>> Stefan
> I think we need to wait for shared memory part to be finalized, right?
>

Michael,

given that David's patchset for the shared memory regions has been
approved, I think we can start discussing on the spec for the
virtio-vhost-user device. A link to the latest version is here: [1].

I really think that this device is useful and should be part of the
VIRTIO specification. I will submit a fifth version of the patchset soon
with some minor changes that seem reasonable to me.

Best regards,
Nikos

[1] https://lists.oasis-open.org/archives/virtio-dev/201906/msg00036.html

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Michael S. Tsirkin]

On Wed, Jul 31, 2019 at 05:49:57PM +0300, Nikos Dragazis wrote:
> On 21/6/19 11:43 μ.μ., Michael S. Tsirkin wrote:
> > On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
> >> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
> >>> Hi everyone,
> >>>
> >>> this PATCH presents an updated version of the RFC virtio device spec for
> >>> the virtio-vhost-user device. The initial RFC implementation can be
> >>> found here: [1].
> >>>
> >>> This PATCH is split into four parts:
> >>>
> >>> 1. the first commit is just a refactored version of the initial RFC
> >>> implementation [1]. I just moved the device spec into a separate .tex
> >>> file and changed the device id from #24 to #28 since the ids #24-#27 are
> >>> reserved.
> >>>
> >>> 2. the second commit fixes some minor issues with the device spec. This
> >>> is quite straightforward.
> >>>
> >>> 3. the third commit enhances the notification capability with some
> >>> device/driver requirements. This makes sense because the notification
> >>> capability behaves similarly to the MSI-X capability, thereby having
> >>> some requirements that need to be mentioned.
> >>>
> >>> 4. the fourth commit synchronizes the shared memory capability with a
> >>> recent patch [2] that attempts to standardize a standalone
> >>> SHARED_MEMORY_CFG virtio capability.
> >>>
> >>> v3 changes:
> >>>  * Device Requirements for the Notification Capability: point out the
> >>>    difference between the MSI-X Table Size stored in the Message Control
> >>>    register of the MSI-X capability structure and the actual MSI-X Table
> >>>    Size
> >>>
> >>> v2 changes:
> >>>  * Change device id from #25 to #28
> >>>
> >>> Looking forward to your comments.
> >> I'm happy with v3.
> >>
> >> Any other comments or shall we proceed to a vote?
> >>
> >> Stefan
> > I think we need to wait for shared memory part to be finalized, right?
> >
> 
> Michael,
> 
> given that David's patchset for the shared memory regions has been
> approved, I think we can start discussing on the spec for the
> virtio-vhost-user device. A link to the latest version is here: [1].
> 
> I really think that this device is useful and should be part of the
> VIRTIO specification. I will submit a fifth version of the patchset soon
> with some minor changes that seem reasonable to me.
> 
> Best regards,
> Nikos
> 
> [1] https://lists.oasis-open.org/archives/virtio-dev/201906/msg00036.html

One part I dislike there is "Additional Device Resources over PCI".
That mostly seems to deal with allocating interrupts.
Can't we (ab)use the virtio pci registers for this?
Just extend VQ number to VQ/notification number.

-- 
MST

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Nikos Dragazis]

On 31/7/19 11:19 μ.μ., Michael S. Tsirkin wrote:
> On Wed, Jul 31, 2019 at 05:49:57PM +0300, Nikos Dragazis wrote:
>> On 21/6/19 11:43 μ.μ., Michael S. Tsirkin wrote:
>>> On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
>>>> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
>>>>> Hi everyone,
>>>>>
>>>>> this PATCH presents an updated version of the RFC virtio device spec for
>>>>> the virtio-vhost-user device. The initial RFC implementation can be
>>>>> found here: [1].
>>>>>
>>>>> This PATCH is split into four parts:
>>>>>
>>>>> 1. the first commit is just a refactored version of the initial RFC
>>>>> implementation [1]. I just moved the device spec into a separate .tex
>>>>> file and changed the device id from #24 to #28 since the ids #24-#27 are
>>>>> reserved.
>>>>>
>>>>> 2. the second commit fixes some minor issues with the device spec. This
>>>>> is quite straightforward.
>>>>>
>>>>> 3. the third commit enhances the notification capability with some
>>>>> device/driver requirements. This makes sense because the notification
>>>>> capability behaves similarly to the MSI-X capability, thereby having
>>>>> some requirements that need to be mentioned.
>>>>>
>>>>> 4. the fourth commit synchronizes the shared memory capability with a
>>>>> recent patch [2] that attempts to standardize a standalone
>>>>> SHARED_MEMORY_CFG virtio capability.
>>>>>
>>>>> v3 changes:
>>>>>  * Device Requirements for the Notification Capability: point out the
>>>>>    difference between the MSI-X Table Size stored in the Message Control
>>>>>    register of the MSI-X capability structure and the actual MSI-X Table
>>>>>    Size
>>>>>
>>>>> v2 changes:
>>>>>  * Change device id from #25 to #28
>>>>>
>>>>> Looking forward to your comments.
>>>> I'm happy with v3.
>>>>
>>>> Any other comments or shall we proceed to a vote?
>>>>
>>>> Stefan
>>> I think we need to wait for shared memory part to be finalized, right?
>>>
>> Michael,
>>
>> given that David's patchset for the shared memory regions has been
>> approved, I think we can start discussing on the spec for the
>> virtio-vhost-user device. A link to the latest version is here: [1].
>>
>> I really think that this device is useful and should be part of the
>> VIRTIO specification. I will submit a fifth version of the patchset soon
>> with some minor changes that seem reasonable to me.
>>
>> Best regards,
>> Nikos
>>
>> [1] https://lists.oasis-open.org/archives/virtio-dev/201906/msg00036.html
> One part I dislike there is "Additional Device Resources over PCI".
> That mostly seems to deal with allocating interrupts.

Actually, it deals with allocating notification addresses (doorbells)
and device interrupts for the vhost-user virtqueues. The
virtio-vhost-user device must be able to handle both its own RX/TX
virtqueues and the vhost-user virtqueues. So, the device must offer
separate notification addresses and interrupt vectors for the vhost-user
virtqueues. We are standardizing these resources with the
VIRTIO_PCI_CAP_DOORBELL_CFG and VIRTIO_PCI_CAP_NOTIFICATION_CFG
configuration structures.

> Can't we (ab)use the virtio pci registers for this?
> Just extend VQ number to VQ/notification number.
>

I am not sure I get your point. But, yes, I think we could use the
existing registers (queue_select, queue_msix_vector, queue_notify_off)
in the common configuration structure for both the device’s virtqueues
and the vhost-user virtqueues. So, for example, we could use indexes 0,
1 for the device’s RX/TX virtqueues and index i+2 for the i-th
vhost-user virtqueue.

However, I don't see why mixing up the configuration of the RX/TX and
the vhost-user virtqueues is a better solution. Wouldn’t it be more
clean if we had separate configuration structures for the device's RX/TX
virtqueues and the vhost-user virtqueues?

--
Nikos

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [PATCH v3 0/4] introduce virtio vhost-user backend device type

[Michael S. Tsirkin]

On Mon, Aug 05, 2019 at 01:32:14AM +0300, Nikos Dragazis wrote:
> On 31/7/19 11:19 μ.μ., Michael S. Tsirkin wrote:
> > On Wed, Jul 31, 2019 at 05:49:57PM +0300, Nikos Dragazis wrote:
> >> On 21/6/19 11:43 μ.μ., Michael S. Tsirkin wrote:
> >>> On Wed, Jun 19, 2019 at 03:54:30PM +0100, Stefan Hajnoczi wrote:
> >>>> On Sat, May 11, 2019 at 09:47:36PM +0300, Nikos Dragazis wrote:
> >>>>> Hi everyone,
> >>>>>
> >>>>> this PATCH presents an updated version of the RFC virtio device spec for
> >>>>> the virtio-vhost-user device. The initial RFC implementation can be
> >>>>> found here: [1].
> >>>>>
> >>>>> This PATCH is split into four parts:
> >>>>>
> >>>>> 1. the first commit is just a refactored version of the initial RFC
> >>>>> implementation [1]. I just moved the device spec into a separate .tex
> >>>>> file and changed the device id from #24 to #28 since the ids #24-#27 are
> >>>>> reserved.
> >>>>>
> >>>>> 2. the second commit fixes some minor issues with the device spec. This
> >>>>> is quite straightforward.
> >>>>>
> >>>>> 3. the third commit enhances the notification capability with some
> >>>>> device/driver requirements. This makes sense because the notification
> >>>>> capability behaves similarly to the MSI-X capability, thereby having
> >>>>> some requirements that need to be mentioned.
> >>>>>
> >>>>> 4. the fourth commit synchronizes the shared memory capability with a
> >>>>> recent patch [2] that attempts to standardize a standalone
> >>>>> SHARED_MEMORY_CFG virtio capability.
> >>>>>
> >>>>> v3 changes:
> >>>>>  * Device Requirements for the Notification Capability: point out the
> >>>>>    difference between the MSI-X Table Size stored in the Message Control
> >>>>>    register of the MSI-X capability structure and the actual MSI-X Table
> >>>>>    Size
> >>>>>
> >>>>> v2 changes:
> >>>>>  * Change device id from #25 to #28
> >>>>>
> >>>>> Looking forward to your comments.
> >>>> I'm happy with v3.
> >>>>
> >>>> Any other comments or shall we proceed to a vote?
> >>>>
> >>>> Stefan
> >>> I think we need to wait for shared memory part to be finalized, right?
> >>>
> >> Michael,
> >>
> >> given that David's patchset for the shared memory regions has been
> >> approved, I think we can start discussing on the spec for the
> >> virtio-vhost-user device. A link to the latest version is here: [1].
> >>
> >> I really think that this device is useful and should be part of the
> >> VIRTIO specification. I will submit a fifth version of the patchset soon
> >> with some minor changes that seem reasonable to me.
> >>
> >> Best regards,
> >> Nikos
> >>
> >> [1] https://lists.oasis-open.org/archives/virtio-dev/201906/msg00036.html
> > One part I dislike there is "Additional Device Resources over PCI".
> > That mostly seems to deal with allocating interrupts.
> 
> Actually, it deals with allocating notification addresses (doorbells)
> and device interrupts for the vhost-user virtqueues. The
> virtio-vhost-user device must be able to handle both its own RX/TX
> virtqueues and the vhost-user virtqueues. So, the device must offer
> separate notification addresses and interrupt vectors for the vhost-user
> virtqueues. We are standardizing these resources with the
> VIRTIO_PCI_CAP_DOORBELL_CFG and VIRTIO_PCI_CAP_NOTIFICATION_CFG
> configuration structures.
> 
> > Can't we (ab)use the virtio pci registers for this?
> > Just extend VQ number to VQ/notification number.
> >
> 
> I am not sure I get your point. But, yes, I think we could use the
> existing registers (queue_select, queue_msix_vector, queue_notify_off)
> in the common configuration structure for both the device’s virtqueues
> and the vhost-user virtqueues. So, for example, we could use indexes 0,
> 1 for the device’s RX/TX virtqueues and index i+2 for the i-th
> vhost-user virtqueue.
> 
> However, I don't see why mixing up the configuration of the RX/TX and
> the vhost-user virtqueues is a better solution.

The reason is that this will automatically buy you support in all
transports: PCI/CCW/MMIO ...

> Wouldn’t it be more
> clean if we had separate configuration structures for the device's RX/TX
> virtqueues and the vhost-user virtqueues?

Flip this on its head and you will see that it buys you nothing except
cosmetics to separate them, and costs you portability across transports.


> --
> Nikos

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org