[virtio] [RFC PATCH 0/3] rework notifications terminology

[virtio] [RFC PATCH 0/3] rework notifications terminology

[Halil Pasic]

While discussing XXXX using available and used buffer notifications
consistently for virqueue notifications (driver to device and vice versa
respectively) seemed like a good idea.

It turned out surprisingly invasive however, and I find it necessary to
confirm, this is really the path we want to walk, before investing even more
time.

This series limits itself to the core and to the ccw transport. It ain't
typical to the device types to make statements about notifications, but at
least net has some words on interrupts. I did not get these immediately so I've
left that out for now. 

I choose ccw as demonstrator on how to bridge the abstract with the transport
specific, because that is the transport I'm most familiar with. I do not expect
difficulties with the other ones.


Halil Pasic (3):
  notifications: unify notifications wording in core
  notifications:notifications as basic virtio facility
  ccw: map common notifications terminology to ccw

 cl-os.tex       |    2 +-
 conformance.tex |    8 ++--
 content.tex     |   92 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 packed-ring.tex |   59 +++++++++++++++++++---------------
 split-ring.tex  |   72 +++++++++++++++++++++++++------------------
 5 files changed, 164 insertions(+), 69 deletions(-)


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Halil Pasic]

Let us unify the wording when talking about notifications. This change
establishes the terms available buffer notification for what was usually
simply called notification or virtqueue notification in v1.0 and used
buffer notification for what was usually called interrupt.

The term configuration change notification in kept where called so and
consolidated where it's called configuration change interrupt or
similar.

The changes done here are limited to the core part, and don't
conceptually involve neither the transports nor the devices (references
are updated though). Future changes should address these parts.

Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
---
 cl-os.tex       |    2 +-
 conformance.tex |    8 +++---
 content.tex     |   20 +++++++++------
 packed-ring.tex |   59 +++++++++++++++++++++++++--------------------
 split-ring.tex  |   72 ++++++++++++++++++++++++++++++++-----------------------
 5 files changed, 92 insertions(+), 69 deletions(-)

diff --git a/cl-os.tex b/cl-os.tex
index da78bf8..fc12cdd 100644
--- a/cl-os.tex
+++ b/cl-os.tex
@@ -2,7 +2,7 @@
 trivial typo
 
 See
-\ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}.
+\ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}.
 } \\
 \hline
 541 & 11 Oct 2015 & Paolo Bonzini & {virtio-blk: fix typo
diff --git a/conformance.tex b/conformance.tex
index e4efe33..de33396 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -37,10 +37,10 @@ A driver MUST conform to the following normative statements:
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Message Framing}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Descriptor Table}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors}
-\item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}
+\item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Available Ring}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
-\item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
+\item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Supplying Buffers to The Device / Updating idx}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Virtqueues / Supplying Buffers to The Device / Notifying The Device}
 \item \ref{drivernormative:General Initialization And Device Operation / Device Initialization}
@@ -158,9 +158,9 @@ A device MUST conform to the following normative statements:
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Message Framing}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Descriptor Table}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Descriptor Table / Indirect Descriptors}
-\item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring}
-\item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
 \item \ref{devicenormative:Reserved Feature Bits}
 \end{itemize}
 
diff --git a/content.tex b/content.tex
index 7a92cb1..4ccb823 100644
--- a/content.tex
+++ b/content.tex
@@ -72,7 +72,8 @@ recover by issuing a reset.
 \devicenormative{\subsection}{Device Status Field}{Basic Facilities of a Virtio Device / Device Status Field}
 The device MUST initialize \field{device status} to 0 upon reset.
 
-The device MUST NOT consume buffers or notify the driver before DRIVER_OK.
+The device MUST NOT consume buffers or send any used buffer
+notifications to the driver before DRIVER_OK.
 
 \label{sec:Basic Facilities of a Virtio Device / Device Status Field / DEVICENEEDSRESET}The device SHOULD set DEVICE_NEEDS_RESET when it enters an error state
 that a reset is needed.  If DRIVER_OK is set, after it sets DEVICE_NEEDS_RESET, the device
@@ -235,12 +236,13 @@ transmit and one for receive.}.
 Driver makes requests available to device by adding
 an available buffer to the queue - i.e. adding a buffer
 describing the request to a virtqueue, and optionally triggering
-a driver event - i.e. sending a notification to the device.
+a driver event - i.e. sending an available buffer notification
+to the device.
 
 Device executes the requests and - when complete - adds
 a used buffer to the queue - i.e. lets the driver
 know by marking the buffer as used. Device can then trigger
-a device event - i.e. send an interrupt to the driver.
+a device event - i.e. send an used buffer notification to the driver.
 
 Device reports the number of bytes it has written to memory for
 each buffer it uses. This is referred to as ``used length''.
@@ -330,7 +332,8 @@ set the FAILED status bit to indicate that it has given up on the
 device (it can reset the device later to restart if desired).  The
 driver MUST NOT continue initialization in that case.
 
-The driver MUST NOT notify the device before setting DRIVER_OK.
+The driver MUST NOT send any buffer available notifications to
+the device before setting DRIVER_OK.
 
 \subsection{Legacy Interface: Device Initialization}\label{sec:General Initialization And Device Operation / Device Initialization / Legacy Interface: Device Initialization}
 Legacy devices did not support the FEATURES_OK status bit, and thus did
@@ -388,10 +391,11 @@ reads unless notified.
 
 \subsection{Notification of Device Configuration Changes}\label{sec:General Initialization And Device Operation / Device Operation / Notification of Device Configuration Changes}
 
-For devices where the device-specific configuration information can be changed, an
-interrupt is delivered when a device-specific configuration change occurs.
+For devices where the device-specific configuration information can be
+changed, a notification is delivered when a device-specific configuration
+change occurs.
 
-In addition, this interrupt is triggered by the device setting
+In addition, this notification is triggered by the device setting
 DEVICE_NEEDS_RESET (see \ref{sec:Basic Facilities of a Virtio Device / Device Status Field / DEVICENEEDSRESET}).
 
 \section{Device Cleanup}\label{sec:General Initialization And Device Operation / Device Cleanup}
@@ -5331,7 +5335,7 @@ Virtqueues / The Virtqueue Descriptor Table / Indirect
 Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Support}~\nameref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Support}.
   \item[VIRTIO_F_RING_EVENT_IDX(29)] This feature enables the \field{used_event}
   and the \field{avail_event} fields as described in
-\ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}, \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring} and \ref{sec:Packed Virtqueues / Driver and Device Event Suppression}.
+\ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}, \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Used Ring} and \ref{sec:Packed Virtqueues / Driver and Device Event Suppression}.
 
 
   \item[VIRTIO_F_VERSION_1(32)] This indicates compliance with this
diff --git a/packed-ring.tex b/packed-ring.tex
index 2ef9559..95a9a35 100644
--- a/packed-ring.tex
+++ b/packed-ring.tex
@@ -43,7 +43,7 @@ The driver then notifies the device. When the device has finished
 processing the buffer, it writes a used device descriptor
 including the Buffer ID into the Descriptor Ring (overwriting a
 driver descriptor previously made available), and sends an
-interrupt.
+used event notification.
 
 The Descriptor Ring is used in a circular manner: the driver writes
 descriptors into the ring in order. After reaching the end of the ring,
@@ -65,11 +65,12 @@ in which their processing is complete.
 
 The Device Event Suppression data structure is write-only by the
 device. It includes information for reducing the number of
-device events - i.e. driver notifications to device.
+device events - i.e. sending an available buffer notification
+to the device.
 
 The Driver Event Suppression data structure is read-only by the
 device. It includes information for reducing the number of
-driver events - i.e. device interrupts to driver.
+driver events - i.e. send an used buffer notification to the driver.
 
 \subsection{Driver and Device Ring Wrap Counters}
 \label{sec:Packed Virtqueues / Driver and Device Ring Wrap Counters}
@@ -309,22 +310,20 @@ in the ring.
 
 \subsection{Driver and Device Event Suppression}
 \label{sec:Packed Virtqueues / Driver and Device Event Suppression}
-In many systems driver and device notifications involve
+In many systems used and available buffer  notifications involve
 significant overhead. To mitigate this overhead,
 each virtqueue includes two identical structures used for
 controlling notifications between the device and the driver.
 
 The Driver Event Suppression structure is read-only by the
-device and controls the events sent by the device
-to the driver (e.g. interrupts).
+device and controls the used buffer notifications (sent by the device
+to the driver).
 
 The Device Event Suppression structure is read-only by
-the driver and controls the events sent by the driver
-to the device (e.g. IO).
+the driver and controls the available buffer notifications (sent by the
+driver to the device).
 
-Each of these Event Suppression structures controls
-both Descriptor Ring events and structure events, and
-each includes the following fields:
+Each of these Event Suppression includes the following fields:
 
 \begin{description}
 \item [Descriptor Ring Change Event Flags] Takes values:
@@ -352,9 +351,9 @@ matches this value and a descriptor is
 made available/used respectively.
 \end{description}
 
-After writing out some descriptors, both the device and the driver
+After writing out some descriptors,the device/driver
 are expected to consult the relevant structure to find out
-whether an interrupt/notification should be sent.
+whether an used/available buffer notification should be sent.
 
 \subsubsection{Structure Size and Alignment}
 \label{sec:Packed Virtqueues / Structure Size and Alignment}
@@ -463,7 +462,7 @@ When notifying the device, driver MUST set
 \field{next_off} and
 \field{next_wrap} to match the next descriptor
 not yet made available to the device.
-A driver MAY send multiple notifications without making
+A driver MAY send multiple available buffer notifications without making
 any new descriptors available to the device.
 
 \drivernormative{\subsection}{Scatter-Gather Support}{Basic Facilities of a
@@ -579,13 +578,14 @@ The driver MUST perform a suitable memory barrier before the
 \field{flags} update, to ensure the
 device sees the most up-to-date copy.
 
-\subsubsection{Notifying The Device}\label{sec:Basic Facilities
-of a Virtio Device / Packed Virtqueues / Supplying Buffers to The Device / Notifying The Device}
+\subsubsection{Sending Available Buffer Notifications}\label{sec:Basic Facilities
+of a Virtio Device / Packed Virtqueues / Supplying Buffers to The Device
+/ Sending Available Buffer Notifications}
 
 The actual method of device notification is bus-specific, but generally
 it can be expensive.  So the device MAY suppress such notifications if it
-doesn't need them, using the Driver Event Suppression structure
-as detailed in section \ref{sec:Basic
+doesn't need them, using the Event Suppression structure comprising the
+Device Area as detailed in section \ref{sec:Basic
 Facilities of a Virtio Device / Packed Virtqueues / Event
 Suppression Structure Format}.
 
@@ -595,7 +595,7 @@ value before checking if notifications are suppressed.
 \subsubsection{Implementation Example}\label{sec:Basic Facilities of a Virtio Device / Packed Virtqueues / Supplying Buffers to The Device / Implementation Example}
 
 Below is a driver code example. It does not attempt to reduce
-the number of device interrupts, neither does it support
+the number of available buffer notifications, neither does it support
 the VIRTIO_F_RING_EVENT_IDX feature.
 
 \begin{lstlisting}
@@ -651,24 +651,31 @@ if (vq->device_event.flags != RING_EVENT_FLAGS_DISABLE) {
 \end{lstlisting}
 
 
-\drivernormative{\paragraph}{Notifying The Device}{Basic Facilities of a Virtio Device / Packed Virtqueues / Supplying Buffers to The Device / Notifying The Device}
+\drivernormative{\paragraph}{Sending Available Buffer Notifications}
+{Basic Facilities of a Virtio Device / Packed Virtqueues / Supplying
+Buffers to The Device / Sending Available Buffer Notifications}
 The driver MUST perform a suitable memory barrier before reading
-the Driver Event Suppression structure, to avoid missing a notification.
+the Event Suppression structure occupying the Device Area. Failing
+to do so could result in mandatory available buffer notifications
+not being sent.
 
 \subsection{Receiving Used Buffers From The Device}\label{sec:Basic Facilities of a Virtio Device / Packed Virtqueues / Receiving Used Buffers From The Device}
 
 Once the device has used buffers referred to by a descriptor (read from or written to them, or
 parts of both, depending on the nature of the virtqueue and the
-device), it interrupts the driver
+device), it sends an used buffer notification to the driver
 as detailed in section \ref{sec:Basic
 Facilities of a Virtio Device / Packed Virtqueues / Event
 Suppression Structure Format}.
 
 \begin{note}
-For optimal performance, a driver MAY disable interrupts while processing
-the used buffers, but beware the problem of missing interrupts between
-emptying the ring and reenabling interrupts.  This is usually handled by
-re-checking for more used buffers after interrups are re-enabled:
+
+For optimal performance, a driver MAY disable used buffer notifications
+while processing the used buffers, but beware the problem of missing
+notifications between emptying the ring and reenabling used buffer
+notifications.  This is usually handled by re-checking for more used
+buffers after notifications are re-enabled:
+
 \end{note}
 
 \begin{lstlisting}
diff --git a/split-ring.tex b/split-ring.tex
index df278fe..9325b12 100644
--- a/split-ring.tex
+++ b/split-ring.tex
@@ -54,7 +54,8 @@ When the driver wants to send a buffer to the device, it fills in
 a slot in the descriptor table (or chains several together), and
 writes the descriptor index into the available ring.  It then
 notifies the device. When the device has finished a buffer, it
-writes the descriptor index into the used ring, and sends an interrupt.
+writes the descriptor index into the used ring, and sends an
+used buffer notification.
 
 \drivernormative{\subsection}{Virtqueues}{Basic Facilities of a Virtio Device / Virtqueues}
 The driver MUST ensure that the physical address of the first byte
@@ -318,44 +319,47 @@ VRING_AVAIL_F_NO_INTERRUPT, but the layout and value were identical.
 A driver MUST NOT decrement the available \field{idx} on a virtqueue (ie.
 there is no way to ``unexpose'' buffers).
 
-\subsection{Virtqueue Interrupt Suppression}\label{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}
+\subsection{Used Buffer Notification Suppression}\label{sec:Basic
+Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 
 If the VIRTIO_F_EVENT_IDX feature bit is not negotiated,
 the \field{flags} field in the available ring offers a crude mechanism for the driver to inform
-the device that it doesn't want interrupts when buffers are used.  Otherwise
+the device that it doesn't want notifications when buffers are used.  Otherwise
 \field{used_event} is a more performant alternative where the driver
-specifies how far the device can progress before interrupting.
+specifies how far the device can progress before a notification is
+required.
 
-Neither of these interrupt suppression methods are reliable, as they
+Neither of these notification suppression methods are reliable, as they
 are not synchronized with the device, but they serve as
 useful optimizations.
 
-\drivernormative{\subsubsection}{Virtqueue Interrupt Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}
+\drivernormative{\subsubsection}{Used Buffer Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:
 \begin{itemize}
 \item The driver MUST set \field{flags} to 0 or 1.
 \item The driver MAY set \field{flags} to 1 to advise
-the device that interrupts are not needed.
+the device that notifications are not needed.
 \end{itemize}
 
 Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:
 \begin{itemize}
 \item The driver MUST set \field{flags} to 0.
-\item The driver MAY use \field{used_event} to advise the device that interrupts are unnecessary until the device writes entry with an index specified by \field{used_event} into the used ring (equivalently, until \field{idx} in the
+\item The driver MAY use \field{used_event} to advise the device that
+notifications are unnecessary until the device writes entry with an index specified by \field{used_event} into the used ring (equivalently, until \field{idx} in the
 used ring will reach the value \field{used_event} + 1).
 \end{itemize}
 
-The driver MUST handle spurious interrupts from the device.
+The driver MUST handle spurious notifications from the device.
 
-\devicenormative{\subsubsection}{Virtqueue Interrupt Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}
+\devicenormative{\subsubsection}{Used Buffer Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}
 
 If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:
 \begin{itemize}
 \item The device MUST ignore the \field{used_event} value.
 \item After the device writes a descriptor index into the used ring:
   \begin{itemize}
-  \item If \field{flags} is 1, the device SHOULD NOT send an interrupt.
-  \item If \field{flags} is 0, the device MUST send an interrupt.
+  \item If \field{flags} is 1, the device SHOULD NOT send a notification.
+  \item If \field{flags} is 0, the device MUST send a notification.
   \end{itemize}
 \end{itemize}
 
@@ -366,14 +370,16 @@ Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:
   \begin{itemize}
   \item If the \field{idx} field in the used ring (which determined
     where that descriptor index was placed) was equal to
-    \field{used_event}, the device MUST send an interrupt.
-  \item Otherwise the device SHOULD NOT send an interrupt.
+    \field{used_event}, the device MUST send a notification.
+  \item Otherwise the device SHOULD NOT send a notification.
   \end{itemize}
 \end{itemize}
 
 \begin{note}
 For example, if \field{used_event} is 0, then a device using
-  VIRTIO_F_EVENT_IDX would interrupt after the first buffer is
+
+  VIRTIO_F_EVENT_IDX would send an used buffer notification
+  to the driver after the first buffer is
   used (and again after the 65536th buffer, etc).
 \end{note}
 
@@ -460,14 +466,16 @@ that uninitialized memory has been overwritten when it has not.
 The driver MUST NOT make assumptions about data in device-writable buffers
 beyond the first \field{len} bytes, and SHOULD ignore this data.
 
-\subsection{Virtqueue Notification Suppression}\label{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
+\subsection{Available Buffer Notification Suppression}\label{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
 
-The device can suppress notifications in a manner analogous to the way
-drivers can suppress interrupts as detailed in section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}.
+The device can suppress available buffer notifications in a manner
+analogous to the way drivers can suppress used buffer notifications as
+detailed in section \ref{sec:Basic Facilities of a Virtio Device /
+Virtqueues / Used Buffer Notification Suppression}.
 The device manipulates \field{flags} or \field{avail_event} in the used ring the
 same way the driver manipulates \field{flags} or \field{used_event} in the available ring.
 
-\drivernormative{\subsubsection}{Virtqueue Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
+\drivernormative{\subsubsection}{Available Buffer Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
 
 The driver MUST initialize \field{flags} in the used ring to 0 when
 allocating the used ring.
@@ -494,7 +502,7 @@ Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:
   \end{itemize}
 \end{itemize}
 
-\devicenormative{\subsubsection}{Virtqueue Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Notification Suppression}
+\devicenormative{\subsubsection}{Available Buffer Notification Suppression}{Basic Facilities of a Virtio Device / Virtqueues / Available Buffer Notification Suppression}
 If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:
 \begin{itemize}
 \item The device MUST set \field{flags} to 0 or 1.
@@ -562,8 +570,8 @@ The driver offers buffers to one of the device's virtqueues as follows:
 \item The driver performs a suitable memory barrier to ensure that it updates
   the \field{idx} field before checking for notification suppression.
 
-\item If notifications are not suppressed, the driver notifies the device
-    of the new available buffers.
+\item The driver sends an available buffers notification to the device if
+    such notifications are not suppressed.
 \end{enumerate}
 
 Note that the above code does not take precautions against the
@@ -659,26 +667,30 @@ The driver MUST perform a suitable memory barrier before reading \field{flags} o
 
 Once the device has used buffers referred to by a descriptor (read from or written to them, or
 parts of both, depending on the nature of the virtqueue and the
-device), it interrupts the driver as detailed in section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Virtqueue Interrupt Suppression}.
+device), it sends an used buffer notification to the driver as detailed
+in section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues /
+Used Buffer Notification Suppression}.
 
 \begin{note}
-For optimal performance, a driver MAY disable interrupts while processing
-the used ring, but beware the problem of missing interrupts between
-emptying the ring and reenabling interrupts.  This is usually handled by
-re-checking for more used buffers after interrups are re-enabled:
+
+For optimal performance, a driver MAY disable used buffer notificatios
+while processing the used ring, but beware the problem of missing
+notifications between emptying the ring and reenabling notifications.  This
+is usually handled by re-checking for more used buffers after
+notifications are re-enabled:
 
 \begin{lstlisting}
-virtq_disable_interrupts(vq);
+virtq_disable_notifications(vq);
 
 for (;;) {
         if (vq->last_seen_used != le16_to_cpu(virtq->used.idx)) {
-                virtq_enable_interrupts(vq);
+                virtq_enable_notifications(vq);
                 mb();
 
                 if (vq->last_seen_used != le16_to_cpu(virtq->used.idx))
                         break;
 
-                virtq_disable_interrupts(vq);
+                virtq_disable_notifications(vq);
         }
 
         struct virtq_used_elem *e = virtq.used->ring[vq->last_seen_used%vsz];
-- 
1.7.1


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] [RFC PATCH 2/3] notifications:notifications as basic virtio facility

[Halil Pasic]

Let's introduce notifications as basic virtio facility early on. This
shall not only increase the cohesion between core and transport
description by having a well-defined  place where notifications are
introduced, but also give us the opportunity to explain some
discrepancies.

Namely notifications send by the device to the driver were often called
interrupts prior v1.1. Getting completely rid of that terminology is
however not viable in case of some names.

Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
---
 content.tex |   31 +++++++++++++++++++++++++++++++
 1 files changed, 31 insertions(+), 0 deletions(-)

diff --git a/content.tex b/content.tex
index 4ccb823..87cc0e2 100644
--- a/content.tex
+++ b/content.tex
@@ -8,6 +8,7 @@ device consists of the following parts:
 \begin{itemize}
 \item Device status field
 \item Feature bits
+\item Notifications
 \item Device Configuration space
 \item One or more virtqueues
 \end{itemize}
@@ -150,6 +151,36 @@ requirements documented within these legacy interface sections.
 Specification text within these sections generally does not apply
 to non-transitional devices.
 
+\section{Notifications}\label{sec:Basic Facilities of a Virtio Device
+/ Notifications}
+
+The notion of sending a notification (driver to device or device
+to driver) plays an important role in this specification. The
+modus operandi of the notifications is transport specific.
+
+There are three types of notifications: 
+\begin{itemize}
+\item device configuration space notification
+\item available buffers notification
+\item used buffer notifications. 
+\end{itemize}
+
+The recipient of device configuration space notifications and use buffer
+notifications is the driver. These indicate that a configuration space
+has changed and that a buffer may have been made used on a virtqueue
+(designated by the notification) respectively. The recipient of available
+buffers notifications is the device. Such notifications indicates that a
+buffer may have been made available on a virtqueue (designated by the
+notification). The semantic as well as other important aspects  of the
+notifications is specified in detail in the following chapters.
+
+Most transports implement notifications sent by the device to the
+driver using interrupts. Due to this in previous versions of this
+specification, these notifications were often called interrupts.
+Some names defined in this specification still retain this interrupt
+terminology. Occasionally we also use the term event to refer to
+a notification or a receipt of a notification.
+
 \section{Device Configuration Space}\label{sec:Basic Facilities of a Virtio Device / Device Configuration Space}
 
 Device configuration space is generally used for rarely-changing or
-- 
1.7.1


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Halil Pasic]

The various notifications are introduced and specified in the common
(i.e. transport agnostic) portion of this specification. How
notifications are realised for a given transport is something each
transport has to specify.

Let's make the relationship between the virtio-ccw terms and the common
terms more obvious.

Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
---
 content.tex |   41 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 41 insertions(+), 0 deletions(-)

diff --git a/content.tex b/content.tex
index 87cc0e2..27aa17b 100644
--- a/content.tex
+++ b/content.tex
@@ -1938,6 +1938,17 @@ Bytes & Description & Contents \\
 \hline
 \end{tabular}
 
+A virtio-ccw proxy device facilitates:
+\begin{itemize} 
+\item Discovery and attachment of  virtio devices (as described above).
+\item Initialization of vitqueues and transport specific facilities (using
+      custom channel commands).
+\item Notifications (via hypercall and virtual interrupts).
+\end{itemize} 
+
+\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Channel Commands for Virtio}
+
 In addition to the basic channel commands, virtio-ccw defines a
 set of channel commands related to configuration and operation of
 virtio:
@@ -1958,6 +1969,36 @@ virtio:
 #define CCW_CMD_READ_STATUS 0x72
 \end{lstlisting}
 
+\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Notifications}
+
+Available buffer notifications are realized as a hypercall. No additional
+setup by the driver is needed. The operation of available buffer
+notifications is described in section \ref{sec:Virtio Transport Options /
+Virtio over channel I/O / Device Operation / Guest->Host Notification}.
+
+Used buffer notifications are realized either as so called classic or as
+adapter interrupts depending on a transport level negotiation. The
+initialization is described in sections \ref{sec:Virtio Transport Options
+/ Virtio over channel I/O / Device Initialization / Setting Up Indicators
+/ Setting Up Classic Queue Indicators} and \ref{sec:Virtio Transport
+Options / Virtio over channel I/O / Device Initialization / Setting Up
+Indicators / Setting Up Two-Stage Queue Indicators} respectively.  The
+operation of each flavor is described in sections \ref{sec:Virtio
+Transport Options / Virtio over channel I/O / Device Operation /
+Host->Guest Notification / Notification via Classic I/O Interrupts} and
+\ref{sec:Virtio Transport Options / Virtio over channel I/O / Device
+Operation / Host->Guest Notification / Notification via Adapter I/O
+Interrupts} respectively. 
+
+Configuration change notifications are done using so called classic
+interrupts. The initialization is described in section \ref{sec:Virtio
+Transport Options / Virtio over channel I/O / Device Initialization /
+Setting Up Indicators / Setting Up Configuration Change Indicators} and
+the operation in section \ref{sec:Virtio Transport Options / Virtio over
+channel I/O / Device Operation / Host->Guest Notification / Notification
+via Classic I/O Interrupts}.
+
 \devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio over channel I/O / Basic Concepts}
 
 The virtio-ccw device acts like a normal channel device, as specified
-- 
1.7.1


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Wed, Apr 11, 2018 at 12:11:25AM +0200, Halil Pasic wrote:
> Let us unify the wording when talking about notifications. This change
> establishes the terms available buffer notification for what was usually
> simply called notification or virtqueue notification in v1.0 and used
> buffer notification for what was usually called interrupt.
> 
> The term configuration change notification in kept where called so and
> consolidated where it's called configuration change interrupt or
> similar.
> 
> The changes done here are limited to the core part, and don't
> conceptually involve neither the transports nor the devices (references
> are updated though). Future changes should address these parts.

Nice, I think the cleanup is worthwhile.

>  \begin{lstlisting}
> -virtq_disable_interrupts(vq);
> +virtq_disable_notifications(vq);

This name is ambiguous.  Only used buffer notifications are disabled,
not configuration change notifications.

How about:

  virtq_disable_used_buffer_notifications(vq);

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

[virtio] Re: [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Cornelia Huck]

On Wed, 11 Apr 2018 00:11:27 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

[Have not yet looked at your other patches, on my list.]

> The various notifications are introduced and specified in the common
> (i.e. transport agnostic) portion of this specification. How
> notifications are realised for a given transport is something each
> transport has to specify.
> 
> Let's make the relationship between the virtio-ccw terms and the common
> terms more obvious.
> 
> Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
> ---
>  content.tex |   41 +++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 41 insertions(+), 0 deletions(-)
> 
> diff --git a/content.tex b/content.tex
> index 87cc0e2..27aa17b 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -1938,6 +1938,17 @@ Bytes & Description & Contents \\
>  \hline
>  \end{tabular}
>  
> +A virtio-ccw proxy device facilitates:
> +\begin{itemize} 
> +\item Discovery and attachment of  virtio devices (as described above).
> +\item Initialization of vitqueues and transport specific facilities (using

s/vitqueues/virtqueues/

> +      custom channel commands).

s/custom/virtio-specific/ ?

In a sense, all channel commands other than the basic ones are
'custom' :) They are always device or control unit type specific, only
obeying some rules.

> +\item Notifications (via hypercall and virtual interrupts).

Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes
both classic and adapter interrupts)?

> +\end{itemize} 
> +
> +\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
> +over channel I/O / Basic Concepts/ Channel Commands for Virtio}
> +
>  In addition to the basic channel commands, virtio-ccw defines a
>  set of channel commands related to configuration and operation of
>  virtio:
> @@ -1958,6 +1969,36 @@ virtio:
>  #define CCW_CMD_READ_STATUS 0x72
>  \end{lstlisting}
>  
> +\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
> +over channel I/O / Basic Concepts/ Notifications}
> +
> +Available buffer notifications are realized as a hypercall. No additional
> +setup by the driver is needed. The operation of available buffer
> +notifications is described in section \ref{sec:Virtio Transport Options /
> +Virtio over channel I/O / Device Operation / Guest->Host Notification}.
> +
> +Used buffer notifications are realized either as so called classic or as
> +adapter interrupts depending on a transport level negotiation. The

"as so-called classic or adapter I/O interrupts"?

(I'd really like a reference to I/O interrupts here... especially as
the old, never standardized s390 transport used external interrupts.)

> +initialization is described in sections \ref{sec:Virtio Transport Options
> +/ Virtio over channel I/O / Device Initialization / Setting Up Indicators
> +/ Setting Up Classic Queue Indicators} and \ref{sec:Virtio Transport
> +Options / Virtio over channel I/O / Device Initialization / Setting Up
> +Indicators / Setting Up Two-Stage Queue Indicators} respectively.  The
> +operation of each flavor is described in sections \ref{sec:Virtio
> +Transport Options / Virtio over channel I/O / Device Operation /
> +Host->Guest Notification / Notification via Classic I/O Interrupts} and
> +\ref{sec:Virtio Transport Options / Virtio over channel I/O / Device
> +Operation / Host->Guest Notification / Notification via Adapter I/O
> +Interrupts} respectively. 
> +
> +Configuration change notifications are done using so called classic
> +interrupts. The initialization is described in section \ref{sec:Virtio

"so-called classic I/O interrupts"

> +Transport Options / Virtio over channel I/O / Device Initialization /
> +Setting Up Indicators / Setting Up Configuration Change Indicators} and
> +the operation in section \ref{sec:Virtio Transport Options / Virtio over
> +channel I/O / Device Operation / Host->Guest Notification / Notification
> +via Classic I/O Interrupts}.
> +
>  \devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio over channel I/O / Basic Concepts}
>  
>  The virtio-ccw device acts like a normal channel device, as specified

In general, I like this.

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Halil Pasic]

On 04/11/2018 04:19 AM, Stefan Hajnoczi wrote:
> On Wed, Apr 11, 2018 at 12:11:25AM +0200, Halil Pasic wrote:
>> Let us unify the wording when talking about notifications. This change
>> establishes the terms available buffer notification for what was usually
>> simply called notification or virtqueue notification in v1.0 and used
>> buffer notification for what was usually called interrupt.
>>
>> The term configuration change notification in kept where called so and
>> consolidated where it's called configuration change interrupt or
>> similar.
>>
>> The changes done here are limited to the core part, and don't
>> conceptually involve neither the transports nor the devices (references
>> are updated though). Future changes should address these parts.
> 
> Nice, I think the cleanup is worthwhile.
> 

Thanks!

>>  \begin{lstlisting}
>> -virtq_disable_interrupts(vq);
>> +virtq_disable_notifications(vq);
> 
> This name is ambiguous.  Only used buffer notifications are disabled,
> not configuration change notifications.
>

One could argue that the only virtqueue scoped notification received
by the driver are used buffer notifications. That's why I thought
this would be sufficient (btw. if it's ambiguous it was ambiguous before
too as both options are called interrupts).

But the more straight-forward the better.
 
> How about:
> 
>   virtq_disable_used_buffer_notifications(vq);
> 

In absence of further discussion I will take this for the next
version.

Thanks again!

Regards,
Halil


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Cornelia Huck]

On Wed, 11 Apr 2018 12:58:37 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> On 04/11/2018 04:19 AM, Stefan Hajnoczi wrote:
> > On Wed, Apr 11, 2018 at 12:11:25AM +0200, Halil Pasic wrote:  

> >>  \begin{lstlisting}
> >> -virtq_disable_interrupts(vq);
> >> +virtq_disable_notifications(vq);  
> > 
> > This name is ambiguous.  Only used buffer notifications are disabled,
> > not configuration change notifications.
> >  
> 
> One could argue that the only virtqueue scoped notification received
> by the driver are used buffer notifications. That's why I thought
> this would be sufficient (btw. if it's ambiguous it was ambiguous before
> too as both options are called interrupts).
> 
> But the more straight-forward the better.

Agreed.

>  
> > How about:
> > 
> >   virtq_disable_used_buffer_notifications(vq);
> >   
> 
> In absence of further discussion I will take this for the next
> version.

Sounds good to me.

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [virtio-dev] Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Paolo Bonzini]

[-- Attachment #1.1: Type: text/plain, Size: 1362 bytes --]

On 11/04/2018 04:19, Stefan Hajnoczi wrote:
> On Wed, Apr 11, 2018 at 12:11:25AM +0200, Halil Pasic wrote:
>> Let us unify the wording when talking about notifications. This change
>> establishes the terms available buffer notification for what was usually
>> simply called notification or virtqueue notification in v1.0 and used
>> buffer notification for what was usually called interrupt.
>>
>> The term configuration change notification in kept where called so and
>> consolidated where it's called configuration change interrupt or
>> similar.
>>
>> The changes done here are limited to the core part, and don't
>> conceptually involve neither the transports nor the devices (references
>> are updated though). Future changes should address these parts.
> 
> Nice, I think the cleanup is worthwhile.

I agree.  I wondered if we should use the term "used buffer interrupt"
and "available buffer notification".  In the common case I think it
would be clearer, though there are cases such as vhost-pci where the
roles are swapped.

Paolo

>>  \begin{lstlisting}
>> -virtq_disable_interrupts(vq);
>> +virtq_disable_notifications(vq);
> 
> This name is ambiguous.  Only used buffer notifications are disabled,
> not configuration change notifications.
> 
> How about:
> 
>   virtq_disable_used_buffer_notifications(vq);
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[virtio] Re: [virtio-dev] Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Cornelia Huck]

On Wed, 11 Apr 2018 14:35:23 +0200
Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 11/04/2018 04:19, Stefan Hajnoczi wrote:
> > On Wed, Apr 11, 2018 at 12:11:25AM +0200, Halil Pasic wrote:  
> >> Let us unify the wording when talking about notifications. This change
> >> establishes the terms available buffer notification for what was usually
> >> simply called notification or virtqueue notification in v1.0 and used
> >> buffer notification for what was usually called interrupt.
> >>
> >> The term configuration change notification in kept where called so and
> >> consolidated where it's called configuration change interrupt or
> >> similar.
> >>
> >> The changes done here are limited to the core part, and don't
> >> conceptually involve neither the transports nor the devices (references
> >> are updated though). Future changes should address these parts.  
> > 
> > Nice, I think the cleanup is worthwhile.  
> 
> I agree.  I wondered if we should use the term "used buffer interrupt"
> and "available buffer notification".  In the common case I think it
> would be clearer, though there are cases such as vhost-pci where the
> roles are swapped.

If it is swapped in some cases, it is bound to cause confusion for
those. I'd vote for using "notification" in both cases, as this patch
is doing.

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [virtio-dev] Re: [virtio] [RFC PATCH 1/3] notifications: unify notifications wording in core

[Paolo Bonzini]

On 11/04/2018 14:55, Cornelia Huck wrote:
>>> Nice, I think the cleanup is worthwhile.  
>> I agree.  I wondered if we should use the term "used buffer interrupt"
>> and "available buffer notification".  In the common case I think it
>> would be clearer, though there are cases such as vhost-pci where the
>> roles are swapped.
>
> If it is swapped in some cases, it is bound to cause confusion for
> those. I'd vote for using "notification" in both cases, as this patch
> is doing.

Fair enough!  I think I agree myself. :)

Paolo

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [RFC PATCH 2/3] notifications:notifications as basic virtio facility

[Cornelia Huck]

On Wed, 11 Apr 2018 00:11:26 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> Let's introduce notifications as basic virtio facility early on. This
> shall not only increase the cohesion between core and transport
> description by having a well-defined  place where notifications are
> introduced, but also give us the opportunity to explain some
> discrepancies.
> 
> Namely notifications send by the device to the driver were often called

s/send/sent/

> interrupts prior v1.1. Getting completely rid of that terminology is

s/prior/prior to/

> however not viable in case of some names.
> 
> Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
> ---
>  content.tex |   31 +++++++++++++++++++++++++++++++
>  1 files changed, 31 insertions(+), 0 deletions(-)
> 
> diff --git a/content.tex b/content.tex
> index 4ccb823..87cc0e2 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -8,6 +8,7 @@ device consists of the following parts:
>  \begin{itemize}
>  \item Device status field
>  \item Feature bits
> +\item Notifications
>  \item Device Configuration space
>  \item One or more virtqueues
>  \end{itemize}
> @@ -150,6 +151,36 @@ requirements documented within these legacy interface sections.
>  Specification text within these sections generally does not apply
>  to non-transitional devices.
>  
> +\section{Notifications}\label{sec:Basic Facilities of a Virtio Device
> +/ Notifications}
> +
> +The notion of sending a notification (driver to device or device
> +to driver) plays an important role in this specification. The
> +modus operandi of the notifications is transport specific.
> +
> +There are three types of notifications: 
> +\begin{itemize}
> +\item device configuration space notification
> +\item available buffers notification
> +\item used buffer notifications. 
> +\end{itemize}
> +
> +The recipient of device configuration space notifications and use buffer

s/use/used/

> +notifications is the driver. These indicate that a configuration space
> +has changed and that a buffer may have been made used on a virtqueue
> +(designated by the notification) respectively. The recipient of available
> +buffers notifications is the device. Such notifications indicates that a

s/indicates/indicate/

> +buffer may have been made available on a virtqueue (designated by the
> +notification). The semantic as well as other important aspects  of the
> +notifications is specified in detail in the following chapters.

Hm... I'd reword that a bit, and I'd not focus explicitly on the
recipient. What about:

"Device configuration space notifications and used buffer notifications
are sent by the device, the recipient is the driver. A device
configuration space notification indicates that a configuration space
has changed; a used buffer notification indicates that a buffer may
have been made used on the virtqueue designated by the notification.

Available buffer notifications are sent by the driver, the recipient is
the device. This type of notification indicates that a buffer may have
been made available on the virtqueue designated by the notification.

The semantic, the transport-specific implementations and other
important aspects of the different notifications are specified in detail
in the following chapters."

> +
> +Most transports implement notifications sent by the device to the
> +driver using interrupts. Due to this in previous versions of this

s/Due to this/Therefore,/ ?

> +specification, these notifications were often called interrupts.
> +Some names defined in this specification still retain this interrupt
> +terminology. Occasionally we also use the term event to refer to
> +a notification or a receipt of a notification.

"Occasionally, the term 'event' is used..." (I'd like to avoid 'we'.)

> +
>  \section{Device Configuration Space}\label{sec:Basic Facilities of a Virtio Device / Device Configuration Space}
>  
>  Device configuration space is generally used for rarely-changing or


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [virtio-dev] Re: [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Halil Pasic]

On 04/11/2018 09:50 AM, Cornelia Huck wrote:
> On Wed, 11 Apr 2018 00:11:27 +0200
> Halil Pasic <pasic@linux.vnet.ibm.com> wrote:
> 
> [Have not yet looked at your other patches, on my list.]
> 
>> The various notifications are introduced and specified in the common
>> (i.e. transport agnostic) portion of this specification. How
>> notifications are realised for a given transport is something each
>> transport has to specify.
>>
>> Let's make the relationship between the virtio-ccw terms and the common
>> terms more obvious.
>>
>> Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
>> ---
>>  content.tex |   41 +++++++++++++++++++++++++++++++++++++++++
>>  1 files changed, 41 insertions(+), 0 deletions(-)
>>
>> diff --git a/content.tex b/content.tex
>> index 87cc0e2..27aa17b 100644
>> --- a/content.tex
>> +++ b/content.tex
>> @@ -1938,6 +1938,17 @@ Bytes & Description & Contents \\
>>  \hline
>>  \end{tabular}
>>  
>> +A virtio-ccw proxy device facilitates:
>> +\begin{itemize} 
>> +\item Discovery and attachment of  virtio devices (as described above).
>> +\item Initialization of vitqueues and transport specific facilities (using
> 
> s/vitqueues/virtqueues/
> 

Will do.

>> +      custom channel commands).
> 
> s/custom/virtio-specific/ ?

I think it's better -- more formal.

> 
> In a sense, all channel commands other than the basic ones are
> 'custom' :) They are always device or control unit type specific, only
> obeying some rules.
> 

Agree. So custom is in that sense accurate, but virtio-specific rings
nicer.

>> +\item Notifications (via hypercall and virtual interrupts).
> 
> Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes
> both classic and adapter interrupts)?
> 

The idea was 'hypercall' and 'virtual' should harmonize well. These
I/O interrupts are kind of 'real' from the perspective of the virtual
machine, but are 'virtual' from the perspective of HW and AR perspective. 

What I mean, there is AFAIU no way to implement a control unit
and device combo in HW attach it to a z box and do virtio over CIO
naively.

Even with classic I/O interrupts we have to do set indicator + inject
subchannel interrupt to realize a notification. This is however
form core perspective one notification/even/interrupt.

So this is why I added this 'virtual' (to hint it may not fit anything
one can find in the PoP perfectly).

>> +\end{itemize} 
>> +
>> +\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
>> +over channel I/O / Basic Concepts/ Channel Commands for Virtio}
>> +
>>  In addition to the basic channel commands, virtio-ccw defines a
>>  set of channel commands related to configuration and operation of
>>  virtio:
>> @@ -1958,6 +1969,36 @@ virtio:
>>  #define CCW_CMD_READ_STATUS 0x72
>>  \end{lstlisting}
>>  
>> +\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
>> +over channel I/O / Basic Concepts/ Notifications}
>> +
>> +Available buffer notifications are realized as a hypercall. No additional
>> +setup by the driver is needed. The operation of available buffer
>> +notifications is described in section \ref{sec:Virtio Transport Options /
>> +Virtio over channel I/O / Device Operation / Guest->Host Notification}.
>> +
>> +Used buffer notifications are realized either as so called classic or as
>> +adapter interrupts depending on a transport level negotiation. The
> 
> "as so-called classic or adapter I/O interrupts"?

Valid. These are indeed called 'adapter I/O interrupts' through out
this spec. I was i a hurry to write up something demonstrating he idea,
so I did not check this. I think these are usually called 'adapter interrupts'
(without the I/O in between) elsewhere, but internal integrity is more
important.

I will take it.

> 
> (I'd really like a reference to I/O interrupts here... especially as
> the old, never standardized s390 transport used external interrupts.)
> 

You mean with the wording you proposed, or something more? If something
more could you do a patch on top (later)?

>> +initialization is described in sections \ref{sec:Virtio Transport Options
>> +/ Virtio over channel I/O / Device Initialization / Setting Up Indicators
>> +/ Setting Up Classic Queue Indicators} and \ref{sec:Virtio Transport
>> +Options / Virtio over channel I/O / Device Initialization / Setting Up
>> +Indicators / Setting Up Two-Stage Queue Indicators} respectively.  The
>> +operation of each flavor is described in sections \ref{sec:Virtio
>> +Transport Options / Virtio over channel I/O / Device Operation /
>> +Host->Guest Notification / Notification via Classic I/O Interrupts} and
>> +\ref{sec:Virtio Transport Options / Virtio over channel I/O / Device
>> +Operation / Host->Guest Notification / Notification via Adapter I/O
>> +Interrupts} respectively. 
>> +
>> +Configuration change notifications are done using so called classic
>> +interrupts. The initialization is described in section \ref{sec:Virtio
> 
> "so-called classic I/O interrupts"
> 

Same here. Will do.

>> +Transport Options / Virtio over channel I/O / Device Initialization /
>> +Setting Up Indicators / Setting Up Configuration Change Indicators} and
>> +the operation in section \ref{sec:Virtio Transport Options / Virtio over
>> +channel I/O / Device Operation / Host->Guest Notification / Notification
>> +via Classic I/O Interrupts}.
>> +
>>  \devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio over channel I/O / Basic Concepts}
>>  
>>  The virtio-ccw device acts like a normal channel device, as specified
> 
> In general, I like this.
> 

Thanks. I intend to wait for Michael's word before jumping a the other
transports to get the series non-rfc ready. Up till now reviews seem
favorable: I guess it's likely to happen.

Thanks for your valuable input!

Regards,
Halil


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [virtio-dev] Re: [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Cornelia Huck]

On Wed, 11 Apr 2018 15:42:34 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> On 04/11/2018 09:50 AM, Cornelia Huck wrote:
> > On Wed, 11 Apr 2018 00:11:27 +0200
> > Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> >> +\item Notifications (via hypercall and virtual interrupts).  
> > 
> > Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes
> > both classic and adapter interrupts)?
> >   
> 
> The idea was 'hypercall' and 'virtual' should harmonize well. These
> I/O interrupts are kind of 'real' from the perspective of the virtual
> machine, but are 'virtual' from the perspective of HW and AR perspective. 

Yes, but that's an implementation detail. I/O interrupts follow the
same architecture in any case, there's nothing special about I/O
interrupts for virtio.

> 
> What I mean, there is AFAIU no way to implement a control unit
> and device combo in HW attach it to a z box and do virtio over CIO
> naively.

It does not seem completely impossible (I/O interrupts are abstractions
already). The diagnose notification might be a problem, though :)

> 
> Even with classic I/O interrupts we have to do set indicator + inject
> subchannel interrupt to realize a notification. This is however
> form core perspective one notification/even/interrupt.

But the I/O interrupt + indicators combination already exits (cf.
QDIO). I don't think we should single out virtio.

> 
> So this is why I added this 'virtual' (to hint it may not fit anything
> one can find in the PoP perfectly).

I certainly would welcome addition of the adapter interrupt
architecture to the PoP :)

> 
> >> +\end{itemize} 
> >> +
> >> +\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
> >> +over channel I/O / Basic Concepts/ Channel Commands for Virtio}
> >> +
> >>  In addition to the basic channel commands, virtio-ccw defines a
> >>  set of channel commands related to configuration and operation of
> >>  virtio:
> >> @@ -1958,6 +1969,36 @@ virtio:
> >>  #define CCW_CMD_READ_STATUS 0x72
> >>  \end{lstlisting}
> >>  
> >> +\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
> >> +over channel I/O / Basic Concepts/ Notifications}
> >> +
> >> +Available buffer notifications are realized as a hypercall. No additional
> >> +setup by the driver is needed. The operation of available buffer
> >> +notifications is described in section \ref{sec:Virtio Transport Options /
> >> +Virtio over channel I/O / Device Operation / Guest->Host Notification}.
> >> +
> >> +Used buffer notifications are realized either as so called classic or as
> >> +adapter interrupts depending on a transport level negotiation. The  
> > 
> > "as so-called classic or adapter I/O interrupts"?  
> 
> Valid. These are indeed called 'adapter I/O interrupts' through out
> this spec. I was i a hurry to write up something demonstrating he idea,
> so I did not check this. I think these are usually called 'adapter interrupts'
> (without the I/O in between) elsewhere, but internal integrity is more
> important.
> 
> I will take it.
> 
> > 
> > (I'd really like a reference to I/O interrupts here... especially as
> > the old, never standardized s390 transport used external interrupts.)
> >   
> 
> You mean with the wording you proposed, or something more? If something
> more could you do a patch on top (later)?

I think simply adding "I/O" should be enough.

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Re: [virtio] Re: [virtio-dev] Re: [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Halil Pasic]

On 04/11/2018 06:00 PM, Cornelia Huck wrote:
> On Wed, 11 Apr 2018 15:42:34 +0200
> Halil Pasic <pasic@linux.vnet.ibm.com> wrote:
> 
>> On 04/11/2018 09:50 AM, Cornelia Huck wrote:
>>> On Wed, 11 Apr 2018 00:11:27 +0200
>>> Halil Pasic <pasic@linux.vnet.ibm.com> wrote:
> 
>>>> +\item Notifications (via hypercall and virtual interrupts).  
>>>
>>> Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes
>>> both classic and adapter interrupts)?
>>>   
>>
>> The idea was 'hypercall' and 'virtual' should harmonize well. These
>> I/O interrupts are kind of 'real' from the perspective of the virtual
>> machine, but are 'virtual' from the perspective of HW and AR perspective. 
> 
> Yes, but that's an implementation detail. I/O interrupts follow the
> same architecture in any case, there's nothing special about I/O
> interrupts for virtio.
> 

I'm not sure about that. For instance, there is this check your
notifiers unsolicited subchannel associated I/O interruption. I don't
think this is just plain old channel I/O interrupt handling.

>>
>> What I mean, there is AFAIU no way to implement a control unit
>> and device combo in HW attach it to a z box and do virtio over CIO
>> naively.
> 
> It does not seem completely impossible (I/O interrupts are abstractions
> already). The diagnose notification might be a problem, though :)
> 

I don't understand. How would the control unit/device (let's say CU
connected to an FCP feature) sent 'classic' indicators?

>>
>> Even with classic I/O interrupts we have to do set indicator + inject
>> subchannel interrupt to realize a notification. This is however
>> form core perspective one notification/even/interrupt.
> 
> But the I/O interrupt + indicators combination already exits (cf.
> QDIO). I don't think we should single out virtio.
> 

Consider the reader of this spec. QDIO is not PoP material. I don't
think the virtio-ccw (as specified here) can be implemented over it either.
My understanding of QDIO is very limited. With that limited understanding
I would say that there are some similarities, but that is it.

Bottom line is the following. The statement 'A driver receiving a virtio
configuration change notifications corresponds to a (classic) I/O 
interruption for the subchannel corresponding to the  virtio-ccw proxy 
device.' is at best an oversimplification for me. (In fact making a I/O
interrupt pending can be optimized away under certain circumstances.)

That is why I used 'Notifications (via hypercall and virtual interrupts).'
instead of 'Notifications (via hypercall and I/O interrupts)'. IMHO the first
one points the people into the right direction, while remaining blurry
enough, and hinting think virtual (that is not strictly architecture).

AFAIU other transports don't have this 'can not be implemented
natively staying within the present architectural boundaries'.

But you seem bothered by this 'virtual' and I respect that. So how about
'Notifications (via hypercall and a combination of I/O interrupts and
indicator bits).'?

>>
>> So this is why I added this 'virtual' (to hint it may not fit anything
>> one can find in the PoP perfectly).
> 
> I certainly would welcome addition of the adapter interrupt
> architecture to the PoP :)
> 

I've adopted a policy of acting on the assumption that the
company has good reasons for publishing what is published
and keeping unpublished what is not. It would be certainly
easier for me to do my work, if all architecture were public.

>>
>>>> +\end{itemize} 
>>>> +
>>>> +\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
>>>> +over channel I/O / Basic Concepts/ Channel Commands for Virtio}
>>>> +
>>>>  In addition to the basic channel commands, virtio-ccw defines a
>>>>  set of channel commands related to configuration and operation of
>>>>  virtio:
>>>> @@ -1958,6 +1969,36 @@ virtio:
>>>>  #define CCW_CMD_READ_STATUS 0x72
>>>>  \end{lstlisting}
>>>>  
>>>> +\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
>>>> +over channel I/O / Basic Concepts/ Notifications}
>>>> +
>>>> +Available buffer notifications are realized as a hypercall. No additional
>>>> +setup by the driver is needed. The operation of available buffer
>>>> +notifications is described in section \ref{sec:Virtio Transport Options /
>>>> +Virtio over channel I/O / Device Operation / Guest->Host Notification}.
>>>> +
>>>> +Used buffer notifications are realized either as so called classic or as
>>>> +adapter interrupts depending on a transport level negotiation. The  
>>>
>>> "as so-called classic or adapter I/O interrupts"?  
>>
>> Valid. These are indeed called 'adapter I/O interrupts' through out
>> this spec. I was i a hurry to write up something demonstrating he idea,
>> so I did not check this. I think these are usually called 'adapter interrupts'
>> (without the I/O in between) elsewhere, but internal integrity is more
>> important.
>>
>> I will take it.
>>
>>>
>>> (I'd really like a reference to I/O interrupts here... especially as
>>> the old, never standardized s390 transport used external interrupts.)
>>>   
>>
>> You mean with the wording you proposed, or something more? If something
>> more could you do a patch on top (later)?
> 
> I think simply adding "I/O" should be enough.
> 

OK.

Regards,
Halil


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

Re: [virtio] Re: [virtio-dev] Re: [RFC PATCH 3/3] ccw: map common notifications terminology to ccw

[Cornelia Huck]

On Thu, 12 Apr 2018 13:12:09 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> On 04/11/2018 06:00 PM, Cornelia Huck wrote:
> > On Wed, 11 Apr 2018 15:42:34 +0200
> > Halil Pasic <pasic@linux.vnet.ibm.com> wrote:
> >   
> >> On 04/11/2018 09:50 AM, Cornelia Huck wrote:  
> >>> On Wed, 11 Apr 2018 00:11:27 +0200
> >>> Halil Pasic <pasic@linux.vnet.ibm.com> wrote:  
> >   
> >>>> +\item Notifications (via hypercall and virtual interrupts).    
> >>>
> >>> Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes
> >>> both classic and adapter interrupts)?
> >>>     
> >>
> >> The idea was 'hypercall' and 'virtual' should harmonize well. These
> >> I/O interrupts are kind of 'real' from the perspective of the virtual
> >> machine, but are 'virtual' from the perspective of HW and AR perspective.   
> > 
> > Yes, but that's an implementation detail. I/O interrupts follow the
> > same architecture in any case, there's nothing special about I/O
> > interrupts for virtio.
> >   
> 
> I'm not sure about that. For instance, there is this check your
> notifiers unsolicited subchannel associated I/O interruption. I don't
> think this is just plain old channel I/O interrupt handling.

I/O interrupts are I/O interrupts, just as in the architecture.
Indicators are additional data, and there's even precedence for that
(that's why I chose that approach in the first place).

[The fact that nothing about those other indicator users is documented
in public beyond some Linux et al. source code is not really the
problem of this spec. And I'm sure people who know me are aware of my
opinions in that regard; so let's not go down that particular rabbit
hole.]

> But you seem bothered by this 'virtual' and I respect that. So how about
> 'Notifications (via hypercall and a combination of I/O interrupts and
> indicator bits).'?

Fine with me.

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [RFC PATCH 0/3] rework notifications terminology

[Halil Pasic]

ping

@Michael: Any feedback from you or should I prepare a non-rfc that
deals with the other transports too.

On 04/11/2018 12:11 AM, Halil Pasic wrote:
> While discussing XXXX using available and used buffer notifications

s;XXXX;'https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg02774.html

> consistently for virqueue notifications (driver to device and vice versa
> respectively) seemed like a good idea.
> 
> It turned out surprisingly invasive however, and I find it necessary to
> confirm, this is really the path we want to walk, before investing even more
> time.
> 
> This series limits itself to the core and to the ccw transport. It ain't
> typical to the device types to make statements about notifications, but at
> least net has some words on interrupts. I did not get these immediately so I've
> left that out for now.
> 
> I choose ccw as demonstrator on how to bridge the abstract with the transport
> specific, because that is the transport I'm most familiar with. I do not expect
> difficulties with the other ones.
> 
> 
> Halil Pasic (3):
>    notifications: unify notifications wording in core
>    notifications:notifications as basic virtio facility
>    ccw: map common notifications terminology to ccw
> 
>   cl-os.tex       |    2 +-
>   conformance.tex |    8 ++--
>   content.tex     |   92 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
>   packed-ring.tex |   59 +++++++++++++++++++---------------
>   split-ring.tex  |   72 +++++++++++++++++++++++++------------------
>   5 files changed, 164 insertions(+), 69 deletions(-)
> 


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 

[virtio] Re: [RFC PATCH 0/3] rework notifications terminology

[Michael S. Tsirkin]

On Fri, Apr 20, 2018 at 04:53:24PM +0200, Halil Pasic wrote:
> ping
> 
> @Michael: Any feedback from you or should I prepare a non-rfc that
> deals with the other transports too.

Looks good to me pls go ahead.

> On 04/11/2018 12:11 AM, Halil Pasic wrote:
> > While discussing XXXX using available and used buffer notifications
> 
> s;XXXX;'https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg02774.html
> 
> > consistently for virqueue notifications (driver to device and vice versa
> > respectively) seemed like a good idea.
> > 
> > It turned out surprisingly invasive however, and I find it necessary to
> > confirm, this is really the path we want to walk, before investing even more
> > time.
> > 
> > This series limits itself to the core and to the ccw transport. It ain't
> > typical to the device types to make statements about notifications, but at
> > least net has some words on interrupts. I did not get these immediately so I've
> > left that out for now.
> > 
> > I choose ccw as demonstrator on how to bridge the abstract with the transport
> > specific, because that is the transport I'm most familiar with. I do not expect
> > difficulties with the other ones.
> > 
> > 
> > Halil Pasic (3):
> >    notifications: unify notifications wording in core
> >    notifications:notifications as basic virtio facility
> >    ccw: map common notifications terminology to ccw
> > 
> >   cl-os.tex       |    2 +-
> >   conformance.tex |    8 ++--
> >   content.tex     |   92 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
> >   packed-ring.tex |   59 +++++++++++++++++++---------------
> >   split-ring.tex  |   72 +++++++++++++++++++++++++------------------
> >   5 files changed, 164 insertions(+), 69 deletions(-)
> > 

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php