* [virtio-dev] [PATCH] snd: Add virtio sound device specification
@ 2019-09-24 14:43 Mikhail Golubev
2019-10-28 16:05 ` Liam Girdwood
0 siblings, 1 reply; 42+ messages in thread
From: Mikhail Golubev @ 2019-09-24 14:43 UTC (permalink / raw)
To: virtio-dev; +Cc: Anton Yakovlev
From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
This patch proposes virtio specification for new virtio sound device.
The virtio sound card is a virtual audio device supporting output and
input PCM streams.
Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
---
Fixes from RFC:
* Use fixed virtqueue numbers for the control and PCM queues
* Use device feature bits to indicate available input/output PCM streams
* Place PCM stream configuration into the device configuration space
* Add sound device and driver requirements into the "Device Initialization"
subsection
* Add sound device and driver requirements into the "Device Operation" subsection
* Add sound device and driver conformance subsections
RFC link:
https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
conformance.tex | 20 +++
content.tex | 1 +
virtio-sound.tex | 405 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 426 insertions(+)
create mode 100644 virtio-sound.tex
diff --git a/conformance.tex b/conformance.tex
index 0ac58aa..31595bf 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -183,6 +183,16 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item \ref{drivernormative:Device Types / Socket Device / Device Operation / Device Events}
\end{itemize}
+\conformance{\subsection}{Sound Driver Conformance}\label{sec:Conformance / Driver Conformance / Sound Driver Conformance}
+
+A sound driver MUST conform to the following normative statements:
+
+\begin{itemize}
+\item \ref{drivernormative:Device Types / Sound Device / Device Initialization}
+\item \ref{drivernormative:Device Types / Sound Device / PCM control requests}
+\item \ref{drivernormative:Device Types / Sound Device / PCM IO requests}
+\end{itemize}
+
\conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance}
A device MUST conform to the following normative statements:
@@ -338,6 +348,16 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item \ref{devicenormative:Device Types / Socket Device / Device Operation / Receive and Transmit}
\end{itemize}
+\conformance{\subsection}{Sound Device Conformance}\label{sec:Conformance / Device Conformance / Sound Device Conformance}
+
+A sound device MUST conform to the following normative statements:
+
+\begin{itemize}
+\item \ref{devicenormative:Device Types / Sound Device / Device Initialization}
+\item \ref{devicenormative:Device Types / Sound Device / PCM control requests}
+\item \ref{devicenormative:Device Types / Sound Device / PCM IO requests}
+\end{itemize}
+
\conformance{\section}{Legacy Interface: Transitional Device and Transitional Driver Conformance}\label{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance}
A conformant implementation MUST be either transitional or
non-transitional, see \ref{intro:Legacy
diff --git a/content.tex b/content.tex
index 37a2190..4a00884 100644
--- a/content.tex
+++ b/content.tex
@@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
\input{virtio-input.tex}
\input{virtio-crypto.tex}
\input{virtio-vsock.tex}
+\input{virtio-sound.tex}
\chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
diff --git a/virtio-sound.tex b/virtio-sound.tex
new file mode 100644
index 0000000..68a606e
--- /dev/null
+++ b/virtio-sound.tex
@@ -0,0 +1,405 @@
+\section{Sound Device}\label{sec:Device Types / Sound Device}
+
+The virtio sound card is a virtual audio device supporting output and input PCM
+streams. All device control requests are placed into the control virtqueue, all
+I/O requests are placed into the PCM virtqueue.
+
+\subsection{Device ID}\label{sec:Device Types / Sound Device / Device ID}
+
+25
+
+\subsection{Virtqueues}\label{sec:Device Types / Sound Device / Virtqueues}
+
+\begin{description}
+\item[0] controlq
+\item[1] pcmq
+\end{description}
+
+The controlq virtqueue always exists, the pcmq virtqueue only exists if
+the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT feature is negotiated.
+
+\subsection{Feature bits}\label{sec:Device Types / Sound Device / Feature bits}
+
+\begin{description}
+\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
+\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
+\end{description}
+
+\subsection{Device configuration layout}\label{sec:Device Types / Sound Device / Device configuration layout}
+
+\begin{lstlisting}
+/* supported PCM sample formats */
+enum {
+ VIRTIO_SND_PCM_FMT_S8 = 0,
+ VIRTIO_SND_PCM_FMT_U8,
+ VIRTIO_SND_PCM_FMT_S16,
+ VIRTIO_SND_PCM_FMT_U16,
+ VIRTIO_SND_PCM_FMT_S32,
+ VIRTIO_SND_PCM_FMT_U32,
+ VIRTIO_SND_PCM_FMT_FLOAT,
+ VIRTIO_SND_PCM_FMT_FLOAT64
+};
+
+/* supported PCM frame rates */
+enum {
+ VIRTIO_SND_PCM_RATE_8000 = 0,
+ VIRTIO_SND_PCM_RATE_11025,
+ VIRTIO_SND_PCM_RATE_16000,
+ VIRTIO_SND_PCM_RATE_22050,
+ VIRTIO_SND_PCM_RATE_32000,
+ VIRTIO_SND_PCM_RATE_44100,
+ VIRTIO_SND_PCM_RATE_48000,
+ VIRTIO_SND_PCM_RATE_64000,
+ VIRTIO_SND_PCM_RATE_88200,
+ VIRTIO_SND_PCM_RATE_96000,
+ VIRTIO_SND_PCM_RATE_176400,
+ VIRTIO_SND_PCM_RATE_192000
+};
+
+/* a PCM stream configuration */
+struct virtio_pcm_stream_config {
+ u8 channels_min;
+ u8 channels_max;
+ le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
+ le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
+ u16 padding;
+};
+
+/* a device configuration space */
+struct virtio_snd_config {
+ struct virtio_pcm_config {
+ struct virtio_pcm_stream_config output;
+ struct virtio_pcm_stream_config input;
+ } pcm;
+};
+\end{lstlisting}
+
+\subsubsection{Device configuration fields}
+
+The \field{pcm.output} and \field{pcm.input} fields contain PCM stream
+configuration:
+
+\begin{description}
+\item[\field{channels_min}] (driver-read-only) is a minimum number of supported
+channels.
+\item[\field{channels_max}] (driver-read-only) is a maximum number of supported
+channels.
+\item[\field{formats}] (driver-read-only) is supported sample format bit map.
+\item[\field{rates}] (driver-read-only) is supported frame rate bit map.
+\end{description}
+
+\subsection{Device Initialization}
+
+\begin{enumerate}
+\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated, \field{pcm.output}
+contains valid output PCM stream configuration.
+\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated, \field{pcm.input}
+contains valid input PCM stream configuration.
+\end{enumerate}
+
+\devicenormative{\subsubsection}{Device Initialization}{Device Types / Sound Device / Device Initialization}
+
+\begin{enumerate}
+\item The device MUST NOT set the not defined format and rate bits.
+\item The device MUST initialize padding bytes \field{pcm.output.padding} and
+\field{pcm.input.padding} to 0.
+\end{enumerate}
+
+\drivernormative{\subsubsection}{Device Initialization}{Device Types / Sound Device / Device Initialization}
+
+\begin{enumerate}
+\item The driver MUST configure and initialize all virtqueues.
+\item The driver SHOULD ignore the not defined format and rate bits.
+\end{enumerate}
+
+\subsection{Device Operation}\label{sec:Device Types / Sound Device / Device Operation}
+
+All control messages are placed into the controlq virtqueue and use the following
+layout structure and definitions:
+
+\begin{lstlisting}
+enum {
+ /* PCM control request types */
+ VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
+ VIRTIO_SND_R_PCM_SET_FORMAT,
+ VIRTIO_SND_R_PCM_PREPARE,
+ VIRTIO_SND_R_PCM_START,
+ VIRTIO_SND_R_PCM_STOP,
+ VIRTIO_SND_R_PCM_PAUSE,
+ VIRTIO_SND_R_PCM_UNPAUSE,
+
+ /* generic status codes */
+ VIRTIO_SND_S_OK = 0x8000,
+ VIRTIO_SND_S_BAD_MSG,
+ VIRTIO_SND_S_NOT_SUPP,
+ VIRTIO_SND_S_IO_ERR
+};
+
+struct virtio_snd_ctl_msg {
+ /* device-read-only data */
+ le32 request_code;
+ u8 request_payload[];
+ /* device-writable data */
+ le32 response_status;
+ u8 response_payload[];
+};
+\end{lstlisting}
+
+A generic control message consists of request and response parts and contains
+the following fields:
+
+\begin{description}
+\item[\field{request_code}] (device-read-only) specifies a device request code
+(VIRTIO_SND_R_*).
+\item[\field{request_payload}] (device-read-only) contains request-specific
+data.
+\item[\field{response_status}] (device-writable) specifies a device response
+status (VIRTIO_SND_S_*).
+\item[\field{response_payload}] (device-writable) contains response-specific
+data.
+\end{description}
+
+Unless stated otherwise, the \field{request_payload} and \field{response_payload}
+fields are empty.
+
+The \field{response_status} field contains one of the following values:
+
+\begin{itemize*}
+\item VIRTIO_SND_S_OK: success.
+\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or contains invalid
+parameters.
+\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters are not supported.
+\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
+\end{itemize*}
+
+\subsubsection{Device Operation: PCM control requests}
+
+A PCM stream has the following command lifecycle:
+
+\begin{enumerate}
+\item Set format
+\item Prepare
+\item Output only: transfer data for prebuffing
+\item Start
+\item Transfer data to/from the PCM device
+\begin{enumerate}
+ \item Pause
+ \item Unpause
+\end{enumerate}
+\item Stop
+\end{enumerate}
+
+PCM control requests have or consist of a fixed header with the following
+layout structure:
+
+\begin{lstlisting}
+/* supported PCM stream types */
+enum {
+ VIRTIO_SND_PCM_T_OUTPUT = 0,
+ VIRTIO_SND_PCM_T_INPUT
+};
+
+/* PCM control request header */
+struct virtio_snd_pcm_hdr {
+ le32 code;
+ le32 stream;
+};
+\end{lstlisting}
+
+PCM control request fields:
+
+\begin{description}
+\item[\field{code}] (device-read-only) specifies a PCM device request code
+(VIRTIO_SND_R_PCM_*).
+\item[\field{stream}] (device-read-only) specifies a PCM stream type
+(VIRTIO_SND_PCM_T_*).
+\end{description}
+
+\begin{description}
+
+\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
+Query a PCM channel map information for specified stream type.
+
+A response uses the following layout structure and definitions:
+
+\begin{lstlisting}
+/* standard channel position definition */
+enum {
+ VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
+ VIRTIO_SND_PCM_CH_NA, /* silent */
+ VIRTIO_SND_PCM_CH_MONO, /* mono stream */
+ VIRTIO_SND_PCM_CH_FL, /* front left */
+ VIRTIO_SND_PCM_CH_FR, /* front right */
+ VIRTIO_SND_PCM_CH_RL, /* rear left */
+ VIRTIO_SND_PCM_CH_RR, /* rear right */
+ VIRTIO_SND_PCM_CH_FC, /* front center */
+ VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
+ VIRTIO_SND_PCM_CH_SL, /* side left */
+ VIRTIO_SND_PCM_CH_SR, /* side right */
+ VIRTIO_SND_PCM_CH_RC, /* rear center */
+ VIRTIO_SND_PCM_CH_FLC, /* front left center */
+ VIRTIO_SND_PCM_CH_FRC, /* front right center */
+ VIRTIO_SND_PCM_CH_RLC, /* rear left center */
+ VIRTIO_SND_PCM_CH_RRC, /* rear right center */
+ VIRTIO_SND_PCM_CH_FLW, /* front left wide */
+ VIRTIO_SND_PCM_CH_FRW, /* front right wide */
+ VIRTIO_SND_PCM_CH_FLH, /* front left high */
+ VIRTIO_SND_PCM_CH_FCH, /* front center high */
+ VIRTIO_SND_PCM_CH_FRH, /* front right high */
+ VIRTIO_SND_PCM_CH_TC, /* top center */
+ VIRTIO_SND_PCM_CH_TFL, /* top front left */
+ VIRTIO_SND_PCM_CH_TFR, /* top front right */
+ VIRTIO_SND_PCM_CH_TFC, /* top front center */
+ VIRTIO_SND_PCM_CH_TRL, /* top rear left */
+ VIRTIO_SND_PCM_CH_TRR, /* top rear right */
+ VIRTIO_SND_PCM_CH_TRC, /* top rear center */
+ VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
+ VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
+ VIRTIO_SND_PCM_CH_TSL, /* top side left */
+ VIRTIO_SND_PCM_CH_TSR, /* top side right */
+ VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
+ VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
+ VIRTIO_SND_PCM_CH_BC, /* bottom center */
+ VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
+ VIRTIO_SND_PCM_CH_BRC /* bottom right center */
+};
+
+/* a maximum possible number of channels */
+#define VIRTIO_SND_PCM_CH_MAX 256
+
+/* response containing a PCM channel map information */
+struct virtio_snd_pcm_chmap_info {
+ le32 status;
+ le32 npositions;
+ u8 positions[VIRTIO_SND_PCM_CH_MAX];
+};
+\end{lstlisting}
+
+PCM channel map information fields:
+
+\begin{description}
+\item[\field{status}] (device-writable) specifies a device response status
+(VIRTIO_SND_S_*).
+\item[\field{npositions}] (device-writable) is a number of valid entries in
+the \field{positions} array.
+\item[\field{positions}] (device-writable) contains PCM channel positions
+(VIRTIO_SND_PCM_CH_*).
+\end{description}
+
+\item[VIRTIO_SND_R_PCM_SET_FORMAT]
+Set selected PCM format.
+
+\begin{lstlisting}
+struct virtio_snd_pcm_set_format {
+ struct virtio_snd_pcm_hdr hdr;
+ le16 channels;
+ le16 format;
+ le16 rate;
+ u16 padding;
+};
+\end{lstlisting}
+
+PCM control request fields:
+
+\begin{description}
+\item[\field{hdr}] (device-read-only) is a PCM control request header.
+\item[\field{channels}] (device-read-only) specifies a desired number of channels.
+\item[\field{format}] (device-read-only) specifies a desired PCM sample format
+(VIRTIO_SND_PCM_FMT_*).
+\item[\field{rate}] (device-read-only) specifies a desired PCM frame rate
+(VIRTIO_SND_PCM_RATE_*).
+\end{description}
+
+\item[VIRTIO_SND_R_PCM_PREPARE]
+Prepare the PCM device.
+
+\item[VIRTIO_SND_R_PCM_START]
+Start the PCM device.
+
+\item[VIRTIO_SND_R_PCM_STOP]
+Stop the PCM device.
+
+\item[VIRTIO_SND_R_PCM_PAUSE]
+Set the PCM device on pause.
+
+\item[VIRTIO_SND_R_PCM_UNPAUSE]
+Unset the PCM device from pause.
+
+\end{description}
+
+\devicenormative{\subsubsection}{PCM control requests}{Device Types / Sound Device / PCM control requests}
+
+In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
+
+\begin{itemize*}
+\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status code, if it does not
+support a channel map for a specified stream type;
+\item the device MUST set the \field{npositions} field to 0, if the operation
+failed.
+\end{itemize*}
+
+\drivernormative{\subsubsection}{PCM control requests}{Device Types / Sound Device / PCM control requests}
+
+The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type if
+the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
+
+The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type if
+the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
+
+In a VIRTIO_SND_R_PCM_SET_FORMAT request:
+
+\begin{itemize*}
+\item the driver MUST NOT specify the \field{channels} value less than the \field{channels_min}
+or greater than the \field{channels_max} values reported in stream configuration;
+\item the driver MUST specify the \field{format} and \field{rate} values according
+to the \field{formats} and \field{rates} values reported in stream configuration;
+\item the driver MUST NOT specify the not defined format and rate values;
+\item the driver MUST initialize \field{padding} bytes to 0.
+\end{itemize*}
+
+\subsubsection{Device Operation: PCM I/O requests}
+
+All I/O requests are placed into the pcmq virtqueue. Each request is of form:
+
+\begin{lstlisting}
+struct virtio_snd_pcm_xfer {
+ le32 stream;
+ u8 data[];
+ le32 status;
+ le32 actual_length;
+};
+\end{lstlisting}
+
+I/O request fields:
+
+\begin{description}
+\item[\field{stream}] (device-read-only) specifies a PCM stream type
+(VIRTIO_SND_PCM_T_*).
+\item[\field{data}]
+\begin{enumerate}
+\item Output: (device-read-only) a buffer with PCM frames to be written to the
+device.
+\item Input: (device-writable) a buffer to be filled with PCM frames from the
+device.
+\end{enumerate}
+\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK if an operation
+is successful, and VIRTIO_SND_S_IO_ERR otherwise.
+\item[\field{actual_length}] (device-writable) specifies an actual amount of
+bytes read from/written to the \field{data} field.
+\end{description}
+
+\devicenormative{\subsubsection}{PCM I/O requests}{Device Types / Sound Device / PCM IO requests}
+
+\begin{enumerate}
+\item The device MUST set the \field{actual_length} field to 0, if the operation
+failed.
+\end{enumerate}
+
+\drivernormative{\subsubsection}{PCM I/O requests}{Device Types / Sound Device / PCM IO requests}
+
+\begin{enumerate}
+\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type if
+the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
+\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type if
+the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
+\end{enumerate}
--
2.23.0
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-09-24 14:43 [virtio-dev] [PATCH] snd: Add virtio sound device specification Mikhail Golubev
@ 2019-10-28 16:05 ` Liam Girdwood
2019-10-29 9:42 ` Anton Yakovlev
[not found] ` <20191028192952.GI5015@sirena.co.uk>
0 siblings, 2 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-10-28 16:05 UTC (permalink / raw)
To: Mikhail Golubev, virtio-dev; +Cc: Anton Yakovlev, Takashi Iwai, Mark Brown
On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>
> This patch proposes virtio specification for new virtio sound device.
>
> The virtio sound card is a virtual audio device supporting output and
> input PCM streams.
>
Apologies for the delay in response, I've just been made aware of this
spec proposal on this list. I've also added Mark and Takashi.
Just to make you aware I'm working on a spec for DSP virtualisation,
where the DSP can do more than just simple audio uses cases. I intend
to support this spec as a fallback for guests who request non DSP audio
functionality, hence my interest :)
This spec looks in the right direction for virtualising non DSP audio
devices (where we have HW for DMAC, I2S/HDA and a codec IC). There are
a few opens that I've detailed below.
> Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
> Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
> ---
> Fixes from RFC:
> * Use fixed virtqueue numbers for the control and PCM queues
> * Use device feature bits to indicate available input/output PCM
> streams
> * Place PCM stream configuration into the device configuration space
> * Add sound device and driver requirements into the "Device
> Initialization"
> subsection
> * Add sound device and driver requirements into the "Device
> Operation" subsection
> * Add sound device and driver conformance subsections
>
> RFC link:
> https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
>
> conformance.tex | 20 +++
> content.tex | 1 +
> virtio-sound.tex | 405
> +++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 426 insertions(+)
> create mode 100644 virtio-sound.tex
>
> diff --git a/conformance.tex b/conformance.tex
> index 0ac58aa..31595bf 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -183,6 +183,16 @@ \section{Conformance
> Targets}\label{sec:Conformance / Conformance Targets}
> \item \ref{drivernormative:Device Types / Socket Device / Device
> Operation / Device Events}
> \end{itemize}
>
> +\conformance{\subsection}{Sound Driver
> Conformance}\label{sec:Conformance / Driver Conformance / Sound
> Driver Conformance}
> +
> +A sound driver MUST conform to the following normative statements:
> +
> +\begin{itemize}
> +\item \ref{drivernormative:Device Types / Sound Device / Device
> Initialization}
> +\item \ref{drivernormative:Device Types / Sound Device / PCM control
> requests}
> +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
> requests}
> +\end{itemize}
> +
> \conformance{\section}{Device Conformance}\label{sec:Conformance /
> Device Conformance}
>
> A device MUST conform to the following normative statements:
> @@ -338,6 +348,16 @@ \section{Conformance
> Targets}\label{sec:Conformance / Conformance Targets}
> \item \ref{devicenormative:Device Types / Socket Device / Device
> Operation / Receive and Transmit}
> \end{itemize}
>
> +\conformance{\subsection}{Sound Device
> Conformance}\label{sec:Conformance / Device Conformance / Sound
> Device Conformance}
> +
> +A sound device MUST conform to the following normative statements:
> +
> +\begin{itemize}
> +\item \ref{devicenormative:Device Types / Sound Device / Device
> Initialization}
> +\item \ref{devicenormative:Device Types / Sound Device / PCM control
> requests}
> +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
> requests}
> +\end{itemize}
> +
> \conformance{\section}{Legacy Interface: Transitional Device and
> Transitional Driver Conformance}\label{sec:Conformance / Legacy
> Interface: Transitional Device and Transitional Driver Conformance}
> A conformant implementation MUST be either transitional or
> non-transitional, see \ref{intro:Legacy
> diff --git a/content.tex b/content.tex
> index 37a2190..4a00884 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
> Requirements}\label{sec:Device
> \input{virtio-input.tex}
> \input{virtio-crypto.tex}
> \input{virtio-vsock.tex}
> +\input{virtio-sound.tex}
>
> \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
>
> diff --git a/virtio-sound.tex b/virtio-sound.tex
> new file mode 100644
> index 0000000..68a606e
> --- /dev/null
> +++ b/virtio-sound.tex
> @@ -0,0 +1,405 @@
> +\section{Sound Device}\label{sec:Device Types / Sound Device}
> +
> +The virtio sound card is a virtual audio device supporting output
> and input PCM
> +streams. All device control requests are placed into the control
> virtqueue, all
> +I/O requests are placed into the PCM virtqueue.
Where are stream position updates placed ? These are usually
synchronous from the driver and enable the player application to render
or read data from the correct locations in the buffers. This is latency
sensitive data.
> +
> +\subsection{Device ID}\label{sec:Device Types / Sound Device /
> Device ID}
> +
> +25
> +
> +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
> Virtqueues}
> +
> +\begin{description}
> +\item[0] controlq
> +\item[1] pcmq
> +\end{description}
> +
> +The controlq virtqueue always exists, the pcmq virtqueue only exists
> if
> +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT feature is
> negotiated.
> +
> +\subsection{Feature bits}\label{sec:Device Types / Sound Device /
> Feature bits}
> +
> +\begin{description}
> +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
> +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
> +\end{description}
> +
> +\subsection{Device configuration layout}\label{sec:Device Types /
> Sound Device / Device configuration layout}
> +
> +\begin{lstlisting}
> +/* supported PCM sample formats */
> +enum {
> + VIRTIO_SND_PCM_FMT_S8 = 0,
> + VIRTIO_SND_PCM_FMT_U8,
> + VIRTIO_SND_PCM_FMT_S16,
> + VIRTIO_SND_PCM_FMT_U16,
> + VIRTIO_SND_PCM_FMT_S32,
> + VIRTIO_SND_PCM_FMT_U32,
> + VIRTIO_SND_PCM_FMT_FLOAT,
> + VIRTIO_SND_PCM_FMT_FLOAT64
> +};
> +
> +/* supported PCM frame rates */
> +enum {
> + VIRTIO_SND_PCM_RATE_8000 = 0,
> + VIRTIO_SND_PCM_RATE_11025,
> + VIRTIO_SND_PCM_RATE_16000,
> + VIRTIO_SND_PCM_RATE_22050,
> + VIRTIO_SND_PCM_RATE_32000,
> + VIRTIO_SND_PCM_RATE_44100,
> + VIRTIO_SND_PCM_RATE_48000,
> + VIRTIO_SND_PCM_RATE_64000,
> + VIRTIO_SND_PCM_RATE_88200,
> + VIRTIO_SND_PCM_RATE_96000,
> + VIRTIO_SND_PCM_RATE_176400,
> + VIRTIO_SND_PCM_RATE_192000
It may be best to use the same rates and format from the ALSA spec,
this covers any holes (like rates that are not in the above list).
> +};
> +
> +/* a PCM stream configuration */
> +struct virtio_pcm_stream_config {
> + u8 channels_min;
> + u8 channels_max;
> + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
> + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
> + u16 padding;
> +};
Should this be packed ? Btw, padding alignment is to a non 32bit size ?
What about buffer/period formats e.g interleaved ?
> +
> +/* a device configuration space */
> +struct virtio_snd_config {
> + struct virtio_pcm_config {
> + struct virtio_pcm_stream_config output;
> + struct virtio_pcm_stream_config input;
> + } pcm;
> +};
> +\end{lstlisting}
I assuming this is sent to configure PCM capabilities ? What about
buffer and period size capabilities ?
> +
> +\subsubsection{Device configuration fields}
> +
> +The \field{pcm.output} and \field{pcm.input} fields contain PCM
> stream
> +configuration:
> +
> +\begin{description}
> +\item[\field{channels_min}] (driver-read-only) is a minimum number
> of supported
> +channels.
> +\item[\field{channels_max}] (driver-read-only) is a maximum number
> of supported
> +channels.
> +\item[\field{formats}] (driver-read-only) is supported sample format
> bit map.
> +\item[\field{rates}] (driver-read-only) is supported frame rate bit
> map.
> +\end{description}
> +
> +\subsection{Device Initialization}
> +
> +\begin{enumerate}
> +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
> \field{pcm.output}
> +contains valid output PCM stream configuration.
> +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
> \field{pcm.input}
> +contains valid input PCM stream configuration.
> +\end{enumerate}
> +
> +\devicenormative{\subsubsection}{Device Initialization}{Device Types
> / Sound Device / Device Initialization}
> +
> +\begin{enumerate}
> +\item The device MUST NOT set the not defined format and rate bits.
> +\item The device MUST initialize padding bytes
> \field{pcm.output.padding} and
> +\field{pcm.input.padding} to 0.
> +\end{enumerate}
> +
> +\drivernormative{\subsubsection}{Device Initialization}{Device Types
> / Sound Device / Device Initialization}
> +
> +\begin{enumerate}
> +\item The driver MUST configure and initialize all virtqueues.
> +\item The driver SHOULD ignore the not defined format and rate bits.
> +\end{enumerate}
> +
> +\subsection{Device Operation}\label{sec:Device Types / Sound Device
> / Device Operation}
> +
> +All control messages are placed into the controlq virtqueue and use
> the following
> +layout structure and definitions:
> +
> +\begin{lstlisting}
> +enum {
> + /* PCM control request types */
> + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
> + VIRTIO_SND_R_PCM_SET_FORMAT,
> + VIRTIO_SND_R_PCM_PREPARE,
> + VIRTIO_SND_R_PCM_START,
> + VIRTIO_SND_R_PCM_STOP,
> + VIRTIO_SND_R_PCM_PAUSE,
> + VIRTIO_SND_R_PCM_UNPAUSE,
> +
> + /* generic status codes */
> + VIRTIO_SND_S_OK = 0x8000,
> + VIRTIO_SND_S_BAD_MSG,
> + VIRTIO_SND_S_NOT_SUPP,
> + VIRTIO_SND_S_IO_ERR
> +};
> +
> +struct virtio_snd_ctl_msg {
> + /* device-read-only data */
> + le32 request_code;
> + u8 request_payload[];
How do I know how much data to read here ? Is size embedded in
request_code ?
> + /* device-writable data */
> + le32 response_status;
> + u8 response_payload[];
> +};
> +\end{lstlisting}
> +
> +A generic control message consists of request and response parts and
> contains
> +the following fields:
> +
> +\begin{description}
> +\item[\field{request_code}] (device-read-only) specifies a device
> request code
> +(VIRTIO_SND_R_*).
> +\item[\field{request_payload}] (device-read-only) contains request-
> specific
> +data.
> +\item[\field{response_status}] (device-writable) specifies a device
> response
> +status (VIRTIO_SND_S_*).
> +\item[\field{response_payload}] (device-writable) contains response-
> specific
> +data.
> +\end{description}
> +
> +Unless stated otherwise, the \field{request_payload} and
> \field{response_payload}
> +fields are empty.
> +
> +The \field{response_status} field contains one of the following
> values:
> +
> +\begin{itemize*}
> +\item VIRTIO_SND_S_OK: success.
> +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
> contains invalid
> +parameters.
> +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters are
> not supported.
> +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
> +\end{itemize*}
> +
> +\subsubsection{Device Operation: PCM control requests}
> +
> +A PCM stream has the following command lifecycle:
> +
> +\begin{enumerate}
> +\item Set format
> +\item Prepare
> +\item Output only: transfer data for prebuffing
> +\item Start
> +\item Transfer data to/from the PCM device
> +\begin{enumerate}
> + \item Pause
> + \item Unpause
> +\end{enumerate}
> +\item Stop
> +\end{enumerate}
> +
> +PCM control requests have or consist of a fixed header with the
> following
> +layout structure:
> +
> +\begin{lstlisting}
> +/* supported PCM stream types */
> +enum {
> + VIRTIO_SND_PCM_T_OUTPUT = 0,
> + VIRTIO_SND_PCM_T_INPUT
Can we rename these PLAYBACK and CAPTURE
> +};
> +
> +/* PCM control request header */
> +struct virtio_snd_pcm_hdr {
> + le32 code;
> + le32 stream;
> +};
> +\end{lstlisting}
> +
> +PCM control request fields:
> +
> +\begin{description}
> +\item[\field{code}] (device-read-only) specifies a PCM device
> request code
> +(VIRTIO_SND_R_PCM_*).
> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
> +(VIRTIO_SND_PCM_T_*).
> +\end{description}
> +
> +\begin{description}
> +
> +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
> +Query a PCM channel map information for specified stream type.
> +
> +A response uses the following layout structure and definitions:
> +
> +\begin{lstlisting}
> +/* standard channel position definition */
> +enum {
> + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
> + VIRTIO_SND_PCM_CH_NA, /* silent */
> + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
> + VIRTIO_SND_PCM_CH_FL, /* front left */
> + VIRTIO_SND_PCM_CH_FR, /* front right */
> + VIRTIO_SND_PCM_CH_RL, /* rear left */
> + VIRTIO_SND_PCM_CH_RR, /* rear right */
> + VIRTIO_SND_PCM_CH_FC, /* front center */
> + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
> + VIRTIO_SND_PCM_CH_SL, /* side left */
> + VIRTIO_SND_PCM_CH_SR, /* side right */
> + VIRTIO_SND_PCM_CH_RC, /* rear center */
> + VIRTIO_SND_PCM_CH_FLC, /* front left center */
> + VIRTIO_SND_PCM_CH_FRC, /* front right center */
> + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
> + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
> + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
> + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
> + VIRTIO_SND_PCM_CH_FLH, /* front left high */
> + VIRTIO_SND_PCM_CH_FCH, /* front center high */
> + VIRTIO_SND_PCM_CH_FRH, /* front right high */
> + VIRTIO_SND_PCM_CH_TC, /* top center */
> + VIRTIO_SND_PCM_CH_TFL, /* top front left */
> + VIRTIO_SND_PCM_CH_TFR, /* top front right */
> + VIRTIO_SND_PCM_CH_TFC, /* top front center */
> + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
> + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
> + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
> + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
> + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
> + VIRTIO_SND_PCM_CH_TSL, /* top side left */
> + VIRTIO_SND_PCM_CH_TSR, /* top side right */
> + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
> + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
> + VIRTIO_SND_PCM_CH_BC, /* bottom center */
> + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
> + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
> +};
> +
> +/* a maximum possible number of channels */
> +#define VIRTIO_SND_PCM_CH_MAX 256
> +
> +/* response containing a PCM channel map information */
> +struct virtio_snd_pcm_chmap_info {
> + le32 status;
> + le32 npositions;
> + u8 positions[VIRTIO_SND_PCM_CH_MAX];
> +};
> +\end{lstlisting}
> +
> +PCM channel map information fields:
> +
> +\begin{description}
> +\item[\field{status}] (device-writable) specifies a device response
> status
> +(VIRTIO_SND_S_*).
> +\item[\field{npositions}] (device-writable) is a number of valid
> entries in
> +the \field{positions} array.
> +\item[\field{positions}] (device-writable) contains PCM channel
> positions
> +(VIRTIO_SND_PCM_CH_*).
> +\end{description}
> +
> +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
> +Set selected PCM format.
> +
> +\begin{lstlisting}
> +struct virtio_snd_pcm_set_format {
> + struct virtio_snd_pcm_hdr hdr;
> + le16 channels;
> + le16 format;
> + le16 rate;
> + u16 padding;
> +};
> +\end{lstlisting}
> +
> +PCM control request fields:
> +
> +\begin{description}
> +\item[\field{hdr}] (device-read-only) is a PCM control request
> header.
> +\item[\field{channels}] (device-read-only) specifies a desired
> number of channels.
> +\item[\field{format}] (device-read-only) specifies a desired PCM
> sample format
> +(VIRTIO_SND_PCM_FMT_*).
> +\item[\field{rate}] (device-read-only) specifies a desired PCM frame
> rate
> +(VIRTIO_SND_PCM_RATE_*).
> +\end{description}
> +
> +\item[VIRTIO_SND_R_PCM_PREPARE]
> +Prepare the PCM device.
> +
> +\item[VIRTIO_SND_R_PCM_START]
> +Start the PCM device.
> +
> +\item[VIRTIO_SND_R_PCM_STOP]
> +Stop the PCM device.
> +
> +\item[VIRTIO_SND_R_PCM_PAUSE]
> +Set the PCM device on pause.
> +
> +\item[VIRTIO_SND_R_PCM_UNPAUSE]
> +Unset the PCM device from pause.
> +
> +\end{description}
> +
> +\devicenormative{\subsubsection}{PCM control requests}{Device Types
> / Sound Device / PCM control requests}
> +
> +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
> +
> +\begin{itemize*}
> +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status code,
> if it does not
> +support a channel map for a specified stream type;
> +\item the device MUST set the \field{npositions} field to 0, if the
> operation
> +failed.
> +\end{itemize*}
> +
> +\drivernormative{\subsubsection}{PCM control requests}{Device Types
> / Sound Device / PCM control requests}
> +
> +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type
> if
> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
> +
> +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type
> if
> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
> +
> +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
> +
> +\begin{itemize*}
> +\item the driver MUST NOT specify the \field{channels} value less
> than the \field{channels_min}
> +or greater than the \field{channels_max} values reported in stream
> configuration;
> +\item the driver MUST specify the \field{format} and \field{rate}
> values according
> +to the \field{formats} and \field{rates} values reported in stream
> configuration;
> +\item the driver MUST NOT specify the not defined format and rate
> values;
> +\item the driver MUST initialize \field{padding} bytes to 0.
> +\end{itemize*}
> +
> +\subsubsection{Device Operation: PCM I/O requests}
> +
> +All I/O requests are placed into the pcmq virtqueue. Each request is
> of form:
> +
> +\begin{lstlisting}
> +struct virtio_snd_pcm_xfer {
> + le32 stream;
> + u8 data[];
> + le32 status;
> + le32 actual_length;
Not following this, is actual_length the size of data[]. If so, it must
preceed data[].
> +};
> +\end{lstlisting}
> +
> +I/O request fields:
> +
> +\begin{description}
> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
> +(VIRTIO_SND_PCM_T_*).
> +\item[\field{data}]
> +\begin{enumerate}
> +\item Output: (device-read-only) a buffer with PCM frames to be
> written to the
> +device.
> +\item Input: (device-writable) a buffer to be filled with PCM frames
> from the
> +device.
> +\end{enumerate}
> +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK if
> an operation
> +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
> +\item[\field{actual_length}] (device-writable) specifies an actual
> amount of
> +bytes read from/written to the \field{data} field.
> +\end{description}
> +
> +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types /
> Sound Device / PCM IO requests}
> +
> +\begin{enumerate}
> +\item The device MUST set the \field{actual_length} field to 0, if
> the operation
> +failed.
> +\end{enumerate}
> +
> +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types /
> Sound Device / PCM IO requests}
> +
> +\begin{enumerate}
> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
> stream type if
> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
> type if
> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
> +\end{enumerate}
> --
> 2.23.0
>
I don't see anything in here about dealing with
1) Underruns and overruns. i.e. how do we recover and resync audio
between guests.
2) Zero copy of audio data and stream positions. Essential for any low
latency audio use cases (if supported by HW, hypervisor, VM framework).
I would strongly recommend reading the ALSA driver developers guide if
you can or looking at the ALSA headers for PCMs and controls (I guess
you have looked a little based on the channel map).
Thanks
Liam
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-10-28 16:05 ` Liam Girdwood
@ 2019-10-29 9:42 ` Anton Yakovlev
2019-10-29 10:14 ` Anton Yakovlev
2019-11-11 15:20 ` Liam Girdwood
[not found] ` <20191028192952.GI5015@sirena.co.uk>
1 sibling, 2 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-10-29 9:42 UTC (permalink / raw)
To: Liam Girdwood, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
Hi Liam,
Thank you for comments! Please, take a look at our answers below.
On 28.10.2019 17:05, Liam Girdwood wrote:
> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>
>> This patch proposes virtio specification for new virtio sound device.
>>
>> The virtio sound card is a virtual audio device supporting output and
>> input PCM streams.
>>
>
> Apologies for the delay in response, I've just been made aware of this
> spec proposal on this list. I've also added Mark and Takashi.
>
> Just to make you aware I'm working on a spec for DSP virtualisation,
> where the DSP can do more than just simple audio uses cases. I intend
> to support this spec as a fallback for guests who request non DSP audio
> functionality, hence my interest :)
>
> This spec looks in the right direction for virtualising non DSP audio
> devices (where we have HW for DMAC, I2S/HDA and a codec IC). There are
> a few opens that I've detailed below.
Originally, this spec was supposed to be extendable. Since PCM is the simplest
and most demandable feature, we decided to start only with this one.
>> Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>> Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
>> ---
>> Fixes from RFC:
>> * Use fixed virtqueue numbers for the control and PCM queues
>> * Use device feature bits to indicate available input/output PCM
>> streams
>> * Place PCM stream configuration into the device configuration space
>> * Add sound device and driver requirements into the "Device
>> Initialization"
>> subsection
>> * Add sound device and driver requirements into the "Device
>> Operation" subsection
>> * Add sound device and driver conformance subsections
>>
>> RFC link:
>> https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
>>
>> conformance.tex | 20 +++
>> content.tex | 1 +
>> virtio-sound.tex | 405
>> +++++++++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 426 insertions(+)
>> create mode 100644 virtio-sound.tex
>>
>> diff --git a/conformance.tex b/conformance.tex
>> index 0ac58aa..31595bf 100644
>> --- a/conformance.tex
>> +++ b/conformance.tex
>> @@ -183,6 +183,16 @@ \section{Conformance
>> Targets}\label{sec:Conformance / Conformance Targets}
>> \item \ref{drivernormative:Device Types / Socket Device / Device
>> Operation / Device Events}
>> \end{itemize}
>>
>> +\conformance{\subsection}{Sound Driver
>> Conformance}\label{sec:Conformance / Driver Conformance / Sound
>> Driver Conformance}
>> +
>> +A sound driver MUST conform to the following normative statements:
>> +
>> +\begin{itemize}
>> +\item \ref{drivernormative:Device Types / Sound Device / Device
>> Initialization}
>> +\item \ref{drivernormative:Device Types / Sound Device / PCM control
>> requests}
>> +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
>> requests}
>> +\end{itemize}
>> +
>> \conformance{\section}{Device Conformance}\label{sec:Conformance /
>> Device Conformance}
>>
>> A device MUST conform to the following normative statements:
>> @@ -338,6 +348,16 @@ \section{Conformance
>> Targets}\label{sec:Conformance / Conformance Targets}
>> \item \ref{devicenormative:Device Types / Socket Device / Device
>> Operation / Receive and Transmit}
>> \end{itemize}
>>
>> +\conformance{\subsection}{Sound Device
>> Conformance}\label{sec:Conformance / Device Conformance / Sound
>> Device Conformance}
>> +
>> +A sound device MUST conform to the following normative statements:
>> +
>> +\begin{itemize}
>> +\item \ref{devicenormative:Device Types / Sound Device / Device
>> Initialization}
>> +\item \ref{devicenormative:Device Types / Sound Device / PCM control
>> requests}
>> +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
>> requests}
>> +\end{itemize}
>> +
>> \conformance{\section}{Legacy Interface: Transitional Device and
>> Transitional Driver Conformance}\label{sec:Conformance / Legacy
>> Interface: Transitional Device and Transitional Driver Conformance}
>> A conformant implementation MUST be either transitional or
>> non-transitional, see \ref{intro:Legacy
>> diff --git a/content.tex b/content.tex
>> index 37a2190..4a00884 100644
>> --- a/content.tex
>> +++ b/content.tex
>> @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
>> Requirements}\label{sec:Device
>> \input{virtio-input.tex}
>> \input{virtio-crypto.tex}
>> \input{virtio-vsock.tex}
>> +\input{virtio-sound.tex}
>>
>> \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
>>
>> diff --git a/virtio-sound.tex b/virtio-sound.tex
>> new file mode 100644
>> index 0000000..68a606e
>> --- /dev/null
>> +++ b/virtio-sound.tex
>> @@ -0,0 +1,405 @@
>> +\section{Sound Device}\label{sec:Device Types / Sound Device}
>> +
>> +The virtio sound card is a virtual audio device supporting output
>> and input PCM
>> +streams. All device control requests are placed into the control
>> virtqueue, all
>> +I/O requests are placed into the PCM virtqueue.
>
> Where are stream position updates placed ? These are usually
> synchronous from the driver and enable the player application to render
> or read data from the correct locations in the buffers. This is latency
> sensitive data.
It happens in an interrupt handler. Since PCM frame transfer is message based,
the only possible way to figure out consumed or captured amount of frames is
to take a look into a message response.
>> +
>> +\subsection{Device ID}\label{sec:Device Types / Sound Device /
>> Device ID}
>> +
>> +25
>> +
>> +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
>> Virtqueues}
>> +
>> +\begin{description}
>> +\item[0] controlq
>> +\item[1] pcmq
>> +\end{description}
>> +
>> +The controlq virtqueue always exists, the pcmq virtqueue only exists
>> if
>> +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT feature is
>> negotiated.
>> +
>> +\subsection{Feature bits}\label{sec:Device Types / Sound Device /
>> Feature bits}
>> +
>> +\begin{description}
>> +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
>> +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
>> +\end{description}
>> +
>> +\subsection{Device configuration layout}\label{sec:Device Types /
>> Sound Device / Device configuration layout}
>> +
>> +\begin{lstlisting}
>> +/* supported PCM sample formats */
>> +enum {
>> + VIRTIO_SND_PCM_FMT_S8 = 0,
>> + VIRTIO_SND_PCM_FMT_U8,
>> + VIRTIO_SND_PCM_FMT_S16,
>> + VIRTIO_SND_PCM_FMT_U16,
>> + VIRTIO_SND_PCM_FMT_S32,
>> + VIRTIO_SND_PCM_FMT_U32,
>> + VIRTIO_SND_PCM_FMT_FLOAT,
>> + VIRTIO_SND_PCM_FMT_FLOAT64
>> +};
>> +
>> +/* supported PCM frame rates */
>> +enum {
>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>> + VIRTIO_SND_PCM_RATE_11025,
>> + VIRTIO_SND_PCM_RATE_16000,
>> + VIRTIO_SND_PCM_RATE_22050,
>> + VIRTIO_SND_PCM_RATE_32000,
>> + VIRTIO_SND_PCM_RATE_44100,
>> + VIRTIO_SND_PCM_RATE_48000,
>> + VIRTIO_SND_PCM_RATE_64000,
>> + VIRTIO_SND_PCM_RATE_88200,
>> + VIRTIO_SND_PCM_RATE_96000,
>> + VIRTIO_SND_PCM_RATE_176400,
>> + VIRTIO_SND_PCM_RATE_192000
>
> It may be best to use the same rates and format from the ALSA spec,
> this covers any holes (like rates that are not in the above list).
These rates are from ALSA definitions. And what formats should be added?
>> +};
>> +
>> +/* a PCM stream configuration */
>> +struct virtio_pcm_stream_config {
>> + u8 channels_min;
>> + u8 channels_max;
>> + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
>> + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
>> + u16 padding;
>> +};
>
> Should this be packed ? Btw, padding alignment is to a non 32bit size ?
It should be naturally packed if padding is correct. And do you mean 8 and
16-bit fields? You can read one or two bytes from device configuration space,
it should not be a problem.
> What about buffer/period formats e.g interleaved ?
Yeah, support only for interleaved layout previously was stated in the
specification but occasionally was lost. Will be fixed!
>
>> +
>> +/* a device configuration space */
>> +struct virtio_snd_config {
>> + struct virtio_pcm_config {
>> + struct virtio_pcm_stream_config output;
>> + struct virtio_pcm_stream_config input;
>> + } pcm;
>> +};
>> +\end{lstlisting}
>
> I assuming this is sent to configure PCM capabilities ?
Yes, minimum amount of information required to setup a PCM stream in any kind
of guest.
> What about buffer and period size capabilities ?
In this specification we assume that buffer size is up to guest decision.
Also, period size is ALSA-only conception and should not be mentioned here at
all (we are not stick only to ALSA).
>> +
>> +\subsubsection{Device configuration fields}
>> +
>> +The \field{pcm.output} and \field{pcm.input} fields contain PCM
>> stream
>> +configuration:
>> +
>> +\begin{description}
>> +\item[\field{channels_min}] (driver-read-only) is a minimum number
>> of supported
>> +channels.
>> +\item[\field{channels_max}] (driver-read-only) is a maximum number
>> of supported
>> +channels.
>> +\item[\field{formats}] (driver-read-only) is supported sample format
>> bit map.
>> +\item[\field{rates}] (driver-read-only) is supported frame rate bit
>> map.
>> +\end{description}
>> +
>> +\subsection{Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
>> \field{pcm.output}
>> +contains valid output PCM stream configuration.
>> +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
>> \field{pcm.input}
>> +contains valid input PCM stream configuration.
>> +\end{enumerate}
>> +
>> +\devicenormative{\subsubsection}{Device Initialization}{Device Types
>> / Sound Device / Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item The device MUST NOT set the not defined format and rate bits.
>> +\item The device MUST initialize padding bytes
>> \field{pcm.output.padding} and
>> +\field{pcm.input.padding} to 0.
>> +\end{enumerate}
>> +
>> +\drivernormative{\subsubsection}{Device Initialization}{Device Types
>> / Sound Device / Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item The driver MUST configure and initialize all virtqueues.
>> +\item The driver SHOULD ignore the not defined format and rate bits.
>> +\end{enumerate}
>> +
>> +\subsection{Device Operation}\label{sec:Device Types / Sound Device
>> / Device Operation}
>> +
>> +All control messages are placed into the controlq virtqueue and use
>> the following
>> +layout structure and definitions:
>> +
>> +\begin{lstlisting}
>> +enum {
>> + /* PCM control request types */
>> + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
>> + VIRTIO_SND_R_PCM_SET_FORMAT,
>> + VIRTIO_SND_R_PCM_PREPARE,
>> + VIRTIO_SND_R_PCM_START,
>> + VIRTIO_SND_R_PCM_STOP,
>> + VIRTIO_SND_R_PCM_PAUSE,
>> + VIRTIO_SND_R_PCM_UNPAUSE,
>> +
>> + /* generic status codes */
>> + VIRTIO_SND_S_OK = 0x8000,
>> + VIRTIO_SND_S_BAD_MSG,
>> + VIRTIO_SND_S_NOT_SUPP,
>> + VIRTIO_SND_S_IO_ERR
>> +};
>> +
>> +struct virtio_snd_ctl_msg {
>> + /* device-read-only data */
>> + le32 request_code;
>> + u8 request_payload[];
>
> How do I know how much data to read here ? Is size embedded in
> request_code ?
Yes, you are right. Actual request/response length can be easily derived from
a request_code.
>
>> + /* device-writable data */
>> + le32 response_status;
>> + u8 response_payload[];
>> +};
>> +\end{lstlisting}
>> +
>> +A generic control message consists of request and response parts and
>> contains
>> +the following fields:
>> +
>> +\begin{description}
>> +\item[\field{request_code}] (device-read-only) specifies a device
>> request code
>> +(VIRTIO_SND_R_*).
>> +\item[\field{request_payload}] (device-read-only) contains request-
>> specific
>> +data.
>> +\item[\field{response_status}] (device-writable) specifies a device
>> response
>> +status (VIRTIO_SND_S_*).
>> +\item[\field{response_payload}] (device-writable) contains response-
>> specific
>> +data.
>> +\end{description}
>> +
>> +Unless stated otherwise, the \field{request_payload} and
>> \field{response_payload}
>> +fields are empty.
>> +
>> +The \field{response_status} field contains one of the following
>> values:
>> +
>> +\begin{itemize*}
>> +\item VIRTIO_SND_S_OK: success.
>> +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
>> contains invalid
>> +parameters.
>> +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters are
>> not supported.
>> +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
>> +\end{itemize*}
>> +
>> +\subsubsection{Device Operation: PCM control requests}
>> +
>> +A PCM stream has the following command lifecycle:
>> +
>> +\begin{enumerate}
>> +\item Set format
>> +\item Prepare
>> +\item Output only: transfer data for prebuffing
>> +\item Start
>> +\item Transfer data to/from the PCM device
>> +\begin{enumerate}
>> + \item Pause
>> + \item Unpause
>> +\end{enumerate}
>> +\item Stop
>> +\end{enumerate}
>> +
>> +PCM control requests have or consist of a fixed header with the
>> following
>> +layout structure:
>> +
>> +\begin{lstlisting}
>> +/* supported PCM stream types */
>> +enum {
>> + VIRTIO_SND_PCM_T_OUTPUT = 0,
>> + VIRTIO_SND_PCM_T_INPUT
>
> Can we rename these PLAYBACK and CAPTURE
Playback and capture are used in ALSA drivers, sink and source are used in
Windows drivers. We decided to stick to something obvious in-between (and it's
just shorter). Although, it's still discussable topic.
>> +};
>> +
>> +/* PCM control request header */
>> +struct virtio_snd_pcm_hdr {
>> + le32 code;
>> + le32 stream;
>> +};
>> +\end{lstlisting}
>> +
>> +PCM control request fields:
>> +
>> +\begin{description}
>> +\item[\field{code}] (device-read-only) specifies a PCM device
>> request code
>> +(VIRTIO_SND_R_PCM_*).
>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>> +(VIRTIO_SND_PCM_T_*).
>> +\end{description}
>> +
>> +\begin{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
>> +Query a PCM channel map information for specified stream type.
>> +
>> +A response uses the following layout structure and definitions:
>> +
>> +\begin{lstlisting}
>> +/* standard channel position definition */
>> +enum {
>> + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
>> + VIRTIO_SND_PCM_CH_NA, /* silent */
>> + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
>> + VIRTIO_SND_PCM_CH_FL, /* front left */
>> + VIRTIO_SND_PCM_CH_FR, /* front right */
>> + VIRTIO_SND_PCM_CH_RL, /* rear left */
>> + VIRTIO_SND_PCM_CH_RR, /* rear right */
>> + VIRTIO_SND_PCM_CH_FC, /* front center */
>> + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
>> + VIRTIO_SND_PCM_CH_SL, /* side left */
>> + VIRTIO_SND_PCM_CH_SR, /* side right */
>> + VIRTIO_SND_PCM_CH_RC, /* rear center */
>> + VIRTIO_SND_PCM_CH_FLC, /* front left center */
>> + VIRTIO_SND_PCM_CH_FRC, /* front right center */
>> + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
>> + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
>> + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
>> + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
>> + VIRTIO_SND_PCM_CH_FLH, /* front left high */
>> + VIRTIO_SND_PCM_CH_FCH, /* front center high */
>> + VIRTIO_SND_PCM_CH_FRH, /* front right high */
>> + VIRTIO_SND_PCM_CH_TC, /* top center */
>> + VIRTIO_SND_PCM_CH_TFL, /* top front left */
>> + VIRTIO_SND_PCM_CH_TFR, /* top front right */
>> + VIRTIO_SND_PCM_CH_TFC, /* top front center */
>> + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
>> + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
>> + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
>> + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
>> + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
>> + VIRTIO_SND_PCM_CH_TSL, /* top side left */
>> + VIRTIO_SND_PCM_CH_TSR, /* top side right */
>> + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
>> + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
>> + VIRTIO_SND_PCM_CH_BC, /* bottom center */
>> + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
>> + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
>> +};
>> +
>> +/* a maximum possible number of channels */
>> +#define VIRTIO_SND_PCM_CH_MAX 256
>> +
>> +/* response containing a PCM channel map information */
>> +struct virtio_snd_pcm_chmap_info {
>> + le32 status;
>> + le32 npositions;
>> + u8 positions[VIRTIO_SND_PCM_CH_MAX];
>> +};
>> +\end{lstlisting}
>> +
>> +PCM channel map information fields:
>> +
>> +\begin{description}
>> +\item[\field{status}] (device-writable) specifies a device response
>> status
>> +(VIRTIO_SND_S_*).
>> +\item[\field{npositions}] (device-writable) is a number of valid
>> entries in
>> +the \field{positions} array.
>> +\item[\field{positions}] (device-writable) contains PCM channel
>> positions
>> +(VIRTIO_SND_PCM_CH_*).
>> +\end{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
>> +Set selected PCM format.
>> +
>> +\begin{lstlisting}
>> +struct virtio_snd_pcm_set_format {
>> + struct virtio_snd_pcm_hdr hdr;
>> + le16 channels;
>> + le16 format;
>> + le16 rate;
>> + u16 padding;
>> +};
>> +\end{lstlisting}
>> +
>> +PCM control request fields:
>> +
>> +\begin{description}
>> +\item[\field{hdr}] (device-read-only) is a PCM control request
>> header.
>> +\item[\field{channels}] (device-read-only) specifies a desired
>> number of channels.
>> +\item[\field{format}] (device-read-only) specifies a desired PCM
>> sample format
>> +(VIRTIO_SND_PCM_FMT_*).
>> +\item[\field{rate}] (device-read-only) specifies a desired PCM frame
>> rate
>> +(VIRTIO_SND_PCM_RATE_*).
>> +\end{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_PREPARE]
>> +Prepare the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_START]
>> +Start the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_STOP]
>> +Stop the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_PAUSE]
>> +Set the PCM device on pause.
>> +
>> +\item[VIRTIO_SND_R_PCM_UNPAUSE]
>> +Unset the PCM device from pause.
>> +
>> +\end{description}
>> +
>> +\devicenormative{\subsubsection}{PCM control requests}{Device Types
>> / Sound Device / PCM control requests}
>> +
>> +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
>> +
>> +\begin{itemize*}
>> +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status code,
>> if it does not
>> +support a channel map for a specified stream type;
>> +\item the device MUST set the \field{npositions} field to 0, if the
>> operation
>> +failed.
>> +\end{itemize*}
>> +
>> +\drivernormative{\subsubsection}{PCM control requests}{Device Types
>> / Sound Device / PCM control requests}
>> +
>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type
>> if
>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>> +
>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type
>> if
>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>> +
>> +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
>> +
>> +\begin{itemize*}
>> +\item the driver MUST NOT specify the \field{channels} value less
>> than the \field{channels_min}
>> +or greater than the \field{channels_max} values reported in stream
>> configuration;
>> +\item the driver MUST specify the \field{format} and \field{rate}
>> values according
>> +to the \field{formats} and \field{rates} values reported in stream
>> configuration;
>> +\item the driver MUST NOT specify the not defined format and rate
>> values;
>> +\item the driver MUST initialize \field{padding} bytes to 0.
>> +\end{itemize*}
>> +
>> +\subsubsection{Device Operation: PCM I/O requests}
>> +
>> +All I/O requests are placed into the pcmq virtqueue. Each request is
>> of form:
>> +
>> +\begin{lstlisting}
>> +struct virtio_snd_pcm_xfer {
>> + le32 stream;
>> + u8 data[];
>> + le32 status;
>> + le32 actual_length;
>
> Not following this, is actual_length the size of data[]. If so, it must
> preceed data[].
No, the actual_length field is supposed to be used by device side to report
actual amount of bytes read from/written to a buffer. In real world scenario,
if an I/O request contains N bytes, a device can play/capture *up to* N bytes.
Thus, it's required to report this length back to a driver.
>
>> +};
>> +\end{lstlisting}
>> +
>> +I/O request fields:
>> +
>> +\begin{description}
>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>> +(VIRTIO_SND_PCM_T_*).
>> +\item[\field{data}]
>> +\begin{enumerate}
>> +\item Output: (device-read-only) a buffer with PCM frames to be
>> written to the
>> +device.
>> +\item Input: (device-writable) a buffer to be filled with PCM frames
>> from the
>> +device.
>> +\end{enumerate}
>> +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK if
>> an operation
>> +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
>> +\item[\field{actual_length}] (device-writable) specifies an actual
>> amount of
>> +bytes read from/written to the \field{data} field.
>> +\end{description}
>> +
>> +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types /
>> Sound Device / PCM IO requests}
>> +
>> +\begin{enumerate}
>> +\item The device MUST set the \field{actual_length} field to 0, if
>> the operation
>> +failed.
>> +\end{enumerate}
>> +
>> +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types /
>> Sound Device / PCM IO requests}
>> +
>> +\begin{enumerate}
>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
>> stream type if
>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
>> type if
>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>> +\end{enumerate}
>> --
>> 2.23.0
>>
>
> I don't see anything in here about dealing with
>
> 1) Underruns and overruns. i.e. how do we recover and resync audio
> between guests.
Since xrun conditions are higher level conceptions, we decided to delegate
such issues to guest application itself. It helps to make overall design
simpler. And it seems there's not so much we can do if xrun condition is
happened on the device side.
> 2) Zero copy of audio data and stream positions. Essential for any low
> latency audio use cases (if supported by HW, hypervisor, VM framework).
What do you mean by zero-copy? Shared memory between device and driver sides?
> I would strongly recommend reading the ALSA driver developers guide if
> you can or looking at the ALSA headers for PCMs and controls (I guess
> you have looked a little based on the channel map).
Yes, we did it. :)
>
> Thanks
>
> Liam
>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-10-29 9:42 ` Anton Yakovlev
@ 2019-10-29 10:14 ` Anton Yakovlev
[not found] ` <20191029121810.GB5253@sirena.co.uk>
2019-11-11 15:20 ` Liam Girdwood
1 sibling, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-10-29 10:14 UTC (permalink / raw)
To: Liam Girdwood, Mikhail Golubev, virtio-dev
Cc: Takashi Iwai, Mark Brown, Marc Zyngier, James Morse,
Julien Thierry, Suzuki K Poulose
Resend with adding more people to the discussion.
On 28.10.2019 17:05, Liam Girdwood wrote:
> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>
>> This patch proposes virtio specification for new virtio sound device.
>>
>> The virtio sound card is a virtual audio device supporting output and
>> input PCM streams.
>>
>
> Apologies for the delay in response, I've just been made aware of this
> spec proposal on this list. I've also added Mark and Takashi.
>
> Just to make you aware I'm working on a spec for DSP virtualisation,
> where the DSP can do more than just simple audio uses cases. I intend
> to support this spec as a fallback for guests who request non DSP audio
> functionality, hence my interest
>
> This spec looks in the right direction for virtualising non DSP audio
> devices (where we have HW for DMAC, I2S/HDA and a codec IC). There are
> a few opens that I've detailed below.
Originally, this spec was supposed to be extendable. Since PCM is the simplest
and most demandable feature, we decided to start only with this one.
>> Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>> Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
>> ---
>> Fixes from RFC:
>> * Use fixed virtqueue numbers for the control and PCM queues
>> * Use device feature bits to indicate available input/output PCM
>> streams
>> * Place PCM stream configuration into the device configuration space
>> * Add sound device and driver requirements into the "Device
>> Initialization"
>> subsection
>> * Add sound device and driver requirements into the "Device
>> Operation" subsection
>> * Add sound device and driver conformance subsections
>>
>> RFC link:
>> https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
>>
>> conformance.tex | 20 +++
>> content.tex | 1 +
>> virtio-sound.tex | 405
>> +++++++++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 426 insertions(+)
>> create mode 100644 virtio-sound.tex
>>
>> diff --git a/conformance.tex b/conformance.tex
>> index 0ac58aa..31595bf 100644
>> --- a/conformance.tex
>> +++ b/conformance.tex
>> @@ -183,6 +183,16 @@ \section{Conformance
>> Targets}\label{sec:Conformance / Conformance Targets}
>> \item \ref{drivernormative:Device Types / Socket Device / Device
>> Operation / Device Events}
>> \end{itemize}
>>
>> +\conformance{\subsection}{Sound Driver
>> Conformance}\label{sec:Conformance / Driver Conformance / Sound
>> Driver Conformance}
>> +
>> +A sound driver MUST conform to the following normative statements:
>> +
>> +\begin{itemize}
>> +\item \ref{drivernormative:Device Types / Sound Device / Device
>> Initialization}
>> +\item \ref{drivernormative:Device Types / Sound Device / PCM control
>> requests}
>> +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
>> requests}
>> +\end{itemize}
>> +
>> \conformance{\section}{Device Conformance}\label{sec:Conformance /
>> Device Conformance}
>>
>> A device MUST conform to the following normative statements:
>> @@ -338,6 +348,16 @@ \section{Conformance
>> Targets}\label{sec:Conformance / Conformance Targets}
>> \item \ref{devicenormative:Device Types / Socket Device / Device
>> Operation / Receive and Transmit}
>> \end{itemize}
>>
>> +\conformance{\subsection}{Sound Device
>> Conformance}\label{sec:Conformance / Device Conformance / Sound
>> Device Conformance}
>> +
>> +A sound device MUST conform to the following normative statements:
>> +
>> +\begin{itemize}
>> +\item \ref{devicenormative:Device Types / Sound Device / Device
>> Initialization}
>> +\item \ref{devicenormative:Device Types / Sound Device / PCM control
>> requests}
>> +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
>> requests}
>> +\end{itemize}
>> +
>> \conformance{\section}{Legacy Interface: Transitional Device and
>> Transitional Driver Conformance}\label{sec:Conformance / Legacy
>> Interface: Transitional Device and Transitional Driver Conformance}
>> A conformant implementation MUST be either transitional or
>> non-transitional, see \ref{intro:Legacy
>> diff --git a/content.tex b/content.tex
>> index 37a2190..4a00884 100644
>> --- a/content.tex
>> +++ b/content.tex
>> @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
>> Requirements}\label{sec:Device
>> \input{virtio-input.tex}
>> \input{virtio-crypto.tex}
>> \input{virtio-vsock.tex}
>> +\input{virtio-sound.tex}
>>
>> \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
>>
>> diff --git a/virtio-sound.tex b/virtio-sound.tex
>> new file mode 100644
>> index 0000000..68a606e
>> --- /dev/null
>> +++ b/virtio-sound.tex
>> @@ -0,0 +1,405 @@
>> +\section{Sound Device}\label{sec:Device Types / Sound Device}
>> +
>> +The virtio sound card is a virtual audio device supporting output
>> and input PCM
>> +streams. All device control requests are placed into the control
>> virtqueue, all
>> +I/O requests are placed into the PCM virtqueue.
>
> Where are stream position updates placed ? These are usually
> synchronous from the driver and enable the player application to render
> or read data from the correct locations in the buffers. This is latency
> sensitive data.
It happens in an interrupt handler. Since PCM frame transfer is message based,
the only possible way to figure out consumed or captured amount of frames is
to take a look into a message response.
>> +
>> +\subsection{Device ID}\label{sec:Device Types / Sound Device /
>> Device ID}
>> +
>> +25
>> +
>> +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
>> Virtqueues}
>> +
>> +\begin{description}
>> +\item[0] controlq
>> +\item[1] pcmq
>> +\end{description}
>> +
>> +The controlq virtqueue always exists, the pcmq virtqueue only exists
>> if
>> +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT feature is
>> negotiated.
>> +
>> +\subsection{Feature bits}\label{sec:Device Types / Sound Device /
>> Feature bits}
>> +
>> +\begin{description}
>> +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
>> +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
>> +\end{description}
>> +
>> +\subsection{Device configuration layout}\label{sec:Device Types /
>> Sound Device / Device configuration layout}
>> +
>> +\begin{lstlisting}
>> +/* supported PCM sample formats */
>> +enum {
>> + VIRTIO_SND_PCM_FMT_S8 = 0,
>> + VIRTIO_SND_PCM_FMT_U8,
>> + VIRTIO_SND_PCM_FMT_S16,
>> + VIRTIO_SND_PCM_FMT_U16,
>> + VIRTIO_SND_PCM_FMT_S32,
>> + VIRTIO_SND_PCM_FMT_U32,
>> + VIRTIO_SND_PCM_FMT_FLOAT,
>> + VIRTIO_SND_PCM_FMT_FLOAT64
>> +};
>> +
>> +/* supported PCM frame rates */
>> +enum {
>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>> + VIRTIO_SND_PCM_RATE_11025,
>> + VIRTIO_SND_PCM_RATE_16000,
>> + VIRTIO_SND_PCM_RATE_22050,
>> + VIRTIO_SND_PCM_RATE_32000,
>> + VIRTIO_SND_PCM_RATE_44100,
>> + VIRTIO_SND_PCM_RATE_48000,
>> + VIRTIO_SND_PCM_RATE_64000,
>> + VIRTIO_SND_PCM_RATE_88200,
>> + VIRTIO_SND_PCM_RATE_96000,
>> + VIRTIO_SND_PCM_RATE_176400,
>> + VIRTIO_SND_PCM_RATE_192000
>
> It may be best to use the same rates and format from the ALSA spec,
> this covers any holes (like rates that are not in the above list).
These rates are from ALSA definitions. And what formats should be added?
>> +};
>> +
>> +/* a PCM stream configuration */
>> +struct virtio_pcm_stream_config {
>> + u8 channels_min;
>> + u8 channels_max;
>> + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
>> + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
>> + u16 padding;
>> +};
>
> Should this be packed ? Btw, padding alignment is to a non 32bit size ?
It should be naturally packed if padding is correct. And do you mean 8 and
16-bit fields? You can read one or two bytes from device configuration space,
it should not be a problem.
> What about buffer/period formats e.g interleaved ?
Yeah, support only for interleaved layout previously was stated in the
specification but occasionally was lost. Will be fixed!
>
>> +
>> +/* a device configuration space */
>> +struct virtio_snd_config {
>> + struct virtio_pcm_config {
>> + struct virtio_pcm_stream_config output;
>> + struct virtio_pcm_stream_config input;
>> + } pcm;
>> +};
>> +\end{lstlisting}
>
> I assuming this is sent to configure PCM capabilities ?
Yes, minimum amount of information required to setup a PCM stream in any kind
of guest.
> What about buffer and period size capabilities ?
In this specification we assume that buffer size is up to guest decision.
Also, period size is ALSA-only conception and should not be mentioned here at
all (we are not stick only to ALSA).
>> +
>> +\subsubsection{Device configuration fields}
>> +
>> +The \field{pcm.output} and \field{pcm.input} fields contain PCM
>> stream
>> +configuration:
>> +
>> +\begin{description}
>> +\item[\field{channels_min}] (driver-read-only) is a minimum number
>> of supported
>> +channels.
>> +\item[\field{channels_max}] (driver-read-only) is a maximum number
>> of supported
>> +channels.
>> +\item[\field{formats}] (driver-read-only) is supported sample format
>> bit map.
>> +\item[\field{rates}] (driver-read-only) is supported frame rate bit
>> map.
>> +\end{description}
>> +
>> +\subsection{Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
>> \field{pcm.output}
>> +contains valid output PCM stream configuration.
>> +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
>> \field{pcm.input}
>> +contains valid input PCM stream configuration.
>> +\end{enumerate}
>> +
>> +\devicenormative{\subsubsection}{Device Initialization}{Device Types
>> / Sound Device / Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item The device MUST NOT set the not defined format and rate bits.
>> +\item The device MUST initialize padding bytes
>> \field{pcm.output.padding} and
>> +\field{pcm.input.padding} to 0.
>> +\end{enumerate}
>> +
>> +\drivernormative{\subsubsection}{Device Initialization}{Device Types
>> / Sound Device / Device Initialization}
>> +
>> +\begin{enumerate}
>> +\item The driver MUST configure and initialize all virtqueues.
>> +\item The driver SHOULD ignore the not defined format and rate bits.
>> +\end{enumerate}
>> +
>> +\subsection{Device Operation}\label{sec:Device Types / Sound Device
>> / Device Operation}
>> +
>> +All control messages are placed into the controlq virtqueue and use
>> the following
>> +layout structure and definitions:
>> +
>> +\begin{lstlisting}
>> +enum {
>> + /* PCM control request types */
>> + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
>> + VIRTIO_SND_R_PCM_SET_FORMAT,
>> + VIRTIO_SND_R_PCM_PREPARE,
>> + VIRTIO_SND_R_PCM_START,
>> + VIRTIO_SND_R_PCM_STOP,
>> + VIRTIO_SND_R_PCM_PAUSE,
>> + VIRTIO_SND_R_PCM_UNPAUSE,
>> +
>> + /* generic status codes */
>> + VIRTIO_SND_S_OK = 0x8000,
>> + VIRTIO_SND_S_BAD_MSG,
>> + VIRTIO_SND_S_NOT_SUPP,
>> + VIRTIO_SND_S_IO_ERR
>> +};
>> +
>> +struct virtio_snd_ctl_msg {
>> + /* device-read-only data */
>> + le32 request_code;
>> + u8 request_payload[];
>
> How do I know how much data to read here ? Is size embedded in
> request_code ?
Yes, you are right. Actual request/response length can be easily derived from
a request_code.
>
>> + /* device-writable data */
>> + le32 response_status;
>> + u8 response_payload[];
>> +};
>> +\end{lstlisting}
>> +
>> +A generic control message consists of request and response parts and
>> contains
>> +the following fields:
>> +
>> +\begin{description}
>> +\item[\field{request_code}] (device-read-only) specifies a device
>> request code
>> +(VIRTIO_SND_R_*).
>> +\item[\field{request_payload}] (device-read-only) contains request-
>> specific
>> +data.
>> +\item[\field{response_status}] (device-writable) specifies a device
>> response
>> +status (VIRTIO_SND_S_*).
>> +\item[\field{response_payload}] (device-writable) contains response-
>> specific
>> +data.
>> +\end{description}
>> +
>> +Unless stated otherwise, the \field{request_payload} and
>> \field{response_payload}
>> +fields are empty.
>> +
>> +The \field{response_status} field contains one of the following
>> values:
>> +
>> +\begin{itemize*}
>> +\item VIRTIO_SND_S_OK: success.
>> +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
>> contains invalid
>> +parameters.
>> +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters are
>> not supported.
>> +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
>> +\end{itemize*}
>> +
>> +\subsubsection{Device Operation: PCM control requests}
>> +
>> +A PCM stream has the following command lifecycle:
>> +
>> +\begin{enumerate}
>> +\item Set format
>> +\item Prepare
>> +\item Output only: transfer data for prebuffing
>> +\item Start
>> +\item Transfer data to/from the PCM device
>> +\begin{enumerate}
>> + \item Pause
>> + \item Unpause
>> +\end{enumerate}
>> +\item Stop
>> +\end{enumerate}
>> +
>> +PCM control requests have or consist of a fixed header with the
>> following
>> +layout structure:
>> +
>> +\begin{lstlisting}
>> +/* supported PCM stream types */
>> +enum {
>> + VIRTIO_SND_PCM_T_OUTPUT = 0,
>> + VIRTIO_SND_PCM_T_INPUT
>
> Can we rename these PLAYBACK and CAPTURE
Playback and capture are used in ALSA drivers, sink and source are used in
Windows drivers. We decided to stick to something obvious in-between (and it's
just shorter). Although, it's still discussable topic.
>> +};
>> +
>> +/* PCM control request header */
>> +struct virtio_snd_pcm_hdr {
>> + le32 code;
>> + le32 stream;
>> +};
>> +\end{lstlisting}
>> +
>> +PCM control request fields:
>> +
>> +\begin{description}
>> +\item[\field{code}] (device-read-only) specifies a PCM device
>> request code
>> +(VIRTIO_SND_R_PCM_*).
>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>> +(VIRTIO_SND_PCM_T_*).
>> +\end{description}
>> +
>> +\begin{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
>> +Query a PCM channel map information for specified stream type.
>> +
>> +A response uses the following layout structure and definitions:
>> +
>> +\begin{lstlisting}
>> +/* standard channel position definition */
>> +enum {
>> + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
>> + VIRTIO_SND_PCM_CH_NA, /* silent */
>> + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
>> + VIRTIO_SND_PCM_CH_FL, /* front left */
>> + VIRTIO_SND_PCM_CH_FR, /* front right */
>> + VIRTIO_SND_PCM_CH_RL, /* rear left */
>> + VIRTIO_SND_PCM_CH_RR, /* rear right */
>> + VIRTIO_SND_PCM_CH_FC, /* front center */
>> + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
>> + VIRTIO_SND_PCM_CH_SL, /* side left */
>> + VIRTIO_SND_PCM_CH_SR, /* side right */
>> + VIRTIO_SND_PCM_CH_RC, /* rear center */
>> + VIRTIO_SND_PCM_CH_FLC, /* front left center */
>> + VIRTIO_SND_PCM_CH_FRC, /* front right center */
>> + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
>> + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
>> + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
>> + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
>> + VIRTIO_SND_PCM_CH_FLH, /* front left high */
>> + VIRTIO_SND_PCM_CH_FCH, /* front center high */
>> + VIRTIO_SND_PCM_CH_FRH, /* front right high */
>> + VIRTIO_SND_PCM_CH_TC, /* top center */
>> + VIRTIO_SND_PCM_CH_TFL, /* top front left */
>> + VIRTIO_SND_PCM_CH_TFR, /* top front right */
>> + VIRTIO_SND_PCM_CH_TFC, /* top front center */
>> + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
>> + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
>> + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
>> + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
>> + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
>> + VIRTIO_SND_PCM_CH_TSL, /* top side left */
>> + VIRTIO_SND_PCM_CH_TSR, /* top side right */
>> + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
>> + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
>> + VIRTIO_SND_PCM_CH_BC, /* bottom center */
>> + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
>> + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
>> +};
>> +
>> +/* a maximum possible number of channels */
>> +#define VIRTIO_SND_PCM_CH_MAX 256
>> +
>> +/* response containing a PCM channel map information */
>> +struct virtio_snd_pcm_chmap_info {
>> + le32 status;
>> + le32 npositions;
>> + u8 positions[VIRTIO_SND_PCM_CH_MAX];
>> +};
>> +\end{lstlisting}
>> +
>> +PCM channel map information fields:
>> +
>> +\begin{description}
>> +\item[\field{status}] (device-writable) specifies a device response
>> status
>> +(VIRTIO_SND_S_*).
>> +\item[\field{npositions}] (device-writable) is a number of valid
>> entries in
>> +the \field{positions} array.
>> +\item[\field{positions}] (device-writable) contains PCM channel
>> positions
>> +(VIRTIO_SND_PCM_CH_*).
>> +\end{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
>> +Set selected PCM format.
>> +
>> +\begin{lstlisting}
>> +struct virtio_snd_pcm_set_format {
>> + struct virtio_snd_pcm_hdr hdr;
>> + le16 channels;
>> + le16 format;
>> + le16 rate;
>> + u16 padding;
>> +};
>> +\end{lstlisting}
>> +
>> +PCM control request fields:
>> +
>> +\begin{description}
>> +\item[\field{hdr}] (device-read-only) is a PCM control request
>> header.
>> +\item[\field{channels}] (device-read-only) specifies a desired
>> number of channels.
>> +\item[\field{format}] (device-read-only) specifies a desired PCM
>> sample format
>> +(VIRTIO_SND_PCM_FMT_*).
>> +\item[\field{rate}] (device-read-only) specifies a desired PCM frame
>> rate
>> +(VIRTIO_SND_PCM_RATE_*).
>> +\end{description}
>> +
>> +\item[VIRTIO_SND_R_PCM_PREPARE]
>> +Prepare the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_START]
>> +Start the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_STOP]
>> +Stop the PCM device.
>> +
>> +\item[VIRTIO_SND_R_PCM_PAUSE]
>> +Set the PCM device on pause.
>> +
>> +\item[VIRTIO_SND_R_PCM_UNPAUSE]
>> +Unset the PCM device from pause.
>> +
>> +\end{description}
>> +
>> +\devicenormative{\subsubsection}{PCM control requests}{Device Types
>> / Sound Device / PCM control requests}
>> +
>> +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
>> +
>> +\begin{itemize*}
>> +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status code,
>> if it does not
>> +support a channel map for a specified stream type;
>> +\item the device MUST set the \field{npositions} field to 0, if the
>> operation
>> +failed.
>> +\end{itemize*}
>> +
>> +\drivernormative{\subsubsection}{PCM control requests}{Device Types
>> / Sound Device / PCM control requests}
>> +
>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type
>> if
>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>> +
>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type
>> if
>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>> +
>> +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
>> +
>> +\begin{itemize*}
>> +\item the driver MUST NOT specify the \field{channels} value less
>> than the \field{channels_min}
>> +or greater than the \field{channels_max} values reported in stream
>> configuration;
>> +\item the driver MUST specify the \field{format} and \field{rate}
>> values according
>> +to the \field{formats} and \field{rates} values reported in stream
>> configuration;
>> +\item the driver MUST NOT specify the not defined format and rate
>> values;
>> +\item the driver MUST initialize \field{padding} bytes to 0.
>> +\end{itemize*}
>> +
>> +\subsubsection{Device Operation: PCM I/O requests}
>> +
>> +All I/O requests are placed into the pcmq virtqueue. Each request is
>> of form:
>> +
>> +\begin{lstlisting}
>> +struct virtio_snd_pcm_xfer {
>> + le32 stream;
>> + u8 data[];
>> + le32 status;
>> + le32 actual_length;
>
> Not following this, is actual_length the size of data[]. If so, it must
> preceed data[].
No, the actual_length field is supposed to be used by device side to report
actual amount of bytes read from/written to a buffer. In real world scenario,
if an I/O request contains N bytes, a device can play/capture *up to* N bytes.
Thus, it's required to report this length back to a driver.
>
>> +};
>> +\end{lstlisting}
>> +
>> +I/O request fields:
>> +
>> +\begin{description}
>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>> +(VIRTIO_SND_PCM_T_*).
>> +\item[\field{data}]
>> +\begin{enumerate}
>> +\item Output: (device-read-only) a buffer with PCM frames to be
>> written to the
>> +device.
>> +\item Input: (device-writable) a buffer to be filled with PCM frames
>> from the
>> +device.
>> +\end{enumerate}
>> +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK if
>> an operation
>> +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
>> +\item[\field{actual_length}] (device-writable) specifies an actual
>> amount of
>> +bytes read from/written to the \field{data} field.
>> +\end{description}
>> +
>> +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types /
>> Sound Device / PCM IO requests}
>> +
>> +\begin{enumerate}
>> +\item The device MUST set the \field{actual_length} field to 0, if
>> the operation
>> +failed.
>> +\end{enumerate}
>> +
>> +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types /
>> Sound Device / PCM IO requests}
>> +
>> +\begin{enumerate}
>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
>> stream type if
>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
>> type if
>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>> +\end{enumerate}
>> --
>> 2.23.0
>>
>
> I don't see anything in here about dealing with
>
> 1) Underruns and overruns. i.e. how do we recover and resync audio
> between guests.
Since xrun conditions are higher level conceptions, we decided to delegate
such issues to guest application itself. It helps to make overall design
simpler. And it seems there's not so much we can do if xrun condition is
happened on the device side.
> 2) Zero copy of audio data and stream positions. Essential for any low
> latency audio use cases (if supported by HW, hypervisor, VM framework).
What do you mean by zero-copy? Shared memory between device and driver sides?
> I would strongly recommend reading the ALSA driver developers guide if
> you can or looking at the ALSA headers for PCMs and controls (I guess
> you have looked a little based on the channel map).
Yes, we did it.
>
> Thanks
>
> Liam
>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191028192952.GI5015@sirena.co.uk>
@ 2019-10-29 10:46 ` Anton Yakovlev
0 siblings, 0 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-10-29 10:46 UTC (permalink / raw)
To: Mark Brown, Liam Girdwood
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Marc Zyngier,
James Morse, Julien Thierry, Suzuki K Poulose
On 28.10.2019 20:29, Mark Brown wrote:
> On Mon, Oct 28, 2019 at 04:05:57PM +0000, Liam Girdwood wrote:
>> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>>> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>>
>>> This patch proposes virtio specification for new virtio sound device.
>>>
>>> The virtio sound card is a virtual audio device supporting output and
>>> input PCM streams.
>
>> Apologies for the delay in response, I've just been made aware of this
>> spec proposal on this list. I've also added Mark and Takashi.
>
> Also adding the ARM virtualization people for their review and not
> cutting any context as a result.
>
>> Just to make you aware I'm working on a spec for DSP virtualisation,
>> where the DSP can do more than just simple audio uses cases. I intend
>> to support this spec as a fallback for guests who request non DSP audio
>> functionality, hence my interest :)
>>
>> This spec looks in the right direction for virtualising non DSP audio
>> devices (where we have HW for DMAC, I2S/HDA and a codec IC). There are
>> a few opens that I've detailed below.
>>
>>> Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>> Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
>>> ---
>>> Fixes from RFC:
>>> * Use fixed virtqueue numbers for the control and PCM queues
>>> * Use device feature bits to indicate available input/output PCM
>>> streams
>>> * Place PCM stream configuration into the device configuration space
>>> * Add sound device and driver requirements into the "Device
>>> Initialization"
>>> subsection
>>> * Add sound device and driver requirements into the "Device
>>> Operation" subsection
>>> * Add sound device and driver conformance subsections
>>>
>>> RFC link:
>>> https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
>>>
>>> conformance.tex | 20 +++
>>> content.tex | 1 +
>>> virtio-sound.tex | 405
>>> +++++++++++++++++++++++++++++++++++++++++++++++
>>> 3 files changed, 426 insertions(+)
>>> create mode 100644 virtio-sound.tex
>>>
>>> diff --git a/conformance.tex b/conformance.tex
>>> index 0ac58aa..31595bf 100644
>>> --- a/conformance.tex
>>> +++ b/conformance.tex
>>> @@ -183,6 +183,16 @@ \section{Conformance
>>> Targets}\label{sec:Conformance / Conformance Targets}
>>> \item \ref{drivernormative:Device Types / Socket Device / Device
>>> Operation / Device Events}
>>> \end{itemize}
>>>
>>> +\conformance{\subsection}{Sound Driver
>>> Conformance}\label{sec:Conformance / Driver Conformance / Sound
>>> Driver Conformance}
>>> +
>>> +A sound driver MUST conform to the following normative statements:
>>> +
>>> +\begin{itemize}
>>> +\item \ref{drivernormative:Device Types / Sound Device / Device
>>> Initialization}
>>> +\item \ref{drivernormative:Device Types / Sound Device / PCM control
>>> requests}
>>> +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
>>> requests}
>>> +\end{itemize}
>>> +
>>> \conformance{\section}{Device Conformance}\label{sec:Conformance /
>>> Device Conformance}
>>>
>>> A device MUST conform to the following normative statements:
>>> @@ -338,6 +348,16 @@ \section{Conformance
>>> Targets}\label{sec:Conformance / Conformance Targets}
>>> \item \ref{devicenormative:Device Types / Socket Device / Device
>>> Operation / Receive and Transmit}
>>> \end{itemize}
>>>
>>> +\conformance{\subsection}{Sound Device
>>> Conformance}\label{sec:Conformance / Device Conformance / Sound
>>> Device Conformance}
>>> +
>>> +A sound device MUST conform to the following normative statements:
>>> +
>>> +\begin{itemize}
>>> +\item \ref{devicenormative:Device Types / Sound Device / Device
>>> Initialization}
>>> +\item \ref{devicenormative:Device Types / Sound Device / PCM control
>>> requests}
>>> +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
>>> requests}
>>> +\end{itemize}
>>> +
>>> \conformance{\section}{Legacy Interface: Transitional Device and
>>> Transitional Driver Conformance}\label{sec:Conformance / Legacy
>>> Interface: Transitional Device and Transitional Driver Conformance}
>>> A conformant implementation MUST be either transitional or
>>> non-transitional, see \ref{intro:Legacy
>>> diff --git a/content.tex b/content.tex
>>> index 37a2190..4a00884 100644
>>> --- a/content.tex
>>> +++ b/content.tex
>>> @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
>>> Requirements}\label{sec:Device
>>> \input{virtio-input.tex}
>>> \input{virtio-crypto.tex}
>>> \input{virtio-vsock.tex}
>>> +\input{virtio-sound.tex}
>>>
>>> \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
>>>
>>> diff --git a/virtio-sound.tex b/virtio-sound.tex
>>> new file mode 100644
>>> index 0000000..68a606e
>>> --- /dev/null
>>> +++ b/virtio-sound.tex
>>> @@ -0,0 +1,405 @@
>>> +\section{Sound Device}\label{sec:Device Types / Sound Device}
>>> +
>>> +The virtio sound card is a virtual audio device supporting output
>>> and input PCM
>>> +streams. All device control requests are placed into the control
>>> virtqueue, all
>>> +I/O requests are placed into the PCM virtqueue.
>>
>> Where are stream position updates placed ? These are usually
>> synchronous from the driver and enable the player application to render
>> or read data from the correct locations in the buffers. This is latency
>> sensitive data.
>
> My reading of this is that it's message based rather than ring buffer
> based. Like you say that does tend to increase latency but it's fine
> for a lot of applications. I guess so long as we can add something more
> low latency friendly later on there's no problem with that?
Ring buffer is involved in any case. With message based approach it's possible
to refer its subparts. But yes, kicking opposite side is still overhead.
The bigger issue with low latency cases is how to fulfill realtime conditions?
Since in such cases ring buffer is filled with small pieces. And due to
different reasons (like scheduling latency on both sides), xrun condition
possibility is quite high. This applies to any approach, whether it's message
based or like shared memory between host and guest.
>>> +
>>> +\subsection{Device ID}\label{sec:Device Types / Sound Device /
>>> Device ID}
>>> +
>>> +25
>>> +
>>> +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
>>> Virtqueues}
>>> +
>>> +\begin{description}
>>> +\item[0] controlq
>>> +\item[1] pcmq
>>> +\end{description}
>>> +
>>> +The controlq virtqueue always exists, the pcmq virtqueue only exists
>>> if
>>> +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT feature is
>>> negotiated.
>>> +
>>> +\subsection{Feature bits}\label{sec:Device Types / Sound Device /
>>> Feature bits}
>>> +
>>> +\begin{description}
>>> +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
>>> +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
>>> +\end{description}
>>> +
>>> +\subsection{Device configuration layout}\label{sec:Device Types /
>>> Sound Device / Device configuration layout}
>>> +
>>> +\begin{lstlisting}
>>> +/* supported PCM sample formats */
>>> +enum {
>>> + VIRTIO_SND_PCM_FMT_S8 = 0,
>>> + VIRTIO_SND_PCM_FMT_U8,
>>> + VIRTIO_SND_PCM_FMT_S16,
>>> + VIRTIO_SND_PCM_FMT_U16,
>>> + VIRTIO_SND_PCM_FMT_S32,
>>> + VIRTIO_SND_PCM_FMT_U32,
>>> + VIRTIO_SND_PCM_FMT_FLOAT,
>>> + VIRTIO_SND_PCM_FMT_FLOAT64
>>> +};
>>> +
>>> +/* supported PCM frame rates */
>>> +enum {
>>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>>> + VIRTIO_SND_PCM_RATE_11025,
>>> + VIRTIO_SND_PCM_RATE_16000,
>>> + VIRTIO_SND_PCM_RATE_22050,
>>> + VIRTIO_SND_PCM_RATE_32000,
>>> + VIRTIO_SND_PCM_RATE_44100,
>>> + VIRTIO_SND_PCM_RATE_48000,
>>> + VIRTIO_SND_PCM_RATE_64000,
>>> + VIRTIO_SND_PCM_RATE_88200,
>>> + VIRTIO_SND_PCM_RATE_96000,
>>> + VIRTIO_SND_PCM_RATE_176400,
>>> + VIRTIO_SND_PCM_RATE_192000
>
>> It may be best to use the same rates and format from the ALSA spec,
>> this covers any holes (like rates that are not in the above list).
>
> This looks to be the same set of standard audio rates in the range it
> covers but missing 5.512kHz at the low end and 352.8kHz and 38400kHz at
> the top end.
>
>>> +};
>>> +
>>> +/* a PCM stream configuration */
>>> +struct virtio_pcm_stream_config {
>>> + u8 channels_min;
>>> + u8 channels_max;
>>> + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
>>> + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
>>> + u16 padding;
>>> +};
>>
>> Should this be packed ? Btw, padding alignment is to a non 32bit size ?
>> What about buffer/period formats e.g interleaved ?
>>
>>> +
>>> +/* a device configuration space */
>>> +struct virtio_snd_config {
>>> + struct virtio_pcm_config {
>>> + struct virtio_pcm_stream_config output;
>>> + struct virtio_pcm_stream_config input;
>>> + } pcm;
>>> +};
>>> +\end{lstlisting}
>>
>> I assuming this is sent to configure PCM capabilities ? What about
>> buffer and period size capabilities ?
>>
>>> +
>>> +\subsubsection{Device configuration fields}
>>> +
>>> +The \field{pcm.output} and \field{pcm.input} fields contain PCM
>>> stream
>>> +configuration:
>>> +
>>> +\begin{description}
>>> +\item[\field{channels_min}] (driver-read-only) is a minimum number
>>> of supported
>>> +channels.
>>> +\item[\field{channels_max}] (driver-read-only) is a maximum number
>>> of supported
>>> +channels.
>>> +\item[\field{formats}] (driver-read-only) is supported sample format
>>> bit map.
>>> +\item[\field{rates}] (driver-read-only) is supported frame rate bit
>>> map.
>>> +\end{description}
>>> +
>>> +\subsection{Device Initialization}
>>> +
>>> +\begin{enumerate}
>>> +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
>>> \field{pcm.output}
>>> +contains valid output PCM stream configuration.
>>> +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
>>> \field{pcm.input}
>>> +contains valid input PCM stream configuration.
>>> +\end{enumerate}
>>> +
>>> +\devicenormative{\subsubsection}{Device Initialization}{Device Types
>>> / Sound Device / Device Initialization}
>>> +
>>> +\begin{enumerate}
>>> +\item The device MUST NOT set the not defined format and rate bits.
>>> +\item The device MUST initialize padding bytes
>>> \field{pcm.output.padding} and
>>> +\field{pcm.input.padding} to 0.
>>> +\end{enumerate}
>>> +
>>> +\drivernormative{\subsubsection}{Device Initialization}{Device Types
>>> / Sound Device / Device Initialization}
>>> +
>>> +\begin{enumerate}
>>> +\item The driver MUST configure and initialize all virtqueues.
>>> +\item The driver SHOULD ignore the not defined format and rate bits.
>>> +\end{enumerate}
>>> +
>>> +\subsection{Device Operation}\label{sec:Device Types / Sound Device
>>> / Device Operation}
>>> +
>>> +All control messages are placed into the controlq virtqueue and use
>>> the following
>>> +layout structure and definitions:
>>> +
>>> +\begin{lstlisting}
>>> +enum {
>>> + /* PCM control request types */
>>> + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
>>> + VIRTIO_SND_R_PCM_SET_FORMAT,
>>> + VIRTIO_SND_R_PCM_PREPARE,
>>> + VIRTIO_SND_R_PCM_START,
>>> + VIRTIO_SND_R_PCM_STOP,
>>> + VIRTIO_SND_R_PCM_PAUSE,
>>> + VIRTIO_SND_R_PCM_UNPAUSE,
>>> +
>>> + /* generic status codes */
>>> + VIRTIO_SND_S_OK = 0x8000,
>>> + VIRTIO_SND_S_BAD_MSG,
>>> + VIRTIO_SND_S_NOT_SUPP,
>>> + VIRTIO_SND_S_IO_ERR
>>> +};
>>> +
>>> +struct virtio_snd_ctl_msg {
>>> + /* device-read-only data */
>>> + le32 request_code;
>>> + u8 request_payload[];
>>
>> How do I know how much data to read here ? Is size embedded in
>> request_code ?
>>
>>> + /* device-writable data */
>>> + le32 response_status;
>>> + u8 response_payload[];
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +A generic control message consists of request and response parts and
>>> contains
>>> +the following fields:
>>> +
>>> +\begin{description}
>>> +\item[\field{request_code}] (device-read-only) specifies a device
>>> request code
>>> +(VIRTIO_SND_R_*).
>>> +\item[\field{request_payload}] (device-read-only) contains request-
>>> specific
>>> +data.
>>> +\item[\field{response_status}] (device-writable) specifies a device
>>> response
>>> +status (VIRTIO_SND_S_*).
>>> +\item[\field{response_payload}] (device-writable) contains response-
>>> specific
>>> +data.
>>> +\end{description}
>>> +
>>> +Unless stated otherwise, the \field{request_payload} and
>>> \field{response_payload}
>>> +fields are empty.
>>> +
>>> +The \field{response_status} field contains one of the following
>>> values:
>>> +
>>> +\begin{itemize*}
>>> +\item VIRTIO_SND_S_OK: success.
>>> +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
>>> contains invalid
>>> +parameters.
>>> +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters are
>>> not supported.
>>> +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
>>> +\end{itemize*}
>>> +
>>> +\subsubsection{Device Operation: PCM control requests}
>>> +
>>> +A PCM stream has the following command lifecycle:
>>> +
>>> +\begin{enumerate}
>>> +\item Set format
>>> +\item Prepare
>>> +\item Output only: transfer data for prebuffing
>>> +\item Start
>>> +\item Transfer data to/from the PCM device
>>> +\begin{enumerate}
>>> + \item Pause
>>> + \item Unpause
>>> +\end{enumerate}
>>> +\item Stop
>>> +\end{enumerate}
>>> +
>>> +PCM control requests have or consist of a fixed header with the
>>> following
>>> +layout structure:
>>> +
>>> +\begin{lstlisting}
>>> +/* supported PCM stream types */
>>> +enum {
>>> + VIRTIO_SND_PCM_T_OUTPUT = 0,
>>> + VIRTIO_SND_PCM_T_INPUT
>>
>> Can we rename these PLAYBACK and CAPTURE
>>
>>> +};
>>> +
>>> +/* PCM control request header */
>>> +struct virtio_snd_pcm_hdr {
>>> + le32 code;
>>> + le32 stream;
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +PCM control request fields:
>>> +
>>> +\begin{description}
>>> +\item[\field{code}] (device-read-only) specifies a PCM device
>>> request code
>>> +(VIRTIO_SND_R_PCM_*).
>>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>>> +(VIRTIO_SND_PCM_T_*).
>>> +\end{description}
>>> +
>>> +\begin{description}
>>> +
>>> +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
>>> +Query a PCM channel map information for specified stream type.
>>> +
>>> +A response uses the following layout structure and definitions:
>>> +
>>> +\begin{lstlisting}
>>> +/* standard channel position definition */
>>> +enum {
>>> + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
>>> + VIRTIO_SND_PCM_CH_NA, /* silent */
>>> + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
>>> + VIRTIO_SND_PCM_CH_FL, /* front left */
>>> + VIRTIO_SND_PCM_CH_FR, /* front right */
>>> + VIRTIO_SND_PCM_CH_RL, /* rear left */
>>> + VIRTIO_SND_PCM_CH_RR, /* rear right */
>>> + VIRTIO_SND_PCM_CH_FC, /* front center */
>>> + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
>>> + VIRTIO_SND_PCM_CH_SL, /* side left */
>>> + VIRTIO_SND_PCM_CH_SR, /* side right */
>>> + VIRTIO_SND_PCM_CH_RC, /* rear center */
>>> + VIRTIO_SND_PCM_CH_FLC, /* front left center */
>>> + VIRTIO_SND_PCM_CH_FRC, /* front right center */
>>> + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
>>> + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
>>> + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
>>> + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
>>> + VIRTIO_SND_PCM_CH_FLH, /* front left high */
>>> + VIRTIO_SND_PCM_CH_FCH, /* front center high */
>>> + VIRTIO_SND_PCM_CH_FRH, /* front right high */
>>> + VIRTIO_SND_PCM_CH_TC, /* top center */
>>> + VIRTIO_SND_PCM_CH_TFL, /* top front left */
>>> + VIRTIO_SND_PCM_CH_TFR, /* top front right */
>>> + VIRTIO_SND_PCM_CH_TFC, /* top front center */
>>> + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
>>> + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
>>> + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
>>> + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
>>> + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
>>> + VIRTIO_SND_PCM_CH_TSL, /* top side left */
>>> + VIRTIO_SND_PCM_CH_TSR, /* top side right */
>>> + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
>>> + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
>>> + VIRTIO_SND_PCM_CH_BC, /* bottom center */
>>> + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
>>> + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
>>> +};
>>> +
>>> +/* a maximum possible number of channels */
>>> +#define VIRTIO_SND_PCM_CH_MAX 256
>>> +
>>> +/* response containing a PCM channel map information */
>>> +struct virtio_snd_pcm_chmap_info {
>>> + le32 status;
>>> + le32 npositions;
>>> + u8 positions[VIRTIO_SND_PCM_CH_MAX];
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +PCM channel map information fields:
>>> +
>>> +\begin{description}
>>> +\item[\field{status}] (device-writable) specifies a device response
>>> status
>>> +(VIRTIO_SND_S_*).
>>> +\item[\field{npositions}] (device-writable) is a number of valid
>>> entries in
>>> +the \field{positions} array.
>>> +\item[\field{positions}] (device-writable) contains PCM channel
>>> positions
>>> +(VIRTIO_SND_PCM_CH_*).
>>> +\end{description}
>>> +
>>> +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
>>> +Set selected PCM format.
>>> +
>>> +\begin{lstlisting}
>>> +struct virtio_snd_pcm_set_format {
>>> + struct virtio_snd_pcm_hdr hdr;
>>> + le16 channels;
>>> + le16 format;
>>> + le16 rate;
>>> + u16 padding;
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +PCM control request fields:
>>> +
>>> +\begin{description}
>>> +\item[\field{hdr}] (device-read-only) is a PCM control request
>>> header.
>>> +\item[\field{channels}] (device-read-only) specifies a desired
>>> number of channels.
>>> +\item[\field{format}] (device-read-only) specifies a desired PCM
>>> sample format
>>> +(VIRTIO_SND_PCM_FMT_*).
>>> +\item[\field{rate}] (device-read-only) specifies a desired PCM frame
>>> rate
>>> +(VIRTIO_SND_PCM_RATE_*).
>>> +\end{description}
>>> +
>>> +\item[VIRTIO_SND_R_PCM_PREPARE]
>>> +Prepare the PCM device.
>>> +
>>> +\item[VIRTIO_SND_R_PCM_START]
>>> +Start the PCM device.
>>> +
>>> +\item[VIRTIO_SND_R_PCM_STOP]
>>> +Stop the PCM device.
>>> +
>>> +\item[VIRTIO_SND_R_PCM_PAUSE]
>>> +Set the PCM device on pause.
>>> +
>>> +\item[VIRTIO_SND_R_PCM_UNPAUSE]
>>> +Unset the PCM device from pause.
>>> +
>>> +\end{description}
>>> +
>>> +\devicenormative{\subsubsection}{PCM control requests}{Device Types
>>> / Sound Device / PCM control requests}
>>> +
>>> +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
>>> +
>>> +\begin{itemize*}
>>> +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status code,
>>> if it does not
>>> +support a channel map for a specified stream type;
>>> +\item the device MUST set the \field{npositions} field to 0, if the
>>> operation
>>> +failed.
>>> +\end{itemize*}
>>> +
>>> +\drivernormative{\subsubsection}{PCM control requests}{Device Types
>>> / Sound Device / PCM control requests}
>>> +
>>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream type
>>> if
>>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>>> +
>>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream type
>>> if
>>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>>> +
>>> +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
>>> +
>>> +\begin{itemize*}
>>> +\item the driver MUST NOT specify the \field{channels} value less
>>> than the \field{channels_min}
>>> +or greater than the \field{channels_max} values reported in stream
>>> configuration;
>>> +\item the driver MUST specify the \field{format} and \field{rate}
>>> values according
>>> +to the \field{formats} and \field{rates} values reported in stream
>>> configuration;
>>> +\item the driver MUST NOT specify the not defined format and rate
>>> values;
>>> +\item the driver MUST initialize \field{padding} bytes to 0.
>>> +\end{itemize*}
>>> +
>>> +\subsubsection{Device Operation: PCM I/O requests}
>>> +
>>> +All I/O requests are placed into the pcmq virtqueue. Each request is
>>> of form:
>>> +
>>> +\begin{lstlisting}
>>> +struct virtio_snd_pcm_xfer {
>>> + le32 stream;
>>> + u8 data[];
>>> + le32 status;
>>> + le32 actual_length;
>
>> Not following this, is actual_length the size of data[]. If so, it must
>> preceed data[].
>
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +I/O request fields:
>>> +
>>> +\begin{description}
>>> +\item[\field{stream}] (device-read-only) specifies a PCM stream type
>>> +(VIRTIO_SND_PCM_T_*).
>>> +\item[\field{data}]
>>> +\begin{enumerate}
>>> +\item Output: (device-read-only) a buffer with PCM frames to be
>>> written to the
>>> +device.
>>> +\item Input: (device-writable) a buffer to be filled with PCM frames
>>> from the
>>> +device.
>>> +\end{enumerate}
>>> +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK if
>>> an operation
>>> +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
>>> +\item[\field{actual_length}] (device-writable) specifies an actual
>>> amount of
>>> +bytes read from/written to the \field{data} field.
>>> +\end{description}
>>> +
>>> +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types /
>>> Sound Device / PCM IO requests}
>>> +
>>> +\begin{enumerate}
>>> +\item The device MUST set the \field{actual_length} field to 0, if
>>> the operation
>>> +failed.
>>> +\end{enumerate}
>>> +
>>> +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types /
>>> Sound Device / PCM IO requests}
>>> +
>>> +\begin{enumerate}
>>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
>>> stream type if
>>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
>>> type if
>>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>>> +\end{enumerate}
>
>> I don't see anything in here about dealing with
>
>> 1) Underruns and overruns. i.e. how do we recover and resync audio
>> between guests.
>
>> 2) Zero copy of audio data and stream positions. Essential for any low
>> latency audio use cases (if supported by HW, hypervisor, VM framework).
>
>> I would strongly recommend reading the ALSA driver developers guide if
>> you can or looking at the ALSA headers for PCMs and controls (I guess
>> you have looked a little based on the channel map).
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191029121810.GB5253@sirena.co.uk>
@ 2019-10-29 13:16 ` Anton Yakovlev
[not found] ` <20191030121137.GC6693@sirena.co.uk>
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-10-29 13:16 UTC (permalink / raw)
To: Mark Brown
Cc: Liam Girdwood, Mikhail Golubev, virtio-dev, Takashi Iwai,
Marc Zyngier, James Morse, Julien Thierry, Suzuki K Poulose
On 29.10.2019 13:18, Mark Brown wrote:
> On Tue, Oct 29, 2019 at 11:14:52AM +0100, Anton Yakovlev wrote:
>> On 28.10.2019 17:05, Liam Girdwood wrote:
>>> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>
>>>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>>>> + VIRTIO_SND_PCM_RATE_11025,
>>>> + VIRTIO_SND_PCM_RATE_16000,
>>>> + VIRTIO_SND_PCM_RATE_22050,
>>>> + VIRTIO_SND_PCM_RATE_32000,
>>>> + VIRTIO_SND_PCM_RATE_44100,
>>>> + VIRTIO_SND_PCM_RATE_48000,
>>>> + VIRTIO_SND_PCM_RATE_64000,
>>>> + VIRTIO_SND_PCM_RATE_88200,
>>>> + VIRTIO_SND_PCM_RATE_96000,
>>>> + VIRTIO_SND_PCM_RATE_176400,
>>>> + VIRTIO_SND_PCM_RATE_192000
>
>>> It may be best to use the same rates and format from the ALSA spec,
>>> this covers any holes (like rates that are not in the above list).
>
>> These rates are from ALSA definitions. And what formats should be added?
>
> There was one rate at the lower end which I'm not super convinced is
> useful but would be handy for making the numbers in the enum line up and
> a couple of higher rates up to 384kHz. Those probably *are* useful for
> ultrasonic applications, you may also find some regular audio files
> encoded at that rate but most of the oversampled hifi stuff tops out at
> 192kHz.
I think, you talk about 5512Hz. If it's reasonable, two more values in enum is
not a problem.
>>> What about buffer and period size capabilities ?
>
>> In this specification we assume that buffer size is up to guest decision.
>> Also, period size is ALSA-only conception and should not be mentioned here
>> at all (we are not stick only to ALSA).
>
> Period size isn't just an ALSA thing - it's a widely supported and used
> hardware feature. When using a ring buffer (or just sending large
> amounts of data at once, but mainly ring buffers) it is useful to know
> how the hardware is progressing through the data without having to use
> CPU timers (which may not be well synced with the audio clock), the
> period interrupts do that.
Periods here are kind of notification frequency, right? It's supposed to be
used for sending notifications to driver?
>>>> +\begin{lstlisting}
>>>> +struct virtio_snd_pcm_xfer {
>>>> + le32 stream;
>>>> + u8 data[];
>>>> + le32 status;
>>>> + le32 actual_length;
>
>>> Not following this, is actual_length the size of data[]. If so, it must
>>> preceed data[].
>
>> No, the actual_length field is supposed to be used by device side to report
>> actual amount of bytes read from/written to a buffer. In real world
>> scenario, if an I/O request contains N bytes, a device can play/capture *up
>> to* N bytes. Thus, it's required to report this length back to a driver.
>
> So really there's two structs here, a header struct with just stream in
> it and then a tail struct with the length and status information which
> the consumer locates by looking at the end of the buffer and working
> backwards?
Yes, it is. Since both these structures are coupled together and must be sent
at the same time, in spec they are represented as one "structure".
>>> 1) Underruns and overruns. i.e. how do we recover and resync audio
>>> between guests.
>
>> Since xrun conditions are higher level conceptions, we decided to delegate
>> such issues to guest application itself. It helps to make overall design
>> simpler. And it seems there's not so much we can do if xrun condition is
>> happened on the device side.
>
> I'm not so sure about that - in a message based system I would expect
> the device side to be able to at least say "I am throwing away data" for
> cases where it's arriving too fast to be consumed and the device ran out
> of buffer space, and ideally also say if it underflows. It's more of a
> high level concept with free running ring buffers where the device can
> just happily keep reading to or writing from the buffer regardless of if
> the host is paying attention to it.
That's the point. The spec describes a device. Device is supposed just to read
from/write to hardware buffer regardless of its content.
Also, the main problems with xrun is what to do with this information? Let's
assume that device can report something like "I'm going to run out of frames
for playback soon" or "I have no more frames for playback", then what? In low
latency scenario, user space application will provide small chunks of data. It
means, hardware buffer will be empty in like 95% of cases. And the only thing
driver can do is somehow indicate xrun to upper layer (which will be not true
from application point's of view, if hardware pointer has not crossed
application pointer yet in ALSA). I.e. exactly the same result, where there's
no notification from the device side.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191030121137.GC6693@sirena.co.uk>
@ 2019-11-01 13:37 ` Anton Yakovlev
[not found] ` <20191111193903.GE4264@sirena.co.uk>
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-01 13:37 UTC (permalink / raw)
To: Mark Brown
Cc: Liam Girdwood, Mikhail Golubev, virtio-dev, Takashi Iwai,
Marc Zyngier, James Morse, Julien Thierry, Suzuki K Poulose
On 30.10.2019 13:11, Mark Brown wrote:
> On Tue, Oct 29, 2019 at 02:16:04PM +0100, Anton Yakovlev wrote:
>> On 29.10.2019 13:18, Mark Brown wrote:
>>> On Tue, Oct 29, 2019 at 11:14:52AM +0100, Anton Yakovlev wrote:
>>>> On 28.10.2019 17:05, Liam Girdwood wrote:
>
>>>>> What about buffer and period size capabilities ?
>
>>>> In this specification we assume that buffer size is up to guest decision.
>>>> Also, period size is ALSA-only conception and should not be mentioned here
>>>> at all (we are not stick only to ALSA).
>
>>> Period size isn't just an ALSA thing - it's a widely supported and used
>>> hardware feature. When using a ring buffer (or just sending large
>>> amounts of data at once, but mainly ring buffers) it is useful to know
>>> how the hardware is progressing through the data without having to use
>>> CPU timers (which may not be well synced with the audio clock), the
>>> period interrupts do that.
>
>> Periods here are kind of notification frequency, right? It's supposed to be
>> used for sending notifications to driver?
>
> Yes, the driver gets a notification every time the hardware makes it
> through a periodn of data.
If a device implementation has reliable way to receive and forward such
notifications to a driver, then it's fine. The reason why we didn't add this
to the spec is because usually a device interacts with or implements by itself
some kind of software mixer. And all communications with real hardware are
hidden behind multiple abstraction layers. In such case, it's either very
difficult or even impossible to map period-based notifications to whatever a
sw mixer provides to its clients.
Perhaps it could be an optional feature.
>>>>>> +struct virtio_snd_pcm_xfer {
>>>>>> + le32 stream;
>>>>>> + u8 data[];
>>>>>> + le32 status;
>>>>>> + le32 actual_length;
>
>>>>> Not following this, is actual_length the size of data[]. If so, it must
>>>>> preceed data[].
>
>>>> No, the actual_length field is supposed to be used by device side to report
>>>> actual amount of bytes read from/written to a buffer. In real world
>>>> scenario, if an I/O request contains N bytes, a device can play/capture *up
>>>> to* N bytes. Thus, it's required to report this length back to a driver.
>
>>> So really there's two structs here, a header struct with just stream in
>>> it and then a tail struct with the length and status information which
>>> the consumer locates by looking at the end of the buffer and working
>>> backwards?
>
>> Yes, it is. Since both these structures are coupled together and must be
>> sent at the same time, in spec they are represented as one "structure".
>
> OK... that is a fairly weird and confusing way of writing things.
Not sure, how to describe this in context of the specification in a better way.
>>>> Since xrun conditions are higher level conceptions, we decided to delegate
>>>> such issues to guest application itself. It helps to make overall design
>>>> simpler. And it seems there's not so much we can do if xrun condition is
>>>> happened on the device side.
>
>>> I'm not so sure about that - in a message based system I would expect
>>> the device side to be able to at least say "I am throwing away data" for
>>> cases where it's arriving too fast to be consumed and the device ran out
>>> of buffer space, and ideally also say if it underflows. It's more of a
>>> high level concept with free running ring buffers where the device can
>>> just happily keep reading to or writing from the buffer regardless of if
>>> the host is paying attention to it.
>
>> That's the point. The spec describes a device. Device is supposed just to
>> read from/write to hardware buffer regardless of its content.
>
> The rate at which data is consumed or produced in an audio system is
> driven by the audio hardware. This is very likely to not be synced to
> other clock domains in the system so users are reliant on the hardware
> to drive the rate at which data is driven through the system and hence
> need support from it for figuring out the rate to transfer data and
> handle any problems that occur. Good error reporting is a part of that,
> it makes it much easier to diagnose problems if you can get a direct
> and clear report of what went wrong rather than having to for example
> listen to the audio and work back from that. Like I say this is
> especially true with a message based system where software has less
> visibility of how the hardware is progressing through the buffer.
Yes, it's true. And it's not like we against having error reporting. The
problem is there's no deterministic way how to treat such errors in pv-solution.
It's like with periods discussed above. If a device is a native
ALSA-application and talks directly with hardware in exclusive mode, then it's
capable to signal actual underflow/overflow condition reported by ALSA layer.
But if a device is a client of sw mixer server, then reported error might be
server-specific (like "underflow" means, that server's queue is empty, but
real hardware buffer still might contain 10ms or 50ms of frames for playback;
i.e. guest application probably has enough time to provide new data).
I would say it's an issue with mixing together driver and device context,
where driver context is always predictable, but device' - not. And sometimes
it can lead to unexpected behavior.
At the end, the main source of xrun conditions is delayed execution of a
virtual machine or an application in it (like double scheduling and so on).
And 44100Hz is still ~44100 frames per seconds. After different experiments,
we decided just to stick to simpler design.
Again, it could be an optional feature if device has reliable source of error
notifications.
>> Also, the main problems with xrun is what to do with this information? Let's
>> assume that device can report something like "I'm going to run out of frames
>> for playback soon" or "I have no more frames for playback", then what? In
>> low latency scenario, user space application will provide small chunks of
>> data. It means, hardware buffer will be empty in like 95% of cases. And the
>> only thing driver can do is somehow indicate xrun to upper layer (which will
>> be not true from application point's of view, if hardware pointer has not
>> crossed application pointer yet in ALSA). I.e. exactly the same result,
>> where there's no notification from the device side.
>
> Error handling is the application's problem, usually it will either give
> up entirely or restart the stream. Restarting is useful since if you've
> got a situation where clocks have drifted apart what'll often happen is
> that you'll get constant glitches which are extremely distracting to
> users, when you restart the drift gets reset and things will be OK for
> as long as it takes for drift to reaccumilate. Obviously that's not
> great and the system should be trying to avoid it but it's at least
> mitigation and like I say the explicit notification that something went
> wrong can be very helpful in diagnosing problems.
Yes, I agree. I described our concerns in a comment above.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-10-29 9:42 ` Anton Yakovlev
2019-10-29 10:14 ` Anton Yakovlev
@ 2019-11-11 15:20 ` Liam Girdwood
2019-11-12 11:09 ` Jean-Philippe Brucker
2019-11-12 12:45 ` Anton Yakovlev
1 sibling, 2 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-11-11 15:20 UTC (permalink / raw)
To: Anton Yakovlev, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
On Tue, 2019-10-29 at 10:42 +0100, Anton Yakovlev wrote:
> Hi Liam,
>
> Thank you for comments! Please, take a look at our answers below.
Apologies, been travelling. More comments below.
>
>
> On 28.10.2019 17:05, Liam Girdwood wrote:
> > On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
> > > From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
> > >
> > > This patch proposes virtio specification for new virtio sound
> > > device.
> > >
> > > The virtio sound card is a virtual audio device supporting output
> > > and
> > > input PCM streams.
> > >
> >
> > Apologies for the delay in response, I've just been made aware of
> > this
> > spec proposal on this list. I've also added Mark and Takashi.
> >
> > Just to make you aware I'm working on a spec for DSP
> > virtualisation,
> > where the DSP can do more than just simple audio uses cases. I
> > intend
> > to support this spec as a fallback for guests who request non DSP
> > audio
> > functionality, hence my interest :)
> >
> > This spec looks in the right direction for virtualising non DSP
> > audio
> > devices (where we have HW for DMAC, I2S/HDA and a codec IC). There
> > are
> > a few opens that I've detailed below.
>
> Originally, this spec was supposed to be extendable. Since PCM is the
> simplest
> and most demandable feature, we decided to start only with this one.
>
>
> > > Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
> > > Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
> > > ---
> > > Fixes from RFC:
> > > * Use fixed virtqueue numbers for the control and PCM queues
> > > * Use device feature bits to indicate available input/output
> > > PCM
> > > streams
> > > * Place PCM stream configuration into the device configuration
> > > space
> > > * Add sound device and driver requirements into the "Device
> > > Initialization"
> > > subsection
> > > * Add sound device and driver requirements into the "Device
> > > Operation" subsection
> > > * Add sound device and driver conformance subsections
> > >
> > > RFC link:
> > >
https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
> > >
> > > conformance.tex | 20 +++
> > > content.tex | 1 +
> > > virtio-sound.tex | 405
> > > +++++++++++++++++++++++++++++++++++++++++++++++
> > > 3 files changed, 426 insertions(+)
> > > create mode 100644 virtio-sound.tex
> > >
> > > diff --git a/conformance.tex b/conformance.tex
> > > index 0ac58aa..31595bf 100644
> > > --- a/conformance.tex
> > > +++ b/conformance.tex
> > > @@ -183,6 +183,16 @@ \section{Conformance
> > > Targets}\label{sec:Conformance / Conformance Targets}
> > > \item \ref{drivernormative:Device Types / Socket Device /
> > > Device
> > > Operation / Device Events}
> > > \end{itemize}
> > >
> > > +\conformance{\subsection}{Sound Driver
> > > Conformance}\label{sec:Conformance / Driver Conformance / Sound
> > > Driver Conformance}
> > > +
> > > +A sound driver MUST conform to the following normative
> > > statements:
> > > +
> > > +\begin{itemize}
> > > +\item \ref{drivernormative:Device Types / Sound Device / Device
> > > Initialization}
> > > +\item \ref{drivernormative:Device Types / Sound Device / PCM
> > > control
> > > requests}
> > > +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
> > > requests}
> > > +\end{itemize}
> > > +
> > > \conformance{\section}{Device
> > > Conformance}\label{sec:Conformance /
> > > Device Conformance}
> > >
> > > A device MUST conform to the following normative statements:
> > > @@ -338,6 +348,16 @@ \section{Conformance
> > > Targets}\label{sec:Conformance / Conformance Targets}
> > > \item \ref{devicenormative:Device Types / Socket Device /
> > > Device
> > > Operation / Receive and Transmit}
> > > \end{itemize}
> > >
> > > +\conformance{\subsection}{Sound Device
> > > Conformance}\label{sec:Conformance / Device Conformance / Sound
> > > Device Conformance}
> > > +
> > > +A sound device MUST conform to the following normative
> > > statements:
> > > +
> > > +\begin{itemize}
> > > +\item \ref{devicenormative:Device Types / Sound Device / Device
> > > Initialization}
> > > +\item \ref{devicenormative:Device Types / Sound Device / PCM
> > > control
> > > requests}
> > > +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
> > > requests}
> > > +\end{itemize}
> > > +
> > > \conformance{\section}{Legacy Interface: Transitional Device
> > > and
> > > Transitional Driver Conformance}\label{sec:Conformance / Legacy
> > > Interface: Transitional Device and Transitional Driver
> > > Conformance}
> > > A conformant implementation MUST be either transitional or
> > > non-transitional, see \ref{intro:Legacy
> > > diff --git a/content.tex b/content.tex
> > > index 37a2190..4a00884 100644
> > > --- a/content.tex
> > > +++ b/content.tex
> > > @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
> > > Requirements}\label{sec:Device
> > > \input{virtio-input.tex}
> > > \input{virtio-crypto.tex}
> > > \input{virtio-vsock.tex}
> > > +\input{virtio-sound.tex}
> > >
> > > \chapter{Reserved Feature Bits}\label{sec:Reserved Feature
> > > Bits}
> > >
> > > diff --git a/virtio-sound.tex b/virtio-sound.tex
> > > new file mode 100644
> > > index 0000000..68a606e
> > > --- /dev/null
> > > +++ b/virtio-sound.tex
> > > @@ -0,0 +1,405 @@
> > > +\section{Sound Device}\label{sec:Device Types / Sound Device}
> > > +
> > > +The virtio sound card is a virtual audio device supporting
> > > output
> > > and input PCM
> > > +streams. All device control requests are placed into the control
> > > virtqueue, all
> > > +I/O requests are placed into the PCM virtqueue.
> >
> > Where are stream position updates placed ? These are usually
> > synchronous from the driver and enable the player application to
> > render
> > or read data from the correct locations in the buffers. This is
> > latency
> > sensitive data.
>
> It happens in an interrupt handler. Since PCM frame transfer is
> message based,
> the only possible way to figure out consumed or captured amount of
> frames is
> to take a look into a message response.
>
Ok, this is what I thought. This wont work for low latency audio use
cases since the buffers and position responses are queued (voice calls,
some gaming, some system notifications like keypress or volume
change).
However, it's probably tolerable for most other use cases. What use
cases have you tried in your testing ?
>
> > > +
> > > +\subsection{Device ID}\label{sec:Device Types / Sound Device /
> > > Device ID}
> > > +
> > > +25
> > > +
> > > +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
> > > Virtqueues}
> > > +
> > > +\begin{description}
> > > +\item[0] controlq
> > > +\item[1] pcmq
> > > +\end{description}
> > > +
> > > +The controlq virtqueue always exists, the pcmq virtqueue only
> > > exists
> > > if
> > > +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT
> > > feature is
> > > negotiated.
> > > +
> > > +\subsection{Feature bits}\label{sec:Device Types / Sound Device
> > > /
> > > Feature bits}
> > > +
> > > +\begin{description}
> > > +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
> > > +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
> > > +\end{description}
> > > +
> > > +\subsection{Device configuration layout}\label{sec:Device Types
> > > /
> > > Sound Device / Device configuration layout}
> > > +
> > > +\begin{lstlisting}
> > > +/* supported PCM sample formats */
> > > +enum {
> > > + VIRTIO_SND_PCM_FMT_S8 = 0,
> > > + VIRTIO_SND_PCM_FMT_U8,
> > > + VIRTIO_SND_PCM_FMT_S16,
> > > + VIRTIO_SND_PCM_FMT_U16,
24 bit.
> > > + VIRTIO_SND_PCM_FMT_S32,
> > > + VIRTIO_SND_PCM_FMT_U32,
> > > + VIRTIO_SND_PCM_FMT_FLOAT,
> > > + VIRTIO_SND_PCM_FMT_FLOAT64
> > > +};
> > > +
> > > +/* supported PCM frame rates */
> > > +enum {
> > > + VIRTIO_SND_PCM_RATE_8000 = 0,
> > > + VIRTIO_SND_PCM_RATE_11025,
> > > + VIRTIO_SND_PCM_RATE_16000,
> > > + VIRTIO_SND_PCM_RATE_22050,
> > > + VIRTIO_SND_PCM_RATE_32000,
> > > + VIRTIO_SND_PCM_RATE_44100,
> > > + VIRTIO_SND_PCM_RATE_48000,
> > > + VIRTIO_SND_PCM_RATE_64000,
> > > + VIRTIO_SND_PCM_RATE_88200,
> > > + VIRTIO_SND_PCM_RATE_96000,
> > > + VIRTIO_SND_PCM_RATE_176400,
> > > + VIRTIO_SND_PCM_RATE_192000
> >
> > It may be best to use the same rates and format from the ALSA spec,
> > this covers any holes (like rates that are not in the above list).
>
> These rates are from ALSA definitions. And what formats should be
> added?
>
Rate, Knot - I would copy ALSA supported rates and formats just like
with the channel maps.
>
> > > +};
> > > +
> > > +/* a PCM stream configuration */
> > > +struct virtio_pcm_stream_config {
> > > + u8 channels_min;
> > > + u8 channels_max;
> > > + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
> > > + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
> > > + u16 padding;
> > > +};
> >
> > Should this be packed ? Btw, padding alignment is to a non 32bit
> > size ?
>
> It should be naturally packed if padding is correct. And do you mean
> 8 and
> 16-bit fields? You can read one or two bytes from device
> configuration space,
> it should not be a problem.
>
>
> > What about buffer/period formats e.g interleaved ?
>
> Yeah, support only for interleaved layout previously was stated in
> the
> specification but occasionally was lost. Will be fixed!
>
>
> >
> > > +
> > > +/* a device configuration space */
> > > +struct virtio_snd_config {
> > > + struct virtio_pcm_config {
> > > + struct virtio_pcm_stream_config output;
> > > + struct virtio_pcm_stream_config input;
> > > + } pcm;
> > > +};
> > > +\end{lstlisting}
> >
> > I assuming this is sent to configure PCM capabilities ?
>
> Yes, minimum amount of information required to setup a PCM stream in
> any kind
> of guest.
>
>
> > What about buffer and period size capabilities ?
>
> In this specification we assume that buffer size is up to guest
> decision.
> Also, period size is ALSA-only conception and should not be mentioned
> here at
> all (we are not stick only to ALSA).
>
>
Period size == fragment size == block size (probably a few other naming
conventions too), we should be able to tell the device VM our
period/block/fragment size so we can configure the HW appropriately.
This can also be used to make our stream copying more efficient too.
> > > +
> > > +\subsubsection{Device configuration fields}
> > > +
> > > +The \field{pcm.output} and \field{pcm.input} fields contain PCM
> > > stream
> > > +configuration:
> > > +
> > > +\begin{description}
> > > +\item[\field{channels_min}] (driver-read-only) is a minimum
> > > number
> > > of supported
> > > +channels.
> > > +\item[\field{channels_max}] (driver-read-only) is a maximum
> > > number
> > > of supported
> > > +channels.
> > > +\item[\field{formats}] (driver-read-only) is supported sample
> > > format
> > > bit map.
> > > +\item[\field{rates}] (driver-read-only) is supported frame rate
> > > bit
> > > map.
> > > +\end{description}
> > > +
> > > +\subsection{Device Initialization}
> > > +
> > > +\begin{enumerate}
> > > +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
> > > \field{pcm.output}
> > > +contains valid output PCM stream configuration.
> > > +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
> > > \field{pcm.input}
> > > +contains valid input PCM stream configuration.
> > > +\end{enumerate}
> > > +
> > > +\devicenormative{\subsubsection}{Device Initialization}{Device
> > > Types
> > > / Sound Device / Device Initialization}
> > > +
> > > +\begin{enumerate}
> > > +\item The device MUST NOT set the not defined format and rate
> > > bits.
> > > +\item The device MUST initialize padding bytes
> > > \field{pcm.output.padding} and
> > > +\field{pcm.input.padding} to 0.
> > > +\end{enumerate}
> > > +
> > > +\drivernormative{\subsubsection}{Device Initialization}{Device
> > > Types
> > > / Sound Device / Device Initialization}
> > > +
> > > +\begin{enumerate}
> > > +\item The driver MUST configure and initialize all virtqueues.
> > > +\item The driver SHOULD ignore the not defined format and rate
> > > bits.
> > > +\end{enumerate}
> > > +
> > > +\subsection{Device Operation}\label{sec:Device Types / Sound
> > > Device
> > > / Device Operation}
> > > +
> > > +All control messages are placed into the controlq virtqueue and
> > > use
> > > the following
> > > +layout structure and definitions:
> > > +
> > > +\begin{lstlisting}
> > > +enum {
> > > + /* PCM control request types */
> > > + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
> > > + VIRTIO_SND_R_PCM_SET_FORMAT,
> > > + VIRTIO_SND_R_PCM_PREPARE,
> > > + VIRTIO_SND_R_PCM_START,
> > > + VIRTIO_SND_R_PCM_STOP,
> > > + VIRTIO_SND_R_PCM_PAUSE,
> > > + VIRTIO_SND_R_PCM_UNPAUSE,
> > > +
> > > + /* generic status codes */
> > > + VIRTIO_SND_S_OK = 0x8000,
> > > + VIRTIO_SND_S_BAD_MSG,
> > > + VIRTIO_SND_S_NOT_SUPP,
> > > + VIRTIO_SND_S_IO_ERR
> > > +};
> > > +
> > > +struct virtio_snd_ctl_msg {
> > > + /* device-read-only data */
> > > + le32 request_code;
> > > + u8 request_payload[];
> >
> > How do I know how much data to read here ? Is size embedded in
> > request_code ?
>
> Yes, you are right. Actual request/response length can be easily
> derived from
> a request_code.
>
Nak, please use a separate size and type code otherwise we will have
real problems in the future when we come to add new features (with new
codes or bigger sizes).
>
> >
> > > + /* device-writable data */
> > > + le32 response_status;
> > > + u8 response_payload[];
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +A generic control message consists of request and response parts
> > > and
> > > contains
> > > +the following fields:
> > > +
> > > +\begin{description}
> > > +\item[\field{request_code}] (device-read-only) specifies a
> > > device
> > > request code
> > > +(VIRTIO_SND_R_*).
> > > +\item[\field{request_payload}] (device-read-only) contains
> > > request-
> > > specific
> > > +data.
> > > +\item[\field{response_status}] (device-writable) specifies a
> > > device
> > > response
> > > +status (VIRTIO_SND_S_*).
> > > +\item[\field{response_payload}] (device-writable) contains
> > > response-
> > > specific
> > > +data.
> > > +\end{description}
> > > +
> > > +Unless stated otherwise, the \field{request_payload} and
> > > \field{response_payload}
> > > +fields are empty.
> > > +
> > > +The \field{response_status} field contains one of the following
> > > values:
> > > +
> > > +\begin{itemize*}
> > > +\item VIRTIO_SND_S_OK: success.
> > > +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
> > > contains invalid
> > > +parameters.
> > > +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters
> > > are
> > > not supported.
> > > +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
> > > +\end{itemize*}
> > > +
> > > +\subsubsection{Device Operation: PCM control requests}
> > > +
> > > +A PCM stream has the following command lifecycle:
> > > +
> > > +\begin{enumerate}
> > > +\item Set format
> > > +\item Prepare
> > > +\item Output only: transfer data for prebuffing
> > > +\item Start
> > > +\item Transfer data to/from the PCM device
> > > +\begin{enumerate}
> > > + \item Pause
> > > + \item Unpause
> > > +\end{enumerate}
> > > +\item Stop
> > > +\end{enumerate}
> > > +
> > > +PCM control requests have or consist of a fixed header with the
> > > following
> > > +layout structure:
> > > +
> > > +\begin{lstlisting}
> > > +/* supported PCM stream types */
> > > +enum {
> > > + VIRTIO_SND_PCM_T_OUTPUT = 0,
> > > + VIRTIO_SND_PCM_T_INPUT
> >
> > Can we rename these PLAYBACK and CAPTURE
>
> Playback and capture are used in ALSA drivers, sink and source are
> used in
> Windows drivers. We decided to stick to something obvious in-between
> (and it's
> just shorter). Although, it's still discussable topic.
>
>
> > > +};
> > > +
> > > +/* PCM control request header */
> > > +struct virtio_snd_pcm_hdr {
> > > + le32 code;
> > > + le32 stream;
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +PCM control request fields:
> > > +
> > > +\begin{description}
> > > +\item[\field{code}] (device-read-only) specifies a PCM device
> > > request code
> > > +(VIRTIO_SND_R_PCM_*).
> > > +\item[\field{stream}] (device-read-only) specifies a PCM stream
> > > type
> > > +(VIRTIO_SND_PCM_T_*).
> > > +\end{description}
> > > +
> > > +\begin{description}
> > > +
> > > +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
> > > +Query a PCM channel map information for specified stream type.
> > > +
> > > +A response uses the following layout structure and definitions:
> > > +
> > > +\begin{lstlisting}
> > > +/* standard channel position definition */
> > > +enum {
> > > + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
> > > + VIRTIO_SND_PCM_CH_NA, /* silent */
> > > + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
> > > + VIRTIO_SND_PCM_CH_FL, /* front left */
> > > + VIRTIO_SND_PCM_CH_FR, /* front right */
> > > + VIRTIO_SND_PCM_CH_RL, /* rear left */
> > > + VIRTIO_SND_PCM_CH_RR, /* rear right */
> > > + VIRTIO_SND_PCM_CH_FC, /* front center */
> > > + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
> > > + VIRTIO_SND_PCM_CH_SL, /* side left */
> > > + VIRTIO_SND_PCM_CH_SR, /* side right */
> > > + VIRTIO_SND_PCM_CH_RC, /* rear center */
> > > + VIRTIO_SND_PCM_CH_FLC, /* front left center */
> > > + VIRTIO_SND_PCM_CH_FRC, /* front right center */
> > > + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
> > > + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
> > > + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
> > > + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
> > > + VIRTIO_SND_PCM_CH_FLH, /* front left high */
> > > + VIRTIO_SND_PCM_CH_FCH, /* front center high */
> > > + VIRTIO_SND_PCM_CH_FRH, /* front right high */
> > > + VIRTIO_SND_PCM_CH_TC, /* top center */
> > > + VIRTIO_SND_PCM_CH_TFL, /* top front left */
> > > + VIRTIO_SND_PCM_CH_TFR, /* top front right */
> > > + VIRTIO_SND_PCM_CH_TFC, /* top front center */
> > > + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
> > > + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
> > > + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
> > > + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
> > > + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
> > > + VIRTIO_SND_PCM_CH_TSL, /* top side left */
> > > + VIRTIO_SND_PCM_CH_TSR, /* top side right */
> > > + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
> > > + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
> > > + VIRTIO_SND_PCM_CH_BC, /* bottom center */
> > > + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
> > > + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
> > > +};
> > > +
> > > +/* a maximum possible number of channels */
> > > +#define VIRTIO_SND_PCM_CH_MAX 256
> > > +
> > > +/* response containing a PCM channel map information */
> > > +struct virtio_snd_pcm_chmap_info {
> > > + le32 status;
> > > + le32 npositions;
> > > + u8 positions[VIRTIO_SND_PCM_CH_MAX];
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +PCM channel map information fields:
> > > +
> > > +\begin{description}
> > > +\item[\field{status}] (device-writable) specifies a device
> > > response
> > > status
> > > +(VIRTIO_SND_S_*).
> > > +\item[\field{npositions}] (device-writable) is a number of valid
> > > entries in
> > > +the \field{positions} array.
> > > +\item[\field{positions}] (device-writable) contains PCM channel
> > > positions
> > > +(VIRTIO_SND_PCM_CH_*).
> > > +\end{description}
> > > +
> > > +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
> > > +Set selected PCM format.
> > > +
> > > +\begin{lstlisting}
> > > +struct virtio_snd_pcm_set_format {
> > > + struct virtio_snd_pcm_hdr hdr;
> > > + le16 channels;
> > > + le16 format;
> > > + le16 rate;
> > > + u16 padding;
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +PCM control request fields:
> > > +
> > > +\begin{description}
> > > +\item[\field{hdr}] (device-read-only) is a PCM control request
> > > header.
> > > +\item[\field{channels}] (device-read-only) specifies a desired
> > > number of channels.
> > > +\item[\field{format}] (device-read-only) specifies a desired PCM
> > > sample format
> > > +(VIRTIO_SND_PCM_FMT_*).
> > > +\item[\field{rate}] (device-read-only) specifies a desired PCM
> > > frame
> > > rate
> > > +(VIRTIO_SND_PCM_RATE_*).
> > > +\end{description}
> > > +
> > > +\item[VIRTIO_SND_R_PCM_PREPARE]
> > > +Prepare the PCM device.
> > > +
> > > +\item[VIRTIO_SND_R_PCM_START]
> > > +Start the PCM device.
> > > +
> > > +\item[VIRTIO_SND_R_PCM_STOP]
> > > +Stop the PCM device.
> > > +
> > > +\item[VIRTIO_SND_R_PCM_PAUSE]
> > > +Set the PCM device on pause.
> > > +
> > > +\item[VIRTIO_SND_R_PCM_UNPAUSE]
> > > +Unset the PCM device from pause.
> > > +
> > > +\end{description}
> > > +
> > > +\devicenormative{\subsubsection}{PCM control requests}{Device
> > > Types
> > > / Sound Device / PCM control requests}
> > > +
> > > +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
> > > +
> > > +\begin{itemize*}
> > > +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status
> > > code,
> > > if it does not
> > > +support a channel map for a specified stream type;
> > > +\item the device MUST set the \field{npositions} field to 0, if
> > > the
> > > operation
> > > +failed.
> > > +\end{itemize*}
> > > +
> > > +\drivernormative{\subsubsection}{PCM control requests}{Device
> > > Types
> > > / Sound Device / PCM control requests}
> > > +
> > > +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream
> > > type
> > > if
> > > +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
> > > +
> > > +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
> > > type
> > > if
> > > +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
> > > +
> > > +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
> > > +
> > > +\begin{itemize*}
> > > +\item the driver MUST NOT specify the \field{channels} value
> > > less
> > > than the \field{channels_min}
> > > +or greater than the \field{channels_max} values reported in
> > > stream
> > > configuration;
> > > +\item the driver MUST specify the \field{format} and
> > > \field{rate}
> > > values according
> > > +to the \field{formats} and \field{rates} values reported in
> > > stream
> > > configuration;
> > > +\item the driver MUST NOT specify the not defined format and
> > > rate
> > > values;
> > > +\item the driver MUST initialize \field{padding} bytes to 0.
> > > +\end{itemize*}
> > > +
> > > +\subsubsection{Device Operation: PCM I/O requests}
> > > +
> > > +All I/O requests are placed into the pcmq virtqueue. Each
> > > request is
> > > of form:
> > > +
> > > +\begin{lstlisting}
> > > +struct virtio_snd_pcm_xfer {
> > > + le32 stream;
> > > + u8 data[];
> > > + le32 status;
> > > + le32 actual_length;
> >
> > Not following this, is actual_length the size of data[]. If so, it
> > must
> > preceed data[].
>
> No, the actual_length field is supposed to be used by device side to
> report
> actual amount of bytes read from/written to a buffer. In real world
> scenario,
> if an I/O request contains N bytes, a device can play/capture *up to*
> N bytes.
> Thus, it's required to report this length back to a driver.
>
This is not how PCM audio buffering works. I don't copy extra padding
if I dont need to, I only copy when my buffer is full. See the need for
period size in earlier comments.
>
> >
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +I/O request fields:
> > > +
> > > +\begin{description}
> > > +\item[\field{stream}] (device-read-only) specifies a PCM stream
> > > type
> > > +(VIRTIO_SND_PCM_T_*).
> > > +\item[\field{data}]
> > > +\begin{enumerate}
> > > +\item Output: (device-read-only) a buffer with PCM frames to be
> > > written to the
> > > +device.
> > > +\item Input: (device-writable) a buffer to be filled with PCM
> > > frames
> > > from the
> > > +device.
> > > +\end{enumerate}
> > > +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK
> > > if
> > > an operation
> > > +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
> > > +\item[\field{actual_length}] (device-writable) specifies an
> > > actual
> > > amount of
> > > +bytes read from/written to the \field{data} field.
> > > +\end{description}
> > > +
> > > +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types
> > > /
> > > Sound Device / PCM IO requests}
> > > +
> > > +\begin{enumerate}
> > > +\item The device MUST set the \field{actual_length} field to 0,
> > > if
> > > the operation
> > > +failed.
> > > +\end{enumerate}
> > > +
> > > +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types
> > > /
> > > Sound Device / PCM IO requests}
> > > +
> > > +\begin{enumerate}
> > > +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
> > > stream type if
> > > +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
> > > +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a
> > > stream
> > > type if
> > > +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
> > > +\end{enumerate}
> > > --
> > > 2.23.0
> > >
> >
> > I don't see anything in here about dealing with
> >
> > 1) Underruns and overruns. i.e. how do we recover and resync audio
> > between guests.
>
> Since xrun conditions are higher level conceptions, we decided to
> delegate
> such issues to guest application itself. It helps to make overall
> design
> simpler. And it seems there's not so much we can do if xrun condition
> is
> happened on the device side.
We can inform the guest. The guest userspace can then take any remedial
action if needed.
>
>
> > 2) Zero copy of audio data and stream positions. Essential for any
> > low
> > latency audio use cases (if supported by HW, hypervisor, VM
> > framework).
>
> What do you mean by zero-copy? Shared memory between device and
> driver sides?
Yes, but I see this is now a separate thread as it used by others too.
Thanks
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-11 15:20 ` Liam Girdwood
@ 2019-11-12 11:09 ` Jean-Philippe Brucker
2019-11-12 14:20 ` Liam Girdwood
2019-11-12 12:45 ` Anton Yakovlev
1 sibling, 1 reply; 42+ messages in thread
From: Jean-Philippe Brucker @ 2019-11-12 11:09 UTC (permalink / raw)
To: Liam Girdwood
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
Hi,
On Mon, Nov 11, 2019 at 03:20:55PM +0000, Liam Girdwood wrote:
> > > > +struct virtio_snd_ctl_msg {
> > > > + /* device-read-only data */
> > > > + le32 request_code;
> > > > + u8 request_payload[];
> > >
> > > How do I know how much data to read here ? Is size embedded in
> > > request_code ?
> >
> > Yes, you are right. Actual request/response length can be easily
> > derived from
> > a request_code.
> >
>
> Nak, please use a separate size and type code otherwise we will have
> real problems in the future when we come to add new features (with new
> codes or bigger sizes).
Other virtio devices rely on the buffer length already provided by virtio
for this. For example see the virtio-net control virtqueue
(virtio_net_ctrl*), which is extended with new commands from time to time.
Most recently, the "receive-side scaling" feature introduces a large
structure as payload to virtio_net_ctrl.
[...]
> > > > +\begin{lstlisting}
> > > > +struct virtio_snd_pcm_xfer {
> > > > + le32 stream;
> > > > + u8 data[];
> > > > + le32 status;
> > > > + le32 actual_length;
> > >
> > > Not following this, is actual_length the size of data[]. If so, it
> > > must
> > > preceed data[].
> >
> > No, the actual_length field is supposed to be used by device side to
> > report
> > actual amount of bytes read from/written to a buffer.
By the way the written size is already provided by virtio (field len in
the used descriptor), but not the read size.
> > In real world
> > scenario,
> > if an I/O request contains N bytes, a device can play/capture *up to*
> > N bytes.
> > Thus, it's required to report this length back to a driver.
> >
>
> This is not how PCM audio buffering works. I don't copy extra padding
> if I dont need to, I only copy when my buffer is full. See the need for
> period size in earlier comments.
[...]
> > > 2) Zero copy of audio data and stream positions. Essential for any
> > > low
> > > latency audio use cases (if supported by HW, hypervisor, VM
> > > framework).
> >
> > What do you mean by zero-copy? Shared memory between device and
> > driver sides?
>
> Yes, but I see this is now a separate thread as it used by others too.
The virtio transport already permits zero-copy, because one buffer can be
described with a chain of descriptors, each pointing to a different area
in memory. The driver can split the virtio_snd_pcm_xfer into header and
data:
virtqueue
desc ring
| |
+-------+
| |---------> header (copied)
+- - - -+
| |---------> PCM data (mapped)
+-------+
| |
So it might be a good idea to put all the metadata (stream, status,
actual_length) at the beginning of the virtio_snd_pcm_xfer struct,
otherwise the driver will need three descriptors instead of two to achieve
this.
Thanks,
Jean
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-11 15:20 ` Liam Girdwood
2019-11-12 11:09 ` Jean-Philippe Brucker
@ 2019-11-12 12:45 ` Anton Yakovlev
2019-11-12 15:16 ` Liam Girdwood
1 sibling, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-12 12:45 UTC (permalink / raw)
To: Liam Girdwood, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
On 11.11.2019 16:20, Liam Girdwood wrote:
> On Tue, 2019-10-29 at 10:42 +0100, Anton Yakovlev wrote:
>> Hi Liam,
>>
>> Thank you for comments! Please, take a look at our answers below.
>
> Apologies, been travelling. More comments below.
>
>>
>>
>> On 28.10.2019 17:05, Liam Girdwood wrote:
>>> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>>>> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>>>
>>>> This patch proposes virtio specification for new virtio sound
>>>> device.
>>>>
>>>> The virtio sound card is a virtual audio device supporting output
>>>> and
>>>> input PCM streams.
>>>>
>>>
>>> Apologies for the delay in response, I've just been made aware of
>>> this
>>> spec proposal on this list. I've also added Mark and Takashi.
>>>
>>> Just to make you aware I'm working on a spec for DSP
>>> virtualisation,
>>> where the DSP can do more than just simple audio uses cases. I
>>> intend
>>> to support this spec as a fallback for guests who request non DSP
>>> audio
>>> functionality, hence my interest :)
>>>
>>> This spec looks in the right direction for virtualising non DSP
>>> audio
>>> devices (where we have HW for DMAC, I2S/HDA and a codec IC). There
>>> are
>>> a few opens that I've detailed below.
>>
>> Originally, this spec was supposed to be extendable. Since PCM is the
>> simplest
>> and most demandable feature, we decided to start only with this one.
>>
>>
>>>> Signed-off-by: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>>> Signed-off-by: Mikhail Golubev <Mikhail.Golubev@opensynergy.com>
>>>> ---
>>>> Fixes from RFC:
>>>> * Use fixed virtqueue numbers for the control and PCM queues
>>>> * Use device feature bits to indicate available input/output
>>>> PCM
>>>> streams
>>>> * Place PCM stream configuration into the device configuration
>>>> space
>>>> * Add sound device and driver requirements into the "Device
>>>> Initialization"
>>>> subsection
>>>> * Add sound device and driver requirements into the "Device
>>>> Operation" subsection
>>>> * Add sound device and driver conformance subsections
>>>>
>>>> RFC link:
>>>>
> https://lists.oasis-open.org/archives/virtio-dev/201908/msg00078.html
>>>>
>>>> conformance.tex | 20 +++
>>>> content.tex | 1 +
>>>> virtio-sound.tex | 405
>>>> +++++++++++++++++++++++++++++++++++++++++++++++
>>>> 3 files changed, 426 insertions(+)
>>>> create mode 100644 virtio-sound.tex
>>>>
>>>> diff --git a/conformance.tex b/conformance.tex
>>>> index 0ac58aa..31595bf 100644
>>>> --- a/conformance.tex
>>>> +++ b/conformance.tex
>>>> @@ -183,6 +183,16 @@ \section{Conformance
>>>> Targets}\label{sec:Conformance / Conformance Targets}
>>>> \item \ref{drivernormative:Device Types / Socket Device /
>>>> Device
>>>> Operation / Device Events}
>>>> \end{itemize}
>>>>
>>>> +\conformance{\subsection}{Sound Driver
>>>> Conformance}\label{sec:Conformance / Driver Conformance / Sound
>>>> Driver Conformance}
>>>> +
>>>> +A sound driver MUST conform to the following normative
>>>> statements:
>>>> +
>>>> +\begin{itemize}
>>>> +\item \ref{drivernormative:Device Types / Sound Device / Device
>>>> Initialization}
>>>> +\item \ref{drivernormative:Device Types / Sound Device / PCM
>>>> control
>>>> requests}
>>>> +\item \ref{drivernormative:Device Types / Sound Device / PCM IO
>>>> requests}
>>>> +\end{itemize}
>>>> +
>>>> \conformance{\section}{Device
>>>> Conformance}\label{sec:Conformance /
>>>> Device Conformance}
>>>>
>>>> A device MUST conform to the following normative statements:
>>>> @@ -338,6 +348,16 @@ \section{Conformance
>>>> Targets}\label{sec:Conformance / Conformance Targets}
>>>> \item \ref{devicenormative:Device Types / Socket Device /
>>>> Device
>>>> Operation / Receive and Transmit}
>>>> \end{itemize}
>>>>
>>>> +\conformance{\subsection}{Sound Device
>>>> Conformance}\label{sec:Conformance / Device Conformance / Sound
>>>> Device Conformance}
>>>> +
>>>> +A sound device MUST conform to the following normative
>>>> statements:
>>>> +
>>>> +\begin{itemize}
>>>> +\item \ref{devicenormative:Device Types / Sound Device / Device
>>>> Initialization}
>>>> +\item \ref{devicenormative:Device Types / Sound Device / PCM
>>>> control
>>>> requests}
>>>> +\item \ref{devicenormative:Device Types / Sound Device / PCM IO
>>>> requests}
>>>> +\end{itemize}
>>>> +
>>>> \conformance{\section}{Legacy Interface: Transitional Device
>>>> and
>>>> Transitional Driver Conformance}\label{sec:Conformance / Legacy
>>>> Interface: Transitional Device and Transitional Driver
>>>> Conformance}
>>>> A conformant implementation MUST be either transitional or
>>>> non-transitional, see \ref{intro:Legacy
>>>> diff --git a/content.tex b/content.tex
>>>> index 37a2190..4a00884 100644
>>>> --- a/content.tex
>>>> +++ b/content.tex
>>>> @@ -5682,6 +5682,7 @@ \subsubsection{Legacy Interface: Framing
>>>> Requirements}\label{sec:Device
>>>> \input{virtio-input.tex}
>>>> \input{virtio-crypto.tex}
>>>> \input{virtio-vsock.tex}
>>>> +\input{virtio-sound.tex}
>>>>
>>>> \chapter{Reserved Feature Bits}\label{sec:Reserved Feature
>>>> Bits}
>>>>
>>>> diff --git a/virtio-sound.tex b/virtio-sound.tex
>>>> new file mode 100644
>>>> index 0000000..68a606e
>>>> --- /dev/null
>>>> +++ b/virtio-sound.tex
>>>> @@ -0,0 +1,405 @@
>>>> +\section{Sound Device}\label{sec:Device Types / Sound Device}
>>>> +
>>>> +The virtio sound card is a virtual audio device supporting
>>>> output
>>>> and input PCM
>>>> +streams. All device control requests are placed into the control
>>>> virtqueue, all
>>>> +I/O requests are placed into the PCM virtqueue.
>>>
>>> Where are stream position updates placed ? These are usually
>>> synchronous from the driver and enable the player application to
>>> render
>>> or read data from the correct locations in the buffers. This is
>>> latency
>>> sensitive data.
>>
>> It happens in an interrupt handler. Since PCM frame transfer is
>> message based,
>> the only possible way to figure out consumed or captured amount of
>> frames is
>> to take a look into a message response.
>>
>
> Ok, this is what I thought. This wont work for low latency audio use
> cases since the buffers and position responses are queued (voice calls,
> some gaming, some system notifications like keypress or volume
> change).
But it's not different from typical HW. A driver receives an interrupt once
DMA transaction is finished (i.e. some part of data was copied to/from host).
And hardware pointer value usually is extrapolated based on some internal
device state. A virtio driver could do something similar: it could either
return "real" pointer value (i.e. what it internally uses) or could show fake
"extrapolated" value (like frame_rate * elappsed_time). Although, it's a
different story.
> However, it's probably tolerable for most other use cases. What use
> cases have you tried in your testing ?
All typical cases: multimedia (video + audio or audio only), network
streaming, video calls, system events, volume changing and so on.
By the way, could you define the "low latency" term that we are talking about?
Just to be on the same wave.
>>
>>>> +
>>>> +\subsection{Device ID}\label{sec:Device Types / Sound Device /
>>>> Device ID}
>>>> +
>>>> +25
>>>> +
>>>> +\subsection{Virtqueues}\label{sec:Device Types / Sound Device /
>>>> Virtqueues}
>>>> +
>>>> +\begin{description}
>>>> +\item[0] controlq
>>>> +\item[1] pcmq
>>>> +\end{description}
>>>> +
>>>> +The controlq virtqueue always exists, the pcmq virtqueue only
>>>> exists
>>>> if
>>>> +the VIRTIO_SND_F_PCM_OUTPUT and/or VIRTIO_SND_F_PCM_INPUT
>>>> feature is
>>>> negotiated.
>>>> +
>>>> +\subsection{Feature bits}\label{sec:Device Types / Sound Device
>>>> /
>>>> Feature bits}
>>>> +
>>>> +\begin{description}
>>>> +\item[VIRTIO_SND_F_PCM_OUTPUT (0)] Output PCM stream support.
>>>> +\item[VIRTIO_SND_F_PCM_INPUT (1)] Input PCM stream support.
>>>> +\end{description}
>>>> +
>>>> +\subsection{Device configuration layout}\label{sec:Device Types
>>>> /
>>>> Sound Device / Device configuration layout}
>>>> +
>>>> +\begin{lstlisting}
>>>> +/* supported PCM sample formats */
>>>> +enum {
>>>> + VIRTIO_SND_PCM_FMT_S8 = 0,
>>>> + VIRTIO_SND_PCM_FMT_U8,
>>>> + VIRTIO_SND_PCM_FMT_S16,
>>>> + VIRTIO_SND_PCM_FMT_U16,
>
> 24 bit.
These kinds of sample (18/20/24-bit) were discussed here earlier and caused
some concerns (regarding supporting in a guest, testing and so on), so they
were removed for simplicity.
From other side, if we are up to add these, maybe it's better not to hardcode
format names and use some kind of generic description instead? Something like
struct {
kind; /* S/U/F */
significant_bits;
storage_bits;
}
It allows to describe formats like 24 and 24_3 in unified way and to add
future formats without changing the protocol. What do you think?
>>>> + VIRTIO_SND_PCM_FMT_S32,
>>>> + VIRTIO_SND_PCM_FMT_U32,
>>>> + VIRTIO_SND_PCM_FMT_FLOAT,
>>>> + VIRTIO_SND_PCM_FMT_FLOAT64
>>>> +};
>>>> +
>>>> +/* supported PCM frame rates */
>>>> +enum {
>>>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>>>> + VIRTIO_SND_PCM_RATE_11025,
>>>> + VIRTIO_SND_PCM_RATE_16000,
>>>> + VIRTIO_SND_PCM_RATE_22050,
>>>> + VIRTIO_SND_PCM_RATE_32000,
>>>> + VIRTIO_SND_PCM_RATE_44100,
>>>> + VIRTIO_SND_PCM_RATE_48000,
>>>> + VIRTIO_SND_PCM_RATE_64000,
>>>> + VIRTIO_SND_PCM_RATE_88200,
>>>> + VIRTIO_SND_PCM_RATE_96000,
>>>> + VIRTIO_SND_PCM_RATE_176400,
>>>> + VIRTIO_SND_PCM_RATE_192000
>>>
>>> It may be best to use the same rates and format from the ALSA spec,
>>> this covers any holes (like rates that are not in the above list).
>>
>> These rates are from ALSA definitions. And what formats should be
>> added?
>>
>
> Rate, Knot - I would copy ALSA supported rates and formats just like
> with the channel maps.
See a comment above.
>>
>>>> +};
>>>> +
>>>> +/* a PCM stream configuration */
>>>> +struct virtio_pcm_stream_config {
>>>> + u8 channels_min;
>>>> + u8 channels_max;
>>>> + le16 formats; /* 1 << VIRTIO_SND_PCM_FMT_XXX */
>>>> + le16 rates; /* 1 << VIRTIO_SND_PCM_RATE_XXX */
>>>> + u16 padding;
>>>> +};
>>>
>>> Should this be packed ? Btw, padding alignment is to a non 32bit
>>> size ?
>>
>> It should be naturally packed if padding is correct. And do you mean
>> 8 and
>> 16-bit fields? You can read one or two bytes from device
>> configuration space,
>> it should not be a problem.
>>
>>
>>> What about buffer/period formats e.g interleaved ?
>>
>> Yeah, support only for interleaved layout previously was stated in
>> the
>> specification but occasionally was lost. Will be fixed!
>>
>>
>>>
>>>> +
>>>> +/* a device configuration space */
>>>> +struct virtio_snd_config {
>>>> + struct virtio_pcm_config {
>>>> + struct virtio_pcm_stream_config output;
>>>> + struct virtio_pcm_stream_config input;
>>>> + } pcm;
>>>> +};
>>>> +\end{lstlisting}
>>>
>>> I assuming this is sent to configure PCM capabilities ?
>>
>> Yes, minimum amount of information required to setup a PCM stream in
>> any kind
>> of guest.
>>
>>
>>> What about buffer and period size capabilities ?
>>
>> In this specification we assume that buffer size is up to guest
>> decision.
>> Also, period size is ALSA-only conception and should not be mentioned
>> here at
>> all (we are not stick only to ALSA).
>>
>>
>
> Period size == fragment size == block size (probably a few other naming
> conventions too), we should be able to tell the device VM our
> period/block/fragment size so we can configure the HW appropriately.
> This can also be used to make our stream copying more efficient too.
Then let's talk about playback. As I can understand, you are describing a
case, when
1. ALSA application sets some period size
2. fills the whole buffer with non-silence frames
3. upon every period notification, provides next period-sized part of data
It's one of use cases, but quite rare. More often (especially if we are
talking about latency control and low latency audio) things will be
1. an application fills the whole buffer with silence
2. suppresses period notifications
3. starts stream and almost immediatly seeks to near hw position
4. puts non-silence frames there (size is set according to desired latency)
5. provides new frames at some interval (usually in 5-10ms)
It's how software mixers usually work: PulseAudio, Windows in-kernel mixer,
new Android system mixer and so on. Also, you can find such approach in low
latency player applications (when they work directly with device).
The point is they do not rely on notifications from the driver and use their
own timers. The only thing they querying is current hw position.
The point is we could have period notifications from the device. But it seems
they will be useful only in minority of cases. Also, for example, Windows
audio driver has no such concept at all.
>>>> +
>>>> +\subsubsection{Device configuration fields}
>>>> +
>>>> +The \field{pcm.output} and \field{pcm.input} fields contain PCM
>>>> stream
>>>> +configuration:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{channels_min}] (driver-read-only) is a minimum
>>>> number
>>>> of supported
>>>> +channels.
>>>> +\item[\field{channels_max}] (driver-read-only) is a maximum
>>>> number
>>>> of supported
>>>> +channels.
>>>> +\item[\field{formats}] (driver-read-only) is supported sample
>>>> format
>>>> bit map.
>>>> +\item[\field{rates}] (driver-read-only) is supported frame rate
>>>> bit
>>>> map.
>>>> +\end{description}
>>>> +
>>>> +\subsection{Device Initialization}
>>>> +
>>>> +\begin{enumerate}
>>>> +\item If the VIRTIO_SND_F_PCM_OUTPUT feature is negotiated,
>>>> \field{pcm.output}
>>>> +contains valid output PCM stream configuration.
>>>> +\item If the VIRTIO_SND_F_PCM_INPUT feature is negotiated,
>>>> \field{pcm.input}
>>>> +contains valid input PCM stream configuration.
>>>> +\end{enumerate}
>>>> +
>>>> +\devicenormative{\subsubsection}{Device Initialization}{Device
>>>> Types
>>>> / Sound Device / Device Initialization}
>>>> +
>>>> +\begin{enumerate}
>>>> +\item The device MUST NOT set the not defined format and rate
>>>> bits.
>>>> +\item The device MUST initialize padding bytes
>>>> \field{pcm.output.padding} and
>>>> +\field{pcm.input.padding} to 0.
>>>> +\end{enumerate}
>>>> +
>>>> +\drivernormative{\subsubsection}{Device Initialization}{Device
>>>> Types
>>>> / Sound Device / Device Initialization}
>>>> +
>>>> +\begin{enumerate}
>>>> +\item The driver MUST configure and initialize all virtqueues.
>>>> +\item The driver SHOULD ignore the not defined format and rate
>>>> bits.
>>>> +\end{enumerate}
>>>> +
>>>> +\subsection{Device Operation}\label{sec:Device Types / Sound
>>>> Device
>>>> / Device Operation}
>>>> +
>>>> +All control messages are placed into the controlq virtqueue and
>>>> use
>>>> the following
>>>> +layout structure and definitions:
>>>> +
>>>> +\begin{lstlisting}
>>>> +enum {
>>>> + /* PCM control request types */
>>>> + VIRTIO_SND_R_PCM_CHMAP_INFO = 0,
>>>> + VIRTIO_SND_R_PCM_SET_FORMAT,
>>>> + VIRTIO_SND_R_PCM_PREPARE,
>>>> + VIRTIO_SND_R_PCM_START,
>>>> + VIRTIO_SND_R_PCM_STOP,
>>>> + VIRTIO_SND_R_PCM_PAUSE,
>>>> + VIRTIO_SND_R_PCM_UNPAUSE,
>>>> +
>>>> + /* generic status codes */
>>>> + VIRTIO_SND_S_OK = 0x8000,
>>>> + VIRTIO_SND_S_BAD_MSG,
>>>> + VIRTIO_SND_S_NOT_SUPP,
>>>> + VIRTIO_SND_S_IO_ERR
>>>> +};
>>>> +
>>>> +struct virtio_snd_ctl_msg {
>>>> + /* device-read-only data */
>>>> + le32 request_code;
>>>> + u8 request_payload[];
>>>
>>> How do I know how much data to read here ? Is size embedded in
>>> request_code ?
>>
>> Yes, you are right. Actual request/response length can be easily
>> derived from
>> a request_code.
>>
>
> Nak, please use a separate size and type code otherwise we will have
> real problems in the future when we come to add new features (with new
> codes or bigger sizes).
But virtqueue descriptors already contain size information. Usually, there's
API for getting readable/writable size in virtqueue sg-chains. And it's
possible to get request size using something like
vq_readable_size(...) - sizeof(request_code)
and the same for response size
vq_writable_size(...) - sizeof(response_status)
Originally, the proposed spec contained separate size fields. But these seem
redundant, since virtqueue does all the work.
>>
>>>
>>>> + /* device-writable data */
>>>> + le32 response_status;
>>>> + u8 response_payload[];
>>>> +};
>>>> +\end{lstlisting}
>>>> +
>>>> +A generic control message consists of request and response parts
>>>> and
>>>> contains
>>>> +the following fields:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{request_code}] (device-read-only) specifies a
>>>> device
>>>> request code
>>>> +(VIRTIO_SND_R_*).
>>>> +\item[\field{request_payload}] (device-read-only) contains
>>>> request-
>>>> specific
>>>> +data.
>>>> +\item[\field{response_status}] (device-writable) specifies a
>>>> device
>>>> response
>>>> +status (VIRTIO_SND_S_*).
>>>> +\item[\field{response_payload}] (device-writable) contains
>>>> response-
>>>> specific
>>>> +data.
>>>> +\end{description}
>>>> +
>>>> +Unless stated otherwise, the \field{request_payload} and
>>>> \field{response_payload}
>>>> +fields are empty.
>>>> +
>>>> +The \field{response_status} field contains one of the following
>>>> values:
>>>> +
>>>> +\begin{itemize*}
>>>> +\item VIRTIO_SND_S_OK: success.
>>>> +\item VIRTIO_SND_S_BAD_MSG: a control message is malformed or
>>>> contains invalid
>>>> +parameters.
>>>> +\item VIRTIO_SND_S_NOT_SUPP: requested operation or parameters
>>>> are
>>>> not supported.
>>>> +\item VIRTIO_SND_S_IO_ERR: an I/O error occurred.
>>>> +\end{itemize*}
>>>> +
>>>> +\subsubsection{Device Operation: PCM control requests}
>>>> +
>>>> +A PCM stream has the following command lifecycle:
>>>> +
>>>> +\begin{enumerate}
>>>> +\item Set format
>>>> +\item Prepare
>>>> +\item Output only: transfer data for prebuffing
>>>> +\item Start
>>>> +\item Transfer data to/from the PCM device
>>>> +\begin{enumerate}
>>>> + \item Pause
>>>> + \item Unpause
>>>> +\end{enumerate}
>>>> +\item Stop
>>>> +\end{enumerate}
>>>> +
>>>> +PCM control requests have or consist of a fixed header with the
>>>> following
>>>> +layout structure:
>>>> +
>>>> +\begin{lstlisting}
>>>> +/* supported PCM stream types */
>>>> +enum {
>>>> + VIRTIO_SND_PCM_T_OUTPUT = 0,
>>>> + VIRTIO_SND_PCM_T_INPUT
>>>
>>> Can we rename these PLAYBACK and CAPTURE
>>
>> Playback and capture are used in ALSA drivers, sink and source are
>> used in
>> Windows drivers. We decided to stick to something obvious in-between
>> (and it's
>> just shorter). Although, it's still discussable topic.
>>
>>
>>>> +};
>>>> +
>>>> +/* PCM control request header */
>>>> +struct virtio_snd_pcm_hdr {
>>>> + le32 code;
>>>> + le32 stream;
>>>> +};
>>>> +\end{lstlisting}
>>>> +
>>>> +PCM control request fields:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{code}] (device-read-only) specifies a PCM device
>>>> request code
>>>> +(VIRTIO_SND_R_PCM_*).
>>>> +\item[\field{stream}] (device-read-only) specifies a PCM stream
>>>> type
>>>> +(VIRTIO_SND_PCM_T_*).
>>>> +\end{description}
>>>> +
>>>> +\begin{description}
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_CHMAP_INFO]
>>>> +Query a PCM channel map information for specified stream type.
>>>> +
>>>> +A response uses the following layout structure and definitions:
>>>> +
>>>> +\begin{lstlisting}
>>>> +/* standard channel position definition */
>>>> +enum {
>>>> + VIRTIO_SND_PCM_CH_NONE = 0, /* undefined */
>>>> + VIRTIO_SND_PCM_CH_NA, /* silent */
>>>> + VIRTIO_SND_PCM_CH_MONO, /* mono stream */
>>>> + VIRTIO_SND_PCM_CH_FL, /* front left */
>>>> + VIRTIO_SND_PCM_CH_FR, /* front right */
>>>> + VIRTIO_SND_PCM_CH_RL, /* rear left */
>>>> + VIRTIO_SND_PCM_CH_RR, /* rear right */
>>>> + VIRTIO_SND_PCM_CH_FC, /* front center */
>>>> + VIRTIO_SND_PCM_CH_LFE, /* low frequency (LFE) */
>>>> + VIRTIO_SND_PCM_CH_SL, /* side left */
>>>> + VIRTIO_SND_PCM_CH_SR, /* side right */
>>>> + VIRTIO_SND_PCM_CH_RC, /* rear center */
>>>> + VIRTIO_SND_PCM_CH_FLC, /* front left center */
>>>> + VIRTIO_SND_PCM_CH_FRC, /* front right center */
>>>> + VIRTIO_SND_PCM_CH_RLC, /* rear left center */
>>>> + VIRTIO_SND_PCM_CH_RRC, /* rear right center */
>>>> + VIRTIO_SND_PCM_CH_FLW, /* front left wide */
>>>> + VIRTIO_SND_PCM_CH_FRW, /* front right wide */
>>>> + VIRTIO_SND_PCM_CH_FLH, /* front left high */
>>>> + VIRTIO_SND_PCM_CH_FCH, /* front center high */
>>>> + VIRTIO_SND_PCM_CH_FRH, /* front right high */
>>>> + VIRTIO_SND_PCM_CH_TC, /* top center */
>>>> + VIRTIO_SND_PCM_CH_TFL, /* top front left */
>>>> + VIRTIO_SND_PCM_CH_TFR, /* top front right */
>>>> + VIRTIO_SND_PCM_CH_TFC, /* top front center */
>>>> + VIRTIO_SND_PCM_CH_TRL, /* top rear left */
>>>> + VIRTIO_SND_PCM_CH_TRR, /* top rear right */
>>>> + VIRTIO_SND_PCM_CH_TRC, /* top rear center */
>>>> + VIRTIO_SND_PCM_CH_TFLC, /* top front left center */
>>>> + VIRTIO_SND_PCM_CH_TFRC, /* top front right center */
>>>> + VIRTIO_SND_PCM_CH_TSL, /* top side left */
>>>> + VIRTIO_SND_PCM_CH_TSR, /* top side right */
>>>> + VIRTIO_SND_PCM_CH_LLFE, /* left LFE */
>>>> + VIRTIO_SND_PCM_CH_RLFE, /* right LFE */
>>>> + VIRTIO_SND_PCM_CH_BC, /* bottom center */
>>>> + VIRTIO_SND_PCM_CH_BLC, /* bottom left center */
>>>> + VIRTIO_SND_PCM_CH_BRC /* bottom right center */
>>>> +};
>>>> +
>>>> +/* a maximum possible number of channels */
>>>> +#define VIRTIO_SND_PCM_CH_MAX 256
>>>> +
>>>> +/* response containing a PCM channel map information */
>>>> +struct virtio_snd_pcm_chmap_info {
>>>> + le32 status;
>>>> + le32 npositions;
>>>> + u8 positions[VIRTIO_SND_PCM_CH_MAX];
>>>> +};
>>>> +\end{lstlisting}
>>>> +
>>>> +PCM channel map information fields:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{status}] (device-writable) specifies a device
>>>> response
>>>> status
>>>> +(VIRTIO_SND_S_*).
>>>> +\item[\field{npositions}] (device-writable) is a number of valid
>>>> entries in
>>>> +the \field{positions} array.
>>>> +\item[\field{positions}] (device-writable) contains PCM channel
>>>> positions
>>>> +(VIRTIO_SND_PCM_CH_*).
>>>> +\end{description}
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_SET_FORMAT]
>>>> +Set selected PCM format.
>>>> +
>>>> +\begin{lstlisting}
>>>> +struct virtio_snd_pcm_set_format {
>>>> + struct virtio_snd_pcm_hdr hdr;
>>>> + le16 channels;
>>>> + le16 format;
>>>> + le16 rate;
>>>> + u16 padding;
>>>> +};
>>>> +\end{lstlisting}
>>>> +
>>>> +PCM control request fields:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{hdr}] (device-read-only) is a PCM control request
>>>> header.
>>>> +\item[\field{channels}] (device-read-only) specifies a desired
>>>> number of channels.
>>>> +\item[\field{format}] (device-read-only) specifies a desired PCM
>>>> sample format
>>>> +(VIRTIO_SND_PCM_FMT_*).
>>>> +\item[\field{rate}] (device-read-only) specifies a desired PCM
>>>> frame
>>>> rate
>>>> +(VIRTIO_SND_PCM_RATE_*).
>>>> +\end{description}
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_PREPARE]
>>>> +Prepare the PCM device.
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_START]
>>>> +Start the PCM device.
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_STOP]
>>>> +Stop the PCM device.
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_PAUSE]
>>>> +Set the PCM device on pause.
>>>> +
>>>> +\item[VIRTIO_SND_R_PCM_UNPAUSE]
>>>> +Unset the PCM device from pause.
>>>> +
>>>> +\end{description}
>>>> +
>>>> +\devicenormative{\subsubsection}{PCM control requests}{Device
>>>> Types
>>>> / Sound Device / PCM control requests}
>>>> +
>>>> +In a VIRTIO_SND_R_PCM_CHMAP_INFO request:
>>>> +
>>>> +\begin{itemize*}
>>>> +\item the device MUST return the VIRTIO_SND_S_NOT_SUPP status
>>>> code,
>>>> if it does not
>>>> +support a channel map for a specified stream type;
>>>> +\item the device MUST set the \field{npositions} field to 0, if
>>>> the
>>>> operation
>>>> +failed.
>>>> +\end{itemize*}
>>>> +
>>>> +\drivernormative{\subsubsection}{PCM control requests}{Device
>>>> Types
>>>> / Sound Device / PCM control requests}
>>>> +
>>>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a stream
>>>> type
>>>> if
>>>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>>>> +
>>>> +The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a stream
>>>> type
>>>> if
>>>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>>>> +
>>>> +In a VIRTIO_SND_R_PCM_SET_FORMAT request:
>>>> +
>>>> +\begin{itemize*}
>>>> +\item the driver MUST NOT specify the \field{channels} value
>>>> less
>>>> than the \field{channels_min}
>>>> +or greater than the \field{channels_max} values reported in
>>>> stream
>>>> configuration;
>>>> +\item the driver MUST specify the \field{format} and
>>>> \field{rate}
>>>> values according
>>>> +to the \field{formats} and \field{rates} values reported in
>>>> stream
>>>> configuration;
>>>> +\item the driver MUST NOT specify the not defined format and
>>>> rate
>>>> values;
>>>> +\item the driver MUST initialize \field{padding} bytes to 0.
>>>> +\end{itemize*}
>>>> +
>>>> +\subsubsection{Device Operation: PCM I/O requests}
>>>> +
>>>> +All I/O requests are placed into the pcmq virtqueue. Each
>>>> request is
>>>> of form:
>>>> +
>>>> +\begin{lstlisting}
>>>> +struct virtio_snd_pcm_xfer {
>>>> + le32 stream;
>>>> + u8 data[];
>>>> + le32 status;
>>>> + le32 actual_length;
>>>
>>> Not following this, is actual_length the size of data[]. If so, it
>>> must
>>> preceed data[].
>>
>> No, the actual_length field is supposed to be used by device side to
>> report
>> actual amount of bytes read from/written to a buffer. In real world
>> scenario,
>> if an I/O request contains N bytes, a device can play/capture *up to*
>> N bytes.
>> Thus, it's required to report this length back to a driver.
>>
>
> This is not how PCM audio buffering works. I don't copy extra padding
> if I dont need to, I only copy when my buffer is full. See the need for
> period size in earlier comments.
Originally, it was intended to handle a case when there's less space in device
playback buffer than was expected, or when there's less available data in
device capturing buffer than was expected.
>>
>>>
>>>> +};
>>>> +\end{lstlisting}
>>>> +
>>>> +I/O request fields:
>>>> +
>>>> +\begin{description}
>>>> +\item[\field{stream}] (device-read-only) specifies a PCM stream
>>>> type
>>>> +(VIRTIO_SND_PCM_T_*).
>>>> +\item[\field{data}]
>>>> +\begin{enumerate}
>>>> +\item Output: (device-read-only) a buffer with PCM frames to be
>>>> written to the
>>>> +device.
>>>> +\item Input: (device-writable) a buffer to be filled with PCM
>>>> frames
>>>> from the
>>>> +device.
>>>> +\end{enumerate}
>>>> +\item[\field{status}] (device-writable) contains VIRTIO_SND_S_OK
>>>> if
>>>> an operation
>>>> +is successful, and VIRTIO_SND_S_IO_ERR otherwise.
>>>> +\item[\field{actual_length}] (device-writable) specifies an
>>>> actual
>>>> amount of
>>>> +bytes read from/written to the \field{data} field.
>>>> +\end{description}
>>>> +
>>>> +\devicenormative{\subsubsection}{PCM I/O requests}{Device Types
>>>> /
>>>> Sound Device / PCM IO requests}
>>>> +
>>>> +\begin{enumerate}
>>>> +\item The device MUST set the \field{actual_length} field to 0,
>>>> if
>>>> the operation
>>>> +failed.
>>>> +\end{enumerate}
>>>> +
>>>> +\drivernormative{\subsubsection}{PCM I/O requests}{Device Types
>>>> /
>>>> Sound Device / PCM IO requests}
>>>> +
>>>> +\begin{enumerate}
>>>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_OUTPUT as a
>>>> stream type if
>>>> +the VIRTIO_SND_F_PCM_OUTPUT feature is not negotiated.
>>>> +\item The driver MUST NOT specify VIRTIO_SND_PCM_T_INPUT as a
>>>> stream
>>>> type if
>>>> +the VIRTIO_SND_F_PCM_INPUT feature is not negotiated.
>>>> +\end{enumerate}
>>>> --
>>>> 2.23.0
>>>>
>>>
>>> I don't see anything in here about dealing with
>>>
>>> 1) Underruns and overruns. i.e. how do we recover and resync audio
>>> between guests.
>>
>> Since xrun conditions are higher level conceptions, we decided to
>> delegate
>> such issues to guest application itself. It helps to make overall
>> design
>> simpler. And it seems there's not so much we can do if xrun condition
>> is
>> happened on the device side.
>
> We can inform the guest. The guest userspace can then take any remedial
> action if needed.
Then, we need the third queue for notifications.
>>
>>
>>> 2) Zero copy of audio data and stream positions. Essential for any
>>> low
>>> latency audio use cases (if supported by HW, hypervisor, VM
>>> framework).
>>
>> What do you mean by zero-copy? Shared memory between device and
>> driver sides?
>
> Yes, but I see this is now a separate thread as it used by others too.
Shared memory based approach was proposed in the very first version of the
spec (and we are rooting for this). Then there was discussion and it was
postponed for some future virtio device. One of the reason - it's not suitable
for arch with non-coherent memory between host and guest.
>
> Thanks
>
> Liam
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 11:09 ` Jean-Philippe Brucker
@ 2019-11-12 14:20 ` Liam Girdwood
2019-11-12 16:05 ` Jean-Philippe Brucker
2019-11-13 7:54 ` Anton Yakovlev
0 siblings, 2 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-11-12 14:20 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Tue, 2019-11-12 at 12:09 +0100, Jean-Philippe Brucker wrote:
> Hi,
>
> On Mon, Nov 11, 2019 at 03:20:55PM +0000, Liam Girdwood wrote:
> > > > > +struct virtio_snd_ctl_msg {
> > > > > + /* device-read-only data */
> > > > > + le32 request_code;
> > > > > + u8 request_payload[];
> > > >
> > > > How do I know how much data to read here ? Is size embedded in
> > > > request_code ?
> > >
> > > Yes, you are right. Actual request/response length can be easily
> > > derived from
> > > a request_code.
> > >
> >
> > Nak, please use a separate size and type code otherwise we will
> > have
> > real problems in the future when we come to add new features (with
> > new
> > codes or bigger sizes).
>
> Other virtio devices rely on the buffer length already provided by
> virtio
> for this. For example see the virtio-net control virtqueue
> (virtio_net_ctrl*), which is extended with new commands from time to
> time.
> Most recently, the "receive-side scaling" feature introduces a large
> structure as payload to virtio_net_ctrl.
>
Sorry, not following, what does this have to do with combining type and
size into request_code ? Or are you saying the size is encoded
elsewhere in the virtio transport data ?
> [...]
> > > > > +\begin{lstlisting}
> > > > > +struct virtio_snd_pcm_xfer {
> > > > > + le32 stream;
> > > > > + u8 data[];
> > > > > + le32 status;
> > > > > + le32 actual_length;
> > > >
> > > > Not following this, is actual_length the size of data[]. If so,
> > > > it
> > > > must
> > > > preceed data[].
> > >
> > > No, the actual_length field is supposed to be used by device side
> > > to
> > > report
> > > actual amount of bytes read from/written to a buffer.
>
> By the way the written size is already provided by virtio (field len
> in
> the used descriptor), but not the read size.
>
> > > In real world
> > > scenario,
> > > if an I/O request contains N bytes, a device can play/capture *up
> > > to*
> > > N bytes.
> > > Thus, it's required to report this length back to a driver.
> > >
> >
> > This is not how PCM audio buffering works. I don't copy extra
> > padding
> > if I dont need to, I only copy when my buffer is full. See the need
> > for
> > period size in earlier comments.
>
> [...]
> > > > 2) Zero copy of audio data and stream positions. Essential for
> > > > any
> > > > low
> > > > latency audio use cases (if supported by HW, hypervisor, VM
> > > > framework).
> > >
> > > What do you mean by zero-copy? Shared memory between device and
> > > driver sides?
> >
> > Yes, but I see this is now a separate thread as it used by others
> > too.
>
> The virtio transport already permits zero-copy, because one buffer
> can be
> described with a chain of descriptors, each pointing to a different
> area
> in memory. The driver can split the virtio_snd_pcm_xfer into header
> and
> data:
>
> virtqueue
> desc ring
> | |
> +-------+
> | |---------> header (copied)
> +- - - -+
> | |---------> PCM data (mapped)
> +-------+
> | |
>
> So it might be a good idea to put all the metadata (stream, status,
> actual_length) at the beginning of the virtio_snd_pcm_xfer struct,
> otherwise the driver will need three descriptors instead of two to
> achieve
This would be a good improvement, it's less copying and would likely
improve user experience, however the buffer ptr still suffers latency
as it's queued (same for stream positions going the other way).
Low latency zero (software) copy audio needs to work like this:
1) The guest buffers SG PHY pages are shared with audio device driver
and/or DSP FW at stream init.
2) Device driver or DSP FW programs DMAC IP with SG PHY pages of guest
buffer.
3) DMA is started and audio data is copied directly to audio endpoint
(I2S, HDA, SDW etc) or directly to DSP.
At this point, the audio can be rendered by guests very close to the
DMA R/W positions (low latency) if they knew the HW pointer position
accurately. The guest needs to also be able to read the stream HW
position from the device driver or DSP using a low latency method too:
4) Common timestamp between audio HW/DSP and all guests can be used and
sent over virtio. Allows guests to interpolate any drift and estimate
audio position with good accuracy using common clock.
OR
5) Common stream status between audio HW/DSP and all guests (when
supported by HW). Device driver can export RO page of stream data to
each guest that DSP FW can update with realtime true rendering
positions (Intel HW supports this feature).
Thanks
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 12:45 ` Anton Yakovlev
@ 2019-11-12 15:16 ` Liam Girdwood
2019-11-13 9:05 ` Anton Yakovlev
0 siblings, 1 reply; 42+ messages in thread
From: Liam Girdwood @ 2019-11-12 15:16 UTC (permalink / raw)
To: Anton Yakovlev, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
On Tue, 2019-11-12 at 13:45 +0100, Anton Yakovlev wrote:
> On 11.11.2019 16:20, Liam Girdwood wrote:
> > On Tue, 2019-10-29 at 10:42 +0100, Anton Yakovlev wrote:
> > > Hi Liam,
> > >
> > > Thank you for comments! Please, take a look at our answers below.
> >
> > Apologies, been travelling. More comments below.
> >
> > >
> > >
> > > On 28.10.2019 17:05, Liam Girdwood wrote:
> > > > On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
> > > > > From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
> > > > >
> > > > >
snip
> > > > > diff --git a/virtio-sound.tex b/virtio-sound.tex
> > > > > new file mode 100644
> > > > > index 0000000..68a606e
> > > > > --- /dev/null
> > > > > +++ b/virtio-sound.tex
> > > > > @@ -0,0 +1,405 @@
> > > > > +\section{Sound Device}\label{sec:Device Types / Sound
> > > > > Device}
> > > > > +
> > > > > +The virtio sound card is a virtual audio device supporting
> > > > > output
> > > > > and input PCM
> > > > > +streams. All device control requests are placed into the
> > > > > control
> > > > > virtqueue, all
> > > > > +I/O requests are placed into the PCM virtqueue.
> > > >
> > > > Where are stream position updates placed ? These are usually
> > > > synchronous from the driver and enable the player application
> > > > to
> > > > render
> > > > or read data from the correct locations in the buffers. This is
> > > > latency
> > > > sensitive data.
> > >
> > > It happens in an interrupt handler. Since PCM frame transfer is
> > > message based,
> > > the only possible way to figure out consumed or captured amount
> > > of
> > > frames is
> > > to take a look into a message response.
> > >
> >
> > Ok, this is what I thought. This wont work for low latency audio
> > use
> > cases since the buffers and position responses are queued (voice
> > calls,
> > some gaming, some system notifications like keypress or volume
> > change).
>
> But it's not different from typical HW. A driver receives an
> interrupt once
> DMA transaction is finished (i.e. some part of data was copied
> to/from host).
> And hardware pointer value usually is extrapolated based on some
> internal
> device state.
Modern HW can usually provide the HW position to the nearest word. Some
older/bad HW provides to the nearest period though and this is when we
have to estimate.
Native (non virtual) audio device drivers typically have no latency
reading the stream position and it's read at the same time as the
system clock. Reading the same clocks together is important as it means
the relationship between the two is deterministic and not subject to
any jitter (which can be introduced by reading stream position over
virtio).
> A virtio driver could do something similar: it could either
> return "real" pointer value (i.e. what it internally uses) or could
> show fake
> "extrapolated" value (like frame_rate * elappsed_time). Although,
> it's a
> different story.
>
>
Virtio driver should return real stream position alongside real common
system time (of when stream position was read).
> > However, it's probably tolerable for most other use cases. What use
> > cases have you tried in your testing ?
>
> All typical cases: multimedia (video + audio or audio only), network
> streaming, video calls, system events, volume changing and so on.
>
> By the way, could you define the "low latency" term that we are
> talking about?
> Just to be on the same wave.
>
Low latency audio is when any application or processing can render data
very close to the HW stream R/W position (typically low or sub
millisecond). This needs zero copy buffering to HW on host/VM/user side
and also timely and accurate stream positions.
> es
> > > > > /
> > > > > Sound Device / Device configuration layout}
> > > > > +
> > > > > +\begin{lstlisting}
> > > > > +/* supported PCM sample formats */
> > > > > +enum {
> > > > > + VIRTIO_SND_PCM_FMT_S8 = 0,
> > > > > + VIRTIO_SND_PCM_FMT_U8,
> > > > > + VIRTIO_SND_PCM_FMT_S16,
> > > > > + VIRTIO_SND_PCM_FMT_U16,
> >
> > 24 bit.
>
> These kinds of sample (18/20/24-bit) were discussed here earlier and
> caused
> some concerns (regarding supporting in a guest, testing and so on),
> so they
> were removed for simplicity.
>
> From other side, if we are up to add these, maybe it's better not to
> hardcode
> format names and use some kind of generic description instead?
> Something like
>
> struct {
> kind; /* S/U/F */
> significant_bits;
> storage_bits;
> }
>
> It allows to describe formats like 24 and 24_3 in unified way and to
> add
> future formats without changing the protocol. What do you think?
>
I would recommend copying the exact formats used by ALSA, these formats
are industry standard (although naming will be different on other
OSes).
>
> > > > > + VIRTIO_SND_PCM_FMT_S32,
> > > > > + VIRTIO_SND_PCM_FMT_U32,
> > > > > + VIRTIO_SND_PCM_FMT_FLOAT,
> > > > > + VIRTIO_SND_PCM_FMT_FLOAT64
> > > > > +};
> > > > > +
> > > > > +/* supported PCM frame rates */
> > > > > +enum {
> > > > > + VIRTIO_SND_PCM_RATE_8000 = 0,
> > > > > + VIRTIO_SND_PCM_RATE_11025,
> > > > > + VIRTIO_SND_PCM_RATE_16000,
> > > > > + VIRTIO_SND_PCM_RATE_22050,
> > > > > + VIRTIO_SND_PCM_RATE_32000,
> > > > > + VIRTIO_SND_PCM_RATE_44100,
> > > > > + VIRTIO_SND_PCM_RATE_48000,
> > > > > + VIRTIO_SND_PCM_RATE_64000,
> > > > > + VIRTIO_SND_PCM_RATE_88200,
> > > > > + VIRTIO_SND_PCM_RATE_96000,
> > > > > + VIRTIO_SND_PCM_RATE_176400,
> > > > > + VIRTIO_SND_PCM_RATE_192000
> > > >
> > > > It may be best to use the same rates and format from the ALSA
> > > > spec,
> > > > this covers any holes (like rates that are not in the above
> > > > list).
> > >
> > > These rates are from ALSA definitions. And what formats should be
> > > added?
> > >
> >
> > Rate, Knot - I would copy ALSA supported rates and formats just
> > like
> > with the channel maps.
>
> See a comment above.
>
See comment above :)
>
> > > >
> > > > > +
> > > > > +/* a device configuration space */
> > > > > +struct virtio_snd_config {
> > > > > + struct virtio_pcm_config {
> > > > > + struct virtio_pcm_stream_config output;
> > > > > + struct virtio_pcm_stream_config input;
> > > > > + } pcm;
> > > > > +};
> > > > > +\end{lstlisting}
> > > >
> > > > I assuming this is sent to configure PCM capabilities ?
> > >
> > > Yes, minimum amount of information required to setup a PCM stream
> > > in
> > > any kind
> > > of guest.
> > >
> > >
> > > > What about buffer and period size capabilities ?
> > >
> > > In this specification we assume that buffer size is up to guest
> > > decision.
> > > Also, period size is ALSA-only conception and should not be
> > > mentioned
> > > here at
> > > all (we are not stick only to ALSA).
> > >
> > >
> >
> > Period size == fragment size == block size (probably a few other
> > naming
> > conventions too), we should be able to tell the device VM our
> > period/block/fragment size so we can configure the HW
> > appropriately.
> > This can also be used to make our stream copying more efficient
> > too.
>
> Then let's talk about playback. As I can understand, you are
> describing a
> case, when
> 1. ALSA application sets some period size
> 2. fills the whole buffer with non-silence frames
> 3. upon every period notification, provides next period-sized part of
> data
>
> It's one of use cases, but quite rare.
Ehm, it's bog standard file R/W (non mmap()) audio and extremely common
on Linux and derivatives.
Step 4 is
4) Unblock and wake userspace R/W for next period/block/fragment.
> More often (especially if we are
> talking about latency control and low latency audio) things will be
> 1. an application fills the whole buffer with silence
> 2. suppresses period notifications
> 3. starts stream and almost immediatly seeks to near hw position
> 4. puts non-silence frames there (size is set according to desired
> latency)
> 5. provides new frames at some interval (usually in 5-10ms)
So this describes mmap() audio where we block by poll()ing, read HW
ptr, and the write new frames.
This proposal needs to support both operating modes and there is
nothing I can see with virtio and some minor changes to this proposal
that would prevent that ?
>
> It's how software mixers usually work: PulseAudio, Windows in-kernel
> mixer,
> new Android system mixer and so on. Also, you can find such approach
> in low
> latency player applications (when they work directly with device).
>
> The point is they do not rely on notifications from the driver and
> use their
> own timers. The only thing they querying is current hw position.
>
> The point is we could have period notifications from the device. But
> it seems
> they will be useful only in minority of cases. Also, for example,
> Windows
> audio driver has no such concept at all.
Windows does not need to use periodic mode, but a lot of Linux
userspace will use it and this will currently will break those
applications.
> > >
> > > Since xrun conditions are higher level conceptions, we decided to
> > > delegate
> > > such issues to guest application itself. It helps to make overall
> > > design
> > > simpler. And it seems there's not so much we can do if xrun
> > > condition
> > > is
> > > happened on the device side.
> >
> > We can inform the guest. The guest userspace can then take any
> > remedial
> > action if needed.
>
> Then, we need the third queue for notifications.
Why, this should go in the queue that's used for stream position ?
>
>
> > >
> > >
> > > > 2) Zero copy of audio data and stream positions. Essential for
> > > > any
> > > > low
> > > > latency audio use cases (if supported by HW, hypervisor, VM
> > > > framework).
> > >
> > > What do you mean by zero-copy? Shared memory between device and
> > > driver sides?
> >
> > Yes, but I see this is now a separate thread as it used by others
> > too.
>
> Shared memory based approach was proposed in the very first version
> of the
> spec (and we are rooting for this). Then there was discussion and it
> was
> postponed for some future virtio device. One of the reason - it's not
> suitable
> for arch with non-coherent memory between host and guest.
I'm rooting for this too. I do think we need to be a bit more flexible
so we can deal with non coherent architectures via a SW virtio copy and
support coherent architectures via zero copy.
We should be good as long as we can leave some configuration
space/types in the stream config to allow mapping of operating modes
(zero copy, periodic, mmap etc). The device driver should also send
it's capabilities during init so that guests will know what use cases
and modes are supported.
Btw, please don't take my comments as trying to block your proposal. I
am behind this work, I'll likely be one of the first users. My
intention is to help provide something that is both flexible enough to
cater for all current OS users/use cases on HDA like hardware and has
scope for future enhancements.
Thanks
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 14:20 ` Liam Girdwood
@ 2019-11-12 16:05 ` Jean-Philippe Brucker
2019-11-12 18:02 ` Liam Girdwood
2019-11-13 7:54 ` Anton Yakovlev
1 sibling, 1 reply; 42+ messages in thread
From: Jean-Philippe Brucker @ 2019-11-12 16:05 UTC (permalink / raw)
To: Liam Girdwood
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Tue, Nov 12, 2019 at 02:20:44PM +0000, Liam Girdwood wrote:
> > Other virtio devices rely on the buffer length already provided by
> > virtio
> > for this. For example see the virtio-net control virtqueue
> > (virtio_net_ctrl*), which is extended with new commands from time to
> > time.
> > Most recently, the "receive-side scaling" feature introduces a large
> > structure as payload to virtio_net_ctrl.
> >
>
> Sorry, not following, what does this have to do with combining type and
> size into request_code ? Or are you saying the size is encoded
> elsewhere in the virtio transport data ?
Yes, the buffer size is encoded in the transport data.
[...]
> This would be a good improvement, it's less copying and would likely
> improve user experience, however the buffer ptr still suffers latency
> as it's queued (same for stream positions going the other way).
Right if the queuing overhead is still too large, then I don't think the
current virtqueues can be used.
Thanks,
Jean
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 16:05 ` Jean-Philippe Brucker
@ 2019-11-12 18:02 ` Liam Girdwood
2019-11-12 18:47 ` Matti Moell
2019-11-13 9:44 ` Anton Yakovlev
0 siblings, 2 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-11-12 18:02 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
> > This would be a good improvement, it's less copying and would
> > likely
> > improve user experience, however the buffer ptr still suffers
> > latency
> > as it's queued (same for stream positions going the other way).
>
> Right if the queuing overhead is still too large, then I don't think
> the
> current virtqueues can be used.
They can be used for non low latency audio, I don't see any issues if
latency is not important. e.g. media playback, some gaming, most system
notifications.
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 18:02 ` Liam Girdwood
@ 2019-11-12 18:47 ` Matti Moell
2019-11-19 15:23 ` Liam Girdwood
2019-11-13 9:44 ` Anton Yakovlev
1 sibling, 1 reply; 42+ messages in thread
From: Matti Moell @ 2019-11-12 18:47 UTC (permalink / raw)
To: Liam Girdwood, Jean-Philippe Brucker
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
Hi Liam,
On 12.11.19 19:02, Liam Girdwood wrote:
> On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
>>> This would be a good improvement, it's less copying and would
>>> likely
>>> improve user experience, however the buffer ptr still suffers
>>> latency
>>> as it's queued (same for stream positions going the other way).
>>
>> Right if the queuing overhead is still too large, then I don't think
>> the
>> current virtqueues can be used.
>
> They can be used for non low latency audio, I don't see any issues if
> latency is not important. e.g. media playback, some gaming, most system
> notifications.
You are absolutely right that the virtques come with some overhead that
is probably acceptable in many use cases.
The trick for even lower latency will be to use the fancy new virtio
shared-mem feature, this will allow making a piece of shared memory
between the audio DSP/sound device and the guest available that contains
the current sample buffer and the HW position register. Thus, once this
is set-up, the DSP and the guest can directly communicate without the
VMM or the virtual device being involved at all.
Maybe it's even enough to just have a single register containing the
hardware buffer position in shared mem?
Of course this has to be guarded by a feature flag but it would allow
true zero copy audio where instead of using the queues you would just
have the producer and consumer free-running without interruptions (as
long es everything is fine).
The buffer layout is probably the trickiest thing here, how can we know
what the hardware expects? If we add all known buffer formats now, how
can we know if they are sufficient for future hardware? Does it make
sense to wait and add additional buffer formats as they become necessary
and known?
For the "simple" things like desktop virtualization and all the other
boring audio use cases using the queues and playing with the
driver/device kick and interrupt suppression seems to be sufficient to
reach the respective latency requirements.
Cheers,
Matti
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 14:20 ` Liam Girdwood
2019-11-12 16:05 ` Jean-Philippe Brucker
@ 2019-11-13 7:54 ` Anton Yakovlev
1 sibling, 0 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-13 7:54 UTC (permalink / raw)
To: Liam Girdwood, Jean-Philippe Brucker
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On 12.11.2019 15:20, Liam Girdwood wrote:
> On Tue, 2019-11-12 at 12:09 +0100, Jean-Philippe Brucker wrote:
>> Hi,
>>
>> On Mon, Nov 11, 2019 at 03:20:55PM +0000, Liam Girdwood wrote:
>>>>>> +struct virtio_snd_ctl_msg {
>>>>>> + /* device-read-only data */
>>>>>> + le32 request_code;
>>>>>> + u8 request_payload[];
>>>>>
>>>>> How do I know how much data to read here ? Is size embedded in
>>>>> request_code ?
>>>>
>>>> Yes, you are right. Actual request/response length can be easily
>>>> derived from
>>>> a request_code.
>>>>
>>>
>>> Nak, please use a separate size and type code otherwise we will
>>> have
>>> real problems in the future when we come to add new features (with
>>> new
>>> codes or bigger sizes).
>>
>> Other virtio devices rely on the buffer length already provided by
>> virtio
>> for this. For example see the virtio-net control virtqueue
>> (virtio_net_ctrl*), which is extended with new commands from time to
>> time.
>> Most recently, the "receive-side scaling" feature introduces a large
>> structure as payload to virtio_net_ctrl.
>>
>
> Sorry, not following, what does this have to do with combining type and
> size into request_code ? Or are you saying the size is encoded
> elsewhere in the virtio transport data ?
A virtqueue operates with sg-lists, where every entry has additional property
specifying whether it refers to RO or WO memory. Thus, as with any sg-list,
it's possible to calculate its buffer size.
>> [...]
>>>>>> +\begin{lstlisting}
>>>>>> +struct virtio_snd_pcm_xfer {
>>>>>> + le32 stream;
>>>>>> + u8 data[];
>>>>>> + le32 status;
>>>>>> + le32 actual_length;
>>>>>
>>>>> Not following this, is actual_length the size of data[]. If so,
>>>>> it
>>>>> must
>>>>> preceed data[].
>>>>
>>>> No, the actual_length field is supposed to be used by device side
>>>> to
>>>> report
>>>> actual amount of bytes read from/written to a buffer.
>>
>> By the way the written size is already provided by virtio (field len
>> in
>> the used descriptor), but not the read size.
>>
>>>> In real world
>>>> scenario,
>>>> if an I/O request contains N bytes, a device can play/capture *up
>>>> to*
>>>> N bytes.
>>>> Thus, it's required to report this length back to a driver.
>>>>
>>>
>>> This is not how PCM audio buffering works. I don't copy extra
>>> padding
>>> if I dont need to, I only copy when my buffer is full. See the need
>>> for
>>> period size in earlier comments.
>>
>> [...]
>>>>> 2) Zero copy of audio data and stream positions. Essential for
>>>>> any
>>>>> low
>>>>> latency audio use cases (if supported by HW, hypervisor, VM
>>>>> framework).
>>>>
>>>> What do you mean by zero-copy? Shared memory between device and
>>>> driver sides?
>>>
>>> Yes, but I see this is now a separate thread as it used by others
>>> too.
>>
>> The virtio transport already permits zero-copy, because one buffer
>> can be
>> described with a chain of descriptors, each pointing to a different
>> area
>> in memory. The driver can split the virtio_snd_pcm_xfer into header
>> and
>> data:
>>
>> virtqueue
>> desc ring
>> | |
>> +-------+
>> | |---------> header (copied)
>> +- - - -+
>> | |---------> PCM data (mapped)
>> +-------+
>> | |
>>
>> So it might be a good idea to put all the metadata (stream, status,
>> actual_length) at the beginning of the virtio_snd_pcm_xfer struct,
>> otherwise the driver will need three descriptors instead of two to
>> achieve
>
> This would be a good improvement, it's less copying and would likely
> improve user experience, however the buffer ptr still suffers latency
> as it's queued (same for stream positions going the other way).
It will violate virtio spec.
<quote>
2.6.4.2 Driver Requirements: Message Framing
The driver MUST place any device-writable descriptor elements after any
device-readable descriptor elements.
</quote>
and
<quote>
2.6.5 The Virtqueue Descriptor Table
... Each descriptor describes a buffer which is read-only for the device
(“device-readable”) or write-only for the device (“device-writable”), ...
</quote>
It means, that a device can "map" this memory as RO or WO and will be right.
Thus, you can not put in one descriptor both readable and writable fields. You
can take a look into virtio-blk spec. There is a reason, why they put status
field at the very end of a request.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 15:16 ` Liam Girdwood
@ 2019-11-13 9:05 ` Anton Yakovlev
2019-11-19 15:49 ` Liam Girdwood
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-13 9:05 UTC (permalink / raw)
To: Liam Girdwood, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
On 12.11.2019 16:16, Liam Girdwood wrote:
> On Tue, 2019-11-12 at 13:45 +0100, Anton Yakovlev wrote:
>> On 11.11.2019 16:20, Liam Girdwood wrote:
>>> On Tue, 2019-10-29 at 10:42 +0100, Anton Yakovlev wrote:
>>>> Hi Liam,
>>>>
>>>> Thank you for comments! Please, take a look at our answers below.
>>>
>>> Apologies, been travelling. More comments below.
>>>
>>>>
>>>>
>>>> On 28.10.2019 17:05, Liam Girdwood wrote:
>>>>> On Tue, 2019-09-24 at 15:43 +0100, Mikhail Golubev wrote:
>>>>>> From: Anton Yakovlev <Anton.Yakovlev@opensynergy.com>
>>>>>>
>>>>>>
>
> snip
>
>>>>>> diff --git a/virtio-sound.tex b/virtio-sound.tex
>>>>>> new file mode 100644
>>>>>> index 0000000..68a606e
>>>>>> --- /dev/null
>>>>>> +++ b/virtio-sound.tex
>>>>>> @@ -0,0 +1,405 @@
>>>>>> +\section{Sound Device}\label{sec:Device Types / Sound
>>>>>> Device}
>>>>>> +
>>>>>> +The virtio sound card is a virtual audio device supporting
>>>>>> output
>>>>>> and input PCM
>>>>>> +streams. All device control requests are placed into the
>>>>>> control
>>>>>> virtqueue, all
>>>>>> +I/O requests are placed into the PCM virtqueue.
>>>>>
>>>>> Where are stream position updates placed ? These are usually
>>>>> synchronous from the driver and enable the player application
>>>>> to
>>>>> render
>>>>> or read data from the correct locations in the buffers. This is
>>>>> latency
>>>>> sensitive data.
>>>>
>>>> It happens in an interrupt handler. Since PCM frame transfer is
>>>> message based,
>>>> the only possible way to figure out consumed or captured amount
>>>> of
>>>> frames is
>>>> to take a look into a message response.
>>>>
>>>
>>> Ok, this is what I thought. This wont work for low latency audio
>>> use
>>> cases since the buffers and position responses are queued (voice
>>> calls,
>>> some gaming, some system notifications like keypress or volume
>>> change).
>>
>> But it's not different from typical HW. A driver receives an
>> interrupt once
>> DMA transaction is finished (i.e. some part of data was copied
>> to/from host).
>> And hardware pointer value usually is extrapolated based on some
>> internal
>> device state.
>
> Modern HW can usually provide the HW position to the nearest word. Some
> older/bad HW provides to the nearest period though and this is when we
> have to estimate.
>
> Native (non virtual) audio device drivers typically have no latency
> reading the stream position and it's read at the same time as the
> system clock. Reading the same clocks together is important as it means
> the relationship between the two is deterministic and not subject to
> any jitter (which can be introduced by reading stream position over
> virtio).
>
>> A virtio driver could do something similar: it could either
>> return "real" pointer value (i.e. what it internally uses) or could
>> show fake
>> "extrapolated" value (like frame_rate * elappsed_time). Although,
>> it's a
>> different story.
>>
>>
>
> Virtio driver should return real stream position alongside real common
> system time (of when stream position was read).
Do you propose to share stream position via shared memory? Or will it be read
with special request?
>>> However, it's probably tolerable for most other use cases. What use
>>> cases have you tried in your testing ?
>>
>> All typical cases: multimedia (video + audio or audio only), network
>> streaming, video calls, system events, volume changing and so on.
>>
>> By the way, could you define the "low latency" term that we are
>> talking about?
>> Just to be on the same wave.
>>
>
> Low latency audio is when any application or processing can render data
> very close to the HW stream R/W position (typically low or sub
> millisecond). This needs zero copy buffering to HW on host/VM/user side
> and also timely and accurate stream positions.
Good.
>> es
>>>>>> /
>>>>>> Sound Device / Device configuration layout}
>>>>>> +
>>>>>> +\begin{lstlisting}
>>>>>> +/* supported PCM sample formats */
>>>>>> +enum {
>>>>>> + VIRTIO_SND_PCM_FMT_S8 = 0,
>>>>>> + VIRTIO_SND_PCM_FMT_U8,
>>>>>> + VIRTIO_SND_PCM_FMT_S16,
>>>>>> + VIRTIO_SND_PCM_FMT_U16,
>>>
>>> 24 bit.
>>
>> These kinds of sample (18/20/24-bit) were discussed here earlier and
>> caused
>> some concerns (regarding supporting in a guest, testing and so on),
>> so they
>> were removed for simplicity.
>>
>> From other side, if we are up to add these, maybe it's better not to
>> hardcode
>> format names and use some kind of generic description instead?
>> Something like
>>
>> struct {
>> kind; /* S/U/F */
>> significant_bits;
>> storage_bits;
>> }
>>
>> It allows to describe formats like 24 and 24_3 in unified way and to
>> add
>> future formats without changing the protocol. What do you think?
>>
>
> I would recommend copying the exact formats used by ALSA, these formats
> are industry standard (although naming will be different on other
> OSes)
I mean, there are two options:
1. We can go with defining format names. But then we will need to update the
protocol and (perhaps) existing drivers.
2. We can develop some generic way to report supported formats (as well, as
rates). And it will not require to touch the spec on every update.
>>
>>>>>> + VIRTIO_SND_PCM_FMT_S32,
>>>>>> + VIRTIO_SND_PCM_FMT_U32,
>>>>>> + VIRTIO_SND_PCM_FMT_FLOAT,
>>>>>> + VIRTIO_SND_PCM_FMT_FLOAT64
>>>>>> +};
>>>>>> +
>>>>>> +/* supported PCM frame rates */
>>>>>> +enum {
>>>>>> + VIRTIO_SND_PCM_RATE_8000 = 0,
>>>>>> + VIRTIO_SND_PCM_RATE_11025,
>>>>>> + VIRTIO_SND_PCM_RATE_16000,
>>>>>> + VIRTIO_SND_PCM_RATE_22050,
>>>>>> + VIRTIO_SND_PCM_RATE_32000,
>>>>>> + VIRTIO_SND_PCM_RATE_44100,
>>>>>> + VIRTIO_SND_PCM_RATE_48000,
>>>>>> + VIRTIO_SND_PCM_RATE_64000,
>>>>>> + VIRTIO_SND_PCM_RATE_88200,
>>>>>> + VIRTIO_SND_PCM_RATE_96000,
>>>>>> + VIRTIO_SND_PCM_RATE_176400,
>>>>>> + VIRTIO_SND_PCM_RATE_192000
>>>>>
>>>>> It may be best to use the same rates and format from the ALSA
>>>>> spec,
>>>>> this covers any holes (like rates that are not in the above
>>>>> list).
>>>>
>>>> These rates are from ALSA definitions. And what formats should be
>>>> added?
>>>>
>>>
>>> Rate, Knot - I would copy ALSA supported rates and formats just
>>> like
>>> with the channel maps.
>>
>> See a comment above.
>>
>
> See comment above :)
>>
>>>>>
>>>>>> +
>>>>>> +/* a device configuration space */
>>>>>> +struct virtio_snd_config {
>>>>>> + struct virtio_pcm_config {
>>>>>> + struct virtio_pcm_stream_config output;
>>>>>> + struct virtio_pcm_stream_config input;
>>>>>> + } pcm;
>>>>>> +};
>>>>>> +\end{lstlisting}
>>>>>
>>>>> I assuming this is sent to configure PCM capabilities ?
>>>>
>>>> Yes, minimum amount of information required to setup a PCM stream
>>>> in
>>>> any kind
>>>> of guest.
>>>>
>>>>
>>>>> What about buffer and period size capabilities ?
>>>>
>>>> In this specification we assume that buffer size is up to guest
>>>> decision.
>>>> Also, period size is ALSA-only conception and should not be
>>>> mentioned
>>>> here at
>>>> all (we are not stick only to ALSA).
>>>>
>>>>
>>>
>>> Period size == fragment size == block size (probably a few other
>>> naming
>>> conventions too), we should be able to tell the device VM our
>>> period/block/fragment size so we can configure the HW
>>> appropriately.
>>> This can also be used to make our stream copying more efficient
>>> too.
>>
>> Then let's talk about playback. As I can understand, you are
>> describing a
>> case, when
>> 1. ALSA application sets some period size
>> 2. fills the whole buffer with non-silence frames
>> 3. upon every period notification, provides next period-sized part of
>> data
>>
>> It's one of use cases, but quite rare.
>
> Ehm, it's bog standard file R/W (non mmap()) audio and extremely common
> on Linux and derivatives.
>
> Step 4 is
>
> 4) Unblock and wake userspace R/W for next period/block/fragment.
>
>> More often (especially if we are
>> talking about latency control and low latency audio) things will be
>> 1. an application fills the whole buffer with silence
>> 2. suppresses period notifications
>> 3. starts stream and almost immediatly seeks to near hw position
>> 4. puts non-silence frames there (size is set according to desired
>> latency)
>> 5. provides new frames at some interval (usually in 5-10ms)
>
> So this describes mmap() audio where we block by poll()ing, read HW
> ptr, and the write new frames.
>
> This proposal needs to support both operating modes and there is
> nothing I can see with virtio and some minor changes to this proposal
> that would prevent that ?
Yes, nothing prevents. I explained, why it was not here from the start.
>>
>> It's how software mixers usually work: PulseAudio, Windows in-kernel
>> mixer,
>> new Android system mixer and so on. Also, you can find such approach
>> in low
>> latency player applications (when they work directly with device).
>>
>> The point is they do not rely on notifications from the driver and
>> use their
>> own timers. The only thing they querying is current hw position.
>>
>> The point is we could have period notifications from the device. But
>> it seems
>> they will be useful only in minority of cases. Also, for example,
>> Windows
>> audio driver has no such concept at all.
>
> Windows does not need to use periodic mode, but a lot of Linux
> userspace will use it and this will currently will break those
> applications.
It does not break anything. Since the driver must notify on every elapsed
period in any case. And exactly that happens in an interrupt handler.
>>>>
>>>> Since xrun conditions are higher level conceptions, we decided to
>>>> delegate
>>>> such issues to guest application itself. It helps to make overall
>>>> design
>>>> simpler. And it seems there's not so much we can do if xrun
>>>> condition
>>>> is
>>>> happened on the device side.
>>>
>>> We can inform the guest. The guest userspace can then take any
>>> remedial
>>> action if needed.
>>
>> Then, we need the third queue for notifications.
>
> Why, this should go in the queue that's used for stream position ?
Then, the I/O queue will multiplex already three things: read requests, write
requests and notifications. The question is how rational is it.
>>
>>
>>>>
>>>>
>>>>> 2) Zero copy of audio data and stream positions. Essential for
>>>>> any
>>>>> low
>>>>> latency audio use cases (if supported by HW, hypervisor, VM
>>>>> framework).
>>>>
>>>> What do you mean by zero-copy? Shared memory between device and
>>>> driver sides?
>>>
>>> Yes, but I see this is now a separate thread as it used by others
>>> too.
>>
>> Shared memory based approach was proposed in the very first version
>> of the
>> spec (and we are rooting for this). Then there was discussion and it
>> was
>> postponed for some future virtio device. One of the reason - it's not
>> suitable
>> for arch with non-coherent memory between host and guest.
>
> I'm rooting for this too. I do think we need to be a bit more flexible
> so we can deal with non coherent architectures via a SW virtio copy and
> support coherent architectures via zero copy.
>
> We should be good as long as we can leave some configuration
> space/types in the stream config to allow mapping of operating modes
> (zero copy, periodic, mmap etc). The device driver should also send
> it's capabilities during init so that guests will know what use cases
> and modes are supported.
Yes, we can use device feature bits for this.
> Btw, please don't take my comments as trying to block your proposal. I
> am behind this work, I'll likely be one of the first users. My
> intention is to help provide something that is both flexible enough to
> cater for all current OS users/use cases on HDA like hardware and has
> scope for future enhancements.
As we are! :)
> Thanks
>
> Liam
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 18:02 ` Liam Girdwood
2019-11-12 18:47 ` Matti Moell
@ 2019-11-13 9:44 ` Anton Yakovlev
2019-11-19 16:09 ` Liam Girdwood
1 sibling, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-13 9:44 UTC (permalink / raw)
To: Liam Girdwood, Jean-Philippe Brucker
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On 12.11.2019 19:02, Liam Girdwood wrote:
> On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
>>> This would be a good improvement, it's less copying and would
>>> likely
>>> improve user experience, however the buffer ptr still suffers
>>> latency
>>> as it's queued (same for stream positions going the other way).
>>
>> Right if the queuing overhead is still too large, then I don't think
>> the
>> current virtqueues can be used.
>
> They can be used for non low latency audio, I don't see any issues if
> latency is not important. e.g. media playback, some gaming, most system
> notifications.
It can support low latency as well. We are not talking about high-throughput
device like block or network devices. Bit stream here is constant and quite
low (in compare with), so it's not a problem even with message based approach
to write/read frames near actual hw pointer.
And here I want to talk about real issue. I already mentioned it a few times:
operating system schedulers. You describe low latency solution, but you didn't
explain how are you gonna support realtime properties. The closer to hw
position we read/write - the easier to miss a deadline.
The fact: tasks can be delayed. Using RT-schedulers in soft realtime OSes does
not help much, since they do not give you any guarantees. Thus, delay can be
random and quite big, and it fully depends on number of available cores (and
their speed). If a VM has dedicated cores, it's great. But usually for type 2
hv we will have shared cores, that makes situation even worse.
Since low latency applications in a guest are not virtualization-awared, they
will push latency to the limit. From other side, virtio driver is
virtualization-awared and could assist somehow. In our draft solution with
message based approach we artificially increase a latency to an extent
allowing to avoid xrun condition due to the worst possible delay (just put
additional silence at the beginning of a stream). And it's still far from
ideal, since a length of that additional latency highly depends on hardware
and virtualization setup.
With zero-copy and friends based approach, what are you going to do with such
issues? Because (in general case) you might not have enough cores to run
things smoothly and deadline will be missed quite often.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191111193903.GE4264@sirena.co.uk>
@ 2019-11-13 12:01 ` Anton Yakovlev
[not found] ` <20191114212940.GC4664@sirena.co.uk>
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-13 12:01 UTC (permalink / raw)
To: Mark Brown
Cc: Liam Girdwood, Mikhail Golubev, virtio-dev, Takashi Iwai,
Marc Zyngier, James Morse, Julien Thierry, Suzuki K Poulose
On 11.11.2019 20:39, Mark Brown wrote:
> On Fri, Nov 01, 2019 at 02:37:12PM +0100, Anton Yakovlev wrote:
>> On 30.10.2019 13:11, Mark Brown wrote:
>>> On Tue, Oct 29, 2019 at 02:16:04PM +0100, Anton Yakovlev wrote:
>
>>>> Periods here are kind of notification frequency, right? It's supposed to be
>>>> used for sending notifications to driver?
>
>>> Yes, the driver gets a notification every time the hardware makes it
>>> through a periodn of data.
>
>> If a device implementation has reliable way to receive and forward such
>> notifications to a driver, then it's fine. The reason why we didn't add this
>> to the spec is because usually a device interacts with or implements by
>> itself some kind of software mixer. And all communications with real
>> hardware are hidden behind multiple abstraction layers. In such case, it's
>> either very difficult or even impossible to map period-based notifications
>> to whatever a sw mixer provides to its clients.
>
> Can you provide examples of host systems which have interfaces that
> provide absolutely no timing information that can be used by
> applications? It's common for systems to provide simplified interfaces
> for applications that have simple needs (especially "please play this
> track" style interfaces) but there's usually something richer there for
> the applications that need it.
Two common cases: PulseAudio and QEmu. PA has some timing info, but it states
clear that it's "rough estimations". QEmu provides nothing in this regard. And
it's understandable, since they need to provide some unified API/strategy to
clients while hiding details of actual backends (that could be a PCM device,
bluetooth, network, whatever).
>>>>>> No, the actual_length field is supposed to be used by device side to report
>>>>>> actual amount of bytes read from/written to a buffer. In real world
>>>>>> scenario, if an I/O request contains N bytes, a device can play/capture *up
>>>>>> to* N bytes. Thus, it's required to report this length back to a driver.
>
>>>>> So really there's two structs here, a header struct with just stream in
>>>>> it and then a tail struct with the length and status information which
>>>>> the consumer locates by looking at the end of the buffer and working
>>>>> backwards?
>
>>>> Yes, it is. Since both these structures are coupled together and must be
>>>> sent at the same time, in spec they are represented as one "structure".
>
>>> OK... that is a fairly weird and confusing way of writing things.
>
>> Not sure, how to describe this in context of the specification in a better way.
>
> "The start of the data will be this struct, the end of the data will be
> that struct".
Good, we will re-phrase this sentence. Thanks.
>>>> That's the point. The spec describes a device. Device is supposed just to
>>>> read from/write to hardware buffer regardless of its content.
>
>>> The rate at which data is consumed or produced in an audio system is
>>> driven by the audio hardware. This is very likely to not be synced to
>>> other clock domains in the system so users are reliant on the hardware
>>> to drive the rate at which data is driven through the system and hence
>>> need support from it for figuring out the rate to transfer data and
>>> handle any problems that occur. Good error reporting is a part of that,
>>> it makes it much easier to diagnose problems if you can get a direct
>>> and clear report of what went wrong rather than having to for example
>>> listen to the audio and work back from that. Like I say this is
>>> especially true with a message based system where software has less
>>> visibility of how the hardware is progressing through the buffer.
>
>> Yes, it's true. And it's not like we against having error reporting. The
>> problem is there's no deterministic way how to treat such errors in
>> pv-solution.
>
> I think you are being overly pessimistic about what is posible and not
> providing any room for better quality of implementation. I'm not seeing
> how a virtualized system is that different to an application running
> with a sound server.
That's why we needed discussion. It makes little sense to put something that
you are not sure into a public spec.
>> It's like with periods discussed above. If a device is a native
>> ALSA-application and talks directly with hardware in exclusive mode, then
>> it's capable to signal actual underflow/overflow condition reported by ALSA
>> layer. But if a device is a client of sw mixer server, then reported error
>> might be server-specific (like "underflow" means, that server's queue is
>> empty, but real hardware buffer still might contain 10ms or 50ms of frames
>> for playback; i.e. guest application probably has enough time to provide new
>> data).
>
> You're describing a quality of implementation issue there, if something
> is indicating that it's underlowed when it has not in fact underflowed
> then it's buggy.
Not necessary. It may implement its own workflow. And if our terms do not
align with their, it does not necessary mean they have buggy anything.
> In the above scenario it really depends if the server is able to edit
> the already rendered buffer, if the server is able to edit the data it
> shouldn't be reporting an underflow yet while if it is not then the
> audio will glitch and there has in fact been an underflow (and the
> server side rendering buffer already used a good chunk of the latency
> for many applications).
>
>> At the end, the main source of xrun conditions is delayed execution of a
>> virtual machine or an application in it (like double scheduling and so on).
>> And 44100Hz is still ~44100 frames per seconds. After different experiments,
>> we decided just to stick to simpler design.
>
> People are more than capable of writing software which has trouble even
> without any involvement from virtualization! Applications like VoIP,
> music production and gaming frequently try to push latencies very low.
>
> I'm not sure what you mean by experiments here but I don't really see
> how they are relevant here, obviously if the system is working well then
> none of this will come up - under and overflow are about situations
> where things go wrong.
I meant, what did work good enough and what not.
>> Again, it could be an optional feature if device has reliable source of
>> error notifications.
>
> Right now there is no option to provide the feature.
It's under discussion right now.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191114212940.GC4664@sirena.co.uk>
@ 2019-11-19 9:26 ` Anton Yakovlev
0 siblings, 0 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-19 9:26 UTC (permalink / raw)
To: Mark Brown
Cc: Liam Girdwood, Mikhail Golubev, virtio-dev, Takashi Iwai,
Marc Zyngier, James Morse, Julien Thierry, Suzuki K Poulose
On 14.11.2019 22:29, Mark Brown wrote:
> On Wed, Nov 13, 2019 at 01:01:56PM +0100, Anton Yakovlev wrote:
>> On 11.11.2019 20:39, Mark Brown wrote:
>
>>> Can you provide examples of host systems which have interfaces that
>>> provide absolutely no timing information that can be used by
>>> applications? It's common for systems to provide simplified interfaces
>>> for applications that have simple needs (especially "please play this
>>> track" style interfaces) but there's usually something richer there for
>>> the applications that need it.
>
>> Two common cases: PulseAudio and QEmu. PA has some timing info, but it
>> states clear that it's "rough estimations".
>
> PA does a lot better than it claims, and bear in mind that the PA
> specific APIs are simplified ones and it does expose emulations of far
> richer interfaces.
Yes, certainly, PA itself is pretty good.
>> QEmu provides nothing in this
>> regard. And it's understandable, since they need to provide some unified
>> API/strategy to clients while hiding details of actual backends (that could
>> be a PCM device, bluetooth, network, whatever).
>
> My understanding was that QEmu emulated some regular sound hardware?
Yes, it is. And all emulated sound hardware communicate with built-in QEmu
audio mixer.
>>>> Yes, it's true. And it's not like we against having error reporting. The
>>>> problem is there's no deterministic way how to treat such errors in
>>>> pv-solution.
>
>>> I think you are being overly pessimistic about what is posible and not
>>> providing any room for better quality of implementation. I'm not seeing
>>> how a virtualized system is that different to an application running
>>> with a sound server.
>
>> That's why we needed discussion. It makes little sense to put something that
>> you are not sure into a public spec.
>
> OK.
>
>>>> It's like with periods discussed above. If a device is a native
>>>> ALSA-application and talks directly with hardware in exclusive mode, then
>>>> it's capable to signal actual underflow/overflow condition reported by ALSA
>>>> layer. But if a device is a client of sw mixer server, then reported error
>>>> might be server-specific (like "underflow" means, that server's queue is
>>>> empty, but real hardware buffer still might contain 10ms or 50ms of frames
>>>> for playback; i.e. guest application probably has enough time to provide new
>>>> data).
>
>>> You're describing a quality of implementation issue there, if something
>>> is indicating that it's underlowed when it has not in fact underflowed
>>> then it's buggy.
>
>> Not necessary. It may implement its own workflow. And if our terms do not
>> align with their, it does not necessary mean they have buggy anything.
>
> To me it's about what the spec says - if the spec says X and you do Y
> then that's quality of implementation. There can definitely be issues
> translating the spec language into something that's useful in the
> context of the specific implementation but that's a normal part of
> interacting with a spec.
In that regards yes, you are right. We just talked about features that could
be useful in one particular case and totally useless in another one.
>>>> At the end, the main source of xrun conditions is delayed execution of a
>>>> virtual machine or an application in it (like double scheduling and so on).
>>>> And 44100Hz is still ~44100 frames per seconds. After different experiments,
>>>> we decided just to stick to simpler design.
>
>>> People are more than capable of writing software which has trouble even
>>> without any involvement from virtualization! Applications like VoIP,
>>> music production and gaming frequently try to push latencies very low.
>
>>> I'm not sure what you mean by experiments here but I don't really see
>>> how they are relevant here, obviously if the system is working well then
>>> none of this will come up - under and overflow are about situations
>>> where things go wrong.
>
>> I meant, what did work good enough and what not.
>
> I see. Part of my pessimism about the need for error handling here is
> that there's a great deal of quality variation in audio hardware and
> software out there so lots of potential for things to not behave as
> expected when put under stress. Adding a virtualization layer seems
> like the sort of stress that might cause problems, hopefully not but it
> seems better to at least have the facilities in the spec so it's easier
> to resolve any problems that do come up.
Good, I will summarize all the points from our discussion and we will decide
how to improve current spec draft.
>>>> Again, it could be an optional feature if device has reliable source of
>>>> error notifications.
>
>>> Right now there is no option to provide the feature.
>
>> It's under discussion right now.
>
> Ah, I see - I may be confused about the process, I saw a call for votes
> which looked like things were pretty final.
We had no comments for quite a long time. And it looked like the spec is ready.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-12 18:47 ` Matti Moell
@ 2019-11-19 15:23 ` Liam Girdwood
0 siblings, 0 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-11-19 15:23 UTC (permalink / raw)
To: Matti Moell, Jean-Philippe Brucker
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Tue, 2019-11-12 at 19:47 +0100, Matti Moell wrote:
> Hi Liam,
>
> On 12.11.19 19:02, Liam Girdwood wrote:
> > On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
> > > > This would be a good improvement, it's less copying and would
> > > > likely
> > > > improve user experience, however the buffer ptr still suffers
> > > > latency
> > > > as it's queued (same for stream positions going the other way).
> > >
> > > Right if the queuing overhead is still too large, then I don't
> > > think
> > > the
> > > current virtqueues can be used.
> >
> > They can be used for non low latency audio, I don't see any issues
> > if
> > latency is not important. e.g. media playback, some gaming, most
> > system
> > notifications.
>
> You are absolutely right that the virtques come with some overhead
> that
> is probably acceptable in many use cases.
>
> The trick for even lower latency will be to use the fancy new virtio
> shared-mem feature, this will allow making a piece of shared memory
> between the audio DSP/sound device and the guest available that
> contains
> the current sample buffer and the HW position register. Thus, once
> this
> is set-up, the DSP and the guest can directly communicate without
> the
> VMM or the virtual device being involved at all.
>
> Maybe it's even enough to just have a single register containing the
> hardware buffer position in shared mem?
How would this work on the device driver side. Do "virtual register"
reads map to callbacks on the device driver (with assumed hypervisor
scheduling latencies?).
Fwiw, some audio HW like the Intel audio DSP or some HDA controllers do
expose stream position HW registers and I do intend to map the DSP
stream pointers as RO "registers" to guests (as a virtual device PCI
BAR).
>
> Of course this has to be guarded by a feature flag but it would
> allow
> true zero copy audio where instead of using the queues you would
> just
> have the producer and consumer free-running without interruptions
> (as
> long es everything is fine).
>
> The buffer layout is probably the trickiest thing here, how can we
> know
> what the hardware expects? If we add all known buffer formats now,
> how
> can we know if they are sufficient for future hardware? Does it make
> sense to wait and add additional buffer formats as they become
> necessary
> and known?
My understanding from the spec is that the params from frames and
buffers are sent during PCM initialisation so these should be passed by
the device driver to the HW. I don't see any new formats being added
here either as this has been mature for a long time.
Thanks
Liam
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-13 9:05 ` Anton Yakovlev
@ 2019-11-19 15:49 ` Liam Girdwood
2019-11-20 9:32 ` Gerd Hoffmann
0 siblings, 1 reply; 42+ messages in thread
From: Liam Girdwood @ 2019-11-19 15:49 UTC (permalink / raw)
To: Anton Yakovlev, Mikhail Golubev, virtio-dev; +Cc: Takashi Iwai, Mark Brown
On Wed, 2019-11-13 at 10:05 +0100, Anton Yakovlev wrote:
> > > > >
> > > > > Since xrun conditions are higher level conceptions, we
> > > > > decided to
> > > > > delegate
> > > > > such issues to guest application itself. It helps to make
> > > > > overall
> > > > > design
> > > > > simpler. And it seems there's not so much we can do if xrun
> > > > > condition
> > > > > is
> > > > > happened on the device side.
> > > >
> > > > We can inform the guest. The guest userspace can then take any
> > > > remedial
> > > > action if needed.
> > >
> > > Then, we need the third queue for notifications.
> >
> > Why, this should go in the queue that's used for stream position ?
>
> Then, the I/O queue will multiplex already three things: read
> requests, write
> requests and notifications. The question is how rational is it.
If there is no multiplexing, then we probably need 4 queues per virtual
PCM:
1) Playback data
2) Playback notifications
3) Capture data
4) Capture notifications
additionally, if we include a controls (like volume, mixer etc) we can
also include:
5) control W data
6) control R data
Latency is important for 1 - 4 (where multiplexing may not be
desirable), but 5 & 6 can be easily multiplexed for multiple controls.
>
>
> > >
> > >
> > > > >
> > > > >
> > > > > > 2) Zero copy of audio data and stream positions. Essential
> > > > > > for
> > > > > > any
> > > > > > low
> > > > > > latency audio use cases (if supported by HW, hypervisor, VM
> > > > > > framework).
> > > > >
> > > > > What do you mean by zero-copy? Shared memory between device
> > > > > and
> > > > > driver sides?
> > > >
> > > > Yes, but I see this is now a separate thread as it used by
> > > > others
> > > > too.
> > >
> > > Shared memory based approach was proposed in the very first
> > > version
> > > of the
> > > spec (and we are rooting for this). Then there was discussion and
> > > it
> > > was
> > > postponed for some future virtio device. One of the reason - it's
> > > not
> > > suitable
> > > for arch with non-coherent memory between host and guest.
> >
> > I'm rooting for this too. I do think we need to be a bit more
> > flexible
> > so we can deal with non coherent architectures via a SW virtio copy
> > and
> > support coherent architectures via zero copy.
> >
> > We should be good as long as we can leave some configuration
> > space/types in the stream config to allow mapping of operating
> > modes
> > (zero copy, periodic, mmap etc). The device driver should also send
> > it's capabilities during init so that guests will know what use
> > cases
> > and modes are supported.
>
> Yes, we can use device feature bits for this.
Can we also include some reserved words to ease spec updates ? e.g.
guest is using older version of spec than device driver but we still
want working audio.
Thanks
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-13 9:44 ` Anton Yakovlev
@ 2019-11-19 16:09 ` Liam Girdwood
2019-11-20 14:24 ` Anton Yakovlev
0 siblings, 1 reply; 42+ messages in thread
From: Liam Girdwood @ 2019-11-19 16:09 UTC (permalink / raw)
To: Anton Yakovlev, Jean-Philippe Brucker
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Wed, 2019-11-13 at 10:44 +0100, Anton Yakovlev wrote:
> On 12.11.2019 19:02, Liam Girdwood wrote:
> > On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
> > > > This would be a good improvement, it's less copying and would
> > > > likely
> > > > improve user experience, however the buffer ptr still suffers
> > > > latency
> > > > as it's queued (same for stream positions going the other way).
> > >
> > > Right if the queuing overhead is still too large, then I don't
> > > think
> > > the
> > > current virtqueues can be used.
> >
> > They can be used for non low latency audio, I don't see any issues
> > if
> > latency is not important. e.g. media playback, some gaming, most
> > system
> > notifications.
>
> It can support low latency as well.
Not really, without zero copy buffering and immediate position
reporting it's difficult to get latency below a certain threshold. As
discussed most use cases don't care, but some use cases do.
> We are not talking about high-throughput
> device like block or network devices. Bit stream here is constant and
> quite
> low (in compare with), so it's not a problem even with message based
> approach
> to write/read frames near actual hw pointer.
>
Throughput has no relation to latency. MMC cards have a great
throughput but have high latency. e.g. recording a WAV to MMC using a
small application buffer (say 8 - 16k) will consistently overflow the
buffer since MMC write latency is poor (but once your start writing
throughput is very high).
> And here I want to talk about real issue. I already mentioned it a
> few times:
> operating system schedulers. You describe low latency solution, but
> you didn't
> explain how are you gonna support realtime properties. The closer to
> hw
> position we read/write - the easier to miss a deadline.
>
This is hypervisor/HW/guest specific and unrelated to this proposal.
In general if I have guest core affinity then hypervisor context
switching for that guest is in low 10s of uS (depends on hypervisor of
course and HW). This is good enough for low latency audio.
> The fact: tasks can be delayed. Using RT-schedulers in soft realtime
> OSes does
> not help much, since they do not give you any guarantees. Thus, delay
> can be
> random and quite big, and it fully depends on number of available
> cores (and
> their speed). If a VM has dedicated cores, it's great. But usually
> for type 2
> hv we will have shared cores, that makes situation even worse.
>
> Since low latency applications in a guest are not virtualization-
> awared, they
> will push latency to the limit. From other side, virtio driver is
> virtualization-awared and could assist somehow. In our draft solution
> with
> message based approach we artificially increase a latency to an
> extent
> allowing to avoid xrun condition due to the worst possible delay
> (just put
> additional silence at the beginning of a stream). And it's still far
> from
> ideal, since a length of that additional latency highly depends on
> hardware
> and virtualization setup.
I'm saying there is nothing wrong with the message based approach for
most use cases. We just need to build in support (to the configuration
structures) for zero copy and immediate position reporting so that
guests/drivers that can support it can use it.
I would like the PCM configuration data to be robust enough to be able
to report such capabilities and for guests to configure these
capabilities if supported. I know a lot of the buffer sharing APIs are
at proposal stage, so I think we are good as long as we have headroom
to add these later (without breaking any backward compatibility).
Thanks
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-19 15:49 ` Liam Girdwood
@ 2019-11-20 9:32 ` Gerd Hoffmann
0 siblings, 0 replies; 42+ messages in thread
From: Gerd Hoffmann @ 2019-11-20 9:32 UTC (permalink / raw)
To: Liam Girdwood
Cc: Anton Yakovlev, Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
> > Then, the I/O queue will multiplex already three things: read
> > requests, write
> > requests and notifications. The question is how rational is it.
>
> If there is no multiplexing, then we probably need 4 queues per virtual
> PCM:
>
> 1) Playback data
> 2) Playback notifications
> 3) Capture data
> 4) Capture notifications
>
> additionally, if we include a controls (like volume, mixer etc) we can
> also include:
>
> 5) control W data
> 6) control R data
>
> Latency is important for 1 - 4 (where multiplexing may not be
> desirable), but 5 & 6 can be easily multiplexed for multiple controls.
You need:
(1) control queue (guest -> host commands)
(2) events queue (host -> guest notifications, but see below)
(3) tx queue (playback)
(4) rx queue (recording)
You can allow multiple rx/tx pairs, not sure multiqueue support buys us
much for audio though.
The tx/rx queue requests should have three elements: a small request
header (out), a small request status (in), and one (or more) data
buffers (out for tx, in for rx) for the actual sample data (see also
the more verbose comment elsewhere in this thread).
The request status can probably be used to notify the guest about
overruns and underruns, so maybe the event queue is not needed.
> > Yes, we can use device feature bits for this.
>
> Can we also include some reserved words to ease spec updates ? e.g.
> guest is using older version of spec than device driver but we still
> want working audio.
feature bits handle this. The device knows which features are supported
by the driver and can act accordingly. Reserved fields in structs might
make sense to simplify that (avoid struct layout change depending on
feature bits).
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-19 16:09 ` Liam Girdwood
@ 2019-11-20 14:24 ` Anton Yakovlev
2019-11-21 9:04 ` Gerd Hoffmann
2019-11-25 16:50 ` Liam Girdwood
0 siblings, 2 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-20 14:24 UTC (permalink / raw)
To: Liam Girdwood, Jean-Philippe Brucker
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On 19.11.2019 17:09, Liam Girdwood wrote:
> On Wed, 2019-11-13 at 10:44 +0100, Anton Yakovlev wrote:
>> On 12.11.2019 19:02, Liam Girdwood wrote:
>>> On Tue, 2019-11-12 at 17:05 +0100, Jean-Philippe Brucker wrote:
snip
>> And here I want to talk about real issue. I already mentioned it a
>> few times:
>> operating system schedulers. You describe low latency solution, but
>> you didn't
>> explain how are you gonna support realtime properties. The closer to
>> hw
>> position we read/write - the easier to miss a deadline.
>>
>
> This is hypervisor/HW/guest specific and unrelated to this proposal.
>
> In general if I have guest core affinity then hypervisor context
> switching for that guest is in low 10s of uS (depends on hypervisor of
> course and HW). This is good enough for low latency audio.
Honestly, I'm not sure whether this relates to the proposed spec or not.
I want to describe an experiment we conducted regarding possible zero-copy
approach. We had simple audio device in QEmu and simple ALSA driver in virtual
guest.
The driver basically allocated a buffer, allowed to mmap it into user-space
application and shared this memory with the device (i.e. device could access
the whole buffer at any moment of time). Then we shared yet one memory page
where we placed current application pointer value. We did it in order to avoid
possible issues with sharing actual ALSA control page. Also, we used
the SNDRV_PCM_INFO_SYNC_APPLPTR feature to enforce the application to notify
kernel on every position update. Thus, we could update our shared application
pointer value immediately. As a hardware position we returned estimated one
(frame rate * elapsed time).
On playback stream start, QEmu device activated a dedicated thread (SCHED_FIFO
/ high priority) that waked up every 10ms, shifted its own position value to
10ms and compared it with current application pointer value.
In short, we tried to simulate the closest possible HW behavior for zero-copy
low latency case. It might be not so perfect, but at least it gave us more or
less accurate estimation.
A host had 4 physical cores (2 cores with enabled HT), a guest had 4 vcpus
and we used as a test PA and some low latency applications (like voice call
and so on). And we observed empty buffer ~1-2 times per second. It
would most probably mean underrun conditions if it were a real zero-copy
implementation. Also we found out that different Linux distros as a guest give
different results (but always not so good for audio). :)
From other side, we also had a host with 8 physical cores (4 cores with
enabled HT) and a guest had the same 4 vcpus. And there the same setup worked
perfectly.
That's why we are really curious how other people target such issues with
audio virtualization. And that's why I'm not sure about the spec in this
regard. We could say something like "you can activate zero-copy mode, but it's
up to your HW, OS and luck", but it does not sound very nicely.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-20 14:24 ` Anton Yakovlev
@ 2019-11-21 9:04 ` Gerd Hoffmann
2019-11-21 12:57 ` Anton Yakovlev
2019-11-25 16:50 ` Liam Girdwood
1 sibling, 1 reply; 42+ messages in thread
From: Gerd Hoffmann @ 2019-11-21 9:04 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
> > In general if I have guest core affinity then hypervisor context
> > switching for that guest is in low 10s of uS (depends on hypervisor
> > of course and HW). This is good enough for low latency audio.
> That's why we are really curious how other people target such issues
> with audio virtualization. And that's why I'm not sure about the spec
> in this regard. We could say something like "you can activate
> zero-copy mode, but it's up to your HW, OS and luck", but it does not
> sound very nicely.
real-time stuff in virtual machines is tricky, and real-time audio isn't
special here. I don't think there is a need to discuss that in the
device specification.
The part which is important for the virtual hardware is to design the it
in a way that the number of context switches (system calls, vmexits) is
minimized.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-21 9:04 ` Gerd Hoffmann
@ 2019-11-21 12:57 ` Anton Yakovlev
2019-11-22 7:19 ` Gerd Hoffmann
[not found] ` <20191122123521.GB6849@sirena.org.uk>
0 siblings, 2 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-21 12:57 UTC (permalink / raw)
Cc: Gerd Hoffmann, Liam Girdwood, Jean-Philippe Brucker,
Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
Again, thank you all for your valuable comments! There were quite a lot of
suggestions, so I briefly gathered all of them.
Feature bits
------------
VIRTIO_SND_F_PCM_OUTPUT
VIRTIO_SND_F_PCM_INPUT
and new ones would be
VIRTIO_SND_F_PCM_EVT_PERIOD - support elapsed period notifications
VIRTIO_SND_F_PCM_EVT_XRUN - support underrun/overrun notifications
VIRTIO_SND_F_PCM_MSG - support message-based data transport
VIRTIO_SND_F_PCM_HOST_MEM - support sharing host memory with guest
VIRTIO_SND_F_PCM_GUEST_MEM - support sharing guest memory with host
Not sure about correct naming here.
By the way, Liam mentioned something about using PCI BARs for mapping host
memory. Does it mean, that some features will be transport-specific?
What about virtio mmio?
Virtqueues
----------
As Gerd explained, we need 4 virtqueues in total:
1. control queue (guest->host: commands)
2. events queue (host->guest: notifications)
3. tx queue (guest->host: playback)
4. rx queue (host->guest: capture)
The first one always exists, but all other depend on feature bits. Should we
check and assign these in some pre-defined order?
Configuration space
-------------------
We talked about PCM formats defined in ALSA. In Linux kernel v5.4-rc8 we have
SNDRV_PCM_FORMAT_S8
SNDRV_PCM_FORMAT_U8
SNDRV_PCM_FORMAT_S16
SNDRV_PCM_FORMAT_U16
SNDRV_PCM_FORMAT_S24
SNDRV_PCM_FORMAT_U24
SNDRV_PCM_FORMAT_S32
SNDRV_PCM_FORMAT_U32
SNDRV_PCM_FORMAT_FLOAT
SNDRV_PCM_FORMAT_FLOAT64
SNDRV_PCM_FORMAT_IEC958_SUBFRAME
SNDRV_PCM_FORMAT_MU_LAW
SNDRV_PCM_FORMAT_A_LAW
SNDRV_PCM_FORMAT_IMA_ADPCM
SNDRV_PCM_FORMAT_MPEG
SNDRV_PCM_FORMAT_GSM
SNDRV_PCM_FORMAT_S20
SNDRV_PCM_FORMAT_U20
SNDRV_PCM_FORMAT_S24_3
SNDRV_PCM_FORMAT_U24_3
SNDRV_PCM_FORMAT_S20_3
SNDRV_PCM_FORMAT_U20_3
SNDRV_PCM_FORMAT_S18_3
SNDRV_PCM_FORMAT_U18_3
SNDRV_PCM_FORMAT_G723_24
SNDRV_PCM_FORMAT_G723_24_1B
SNDRV_PCM_FORMAT_G723_40
SNDRV_PCM_FORMAT_G723_40_1B
SNDRV_PCM_FORMAT_DSD_U8
SNDRV_PCM_FORMAT_DSD_U16
SNDRV_PCM_FORMAT_DSD_U32
Do we really need all of them? And I skipped endianess, but should we care
about it as well?
It also affects how large configuration space should be.
PCM control requests
--------------------
There were comments about having additional request for freeing resources
on device side and regarding invalid command sequences. Maybe it's better to
define kind of state machine and describe what is expected from device and
driver at every state/transition? It will help to define valid command
sequences as well.
Common control message structure will be explicitly splitted into two
struct virtio_snd_ctl_req {
le32 type;
u8 payload[];
};
struct virtio_snd_ctl_rsp {
le32 status;
u8 payload[];
};
and the SET_FORMAT request will be replaced with the SET_PARAMS
struct virtio_snd_pcm_set_params {
struct virtio_snd_pcm_hdr hdr;
le32 channels;
le32 format;
le32 rate;
le32 period_size;
};
We only need to decide units for the period_size field (frames or bytes).
PCM notifications
-----------------
I think all types of notification (two at the moment) could be described
by single structure:
enum {
VIRTIO_SND_PCM_EVT_PERIOD,
VIRTIO_SND_PCM_EVT_XRUN
};
struct virtio_snd_event {
le32 type; // VIRTIO_SND_PCM_EVT_XXX
};
It's unclear whether it requires some payload or not.
PCM I/O requests
--------------------
As we get rid of multiplexing, we don't need a header anymore. And a message
will have only status
struct virtio_snd_pcm_status {
le32 status;
le32 actual_length;
};
Hope, I didn't forget anything.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-21 12:57 ` Anton Yakovlev
@ 2019-11-22 7:19 ` Gerd Hoffmann
2019-11-22 13:46 ` Anton Yakovlev
[not found] ` <20191122123521.GB6849@sirena.org.uk>
1 sibling, 1 reply; 42+ messages in thread
From: Gerd Hoffmann @ 2019-11-22 7:19 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
> Feature bits
> ------------
>
> VIRTIO_SND_F_PCM_OUTPUT
> VIRTIO_SND_F_PCM_INPUT
>
> and new ones would be
>
> VIRTIO_SND_F_PCM_EVT_PERIOD - support elapsed period notifications
> VIRTIO_SND_F_PCM_EVT_XRUN - support underrun/overrun notifications
> VIRTIO_SND_F_PCM_MSG - support message-based data transport
> VIRTIO_SND_F_PCM_HOST_MEM - support sharing host memory with guest
> VIRTIO_SND_F_PCM_GUEST_MEM - support sharing guest memory with host
>
> Not sure about correct naming here.
Hmm, I think at least parts of this should be device configuration not
feature bits ...
Instead of PCM_OUTPUT and PCM_INPUT I would put the number of input and
output streams into the device config space (and explicitly allow count
being zero).
PCM_EVT_* probably makes sense to be a per-stream configuration bit.
PCM_MSG -- I would drop the feature bit and make that mandatory, so we
have a common baseline on which all drivers and devices can agree on.
> By the way, Liam mentioned something about using PCI BARs for mapping host
> memory. Does it mean, that some features will be transport-specific?
In the latest spec revision virtio got support for shared memory
regions. This should be used for VIRTIO_SND_F_PCM_HOST_MEM.
> What about virtio mmio?
Right now this is defined for the pci transport only.
> Virtqueues
> ----------
>
> As Gerd explained, we need 4 virtqueues in total:
>
> 1. control queue (guest->host: commands)
> 2. events queue (host->guest: notifications)
> 3. tx queue (guest->host: playback)
> 4. rx queue (host->guest: capture)
>
> The first one always exists, but all other depend on feature bits. Should we
> check and assign these in some pre-defined order?
I'd suggest to make them all mandatory.
> Configuration space
> -------------------
>
> We talked about PCM formats defined in ALSA. In Linux kernel v5.4-rc8 we have
>
> Do we really need all of them? And I skipped endianess, but should we care
> about it as well?
I'd keep the list as short as possible. The formats listed in the patch
looked like a good start.
> It also affects how large configuration space should be.
I'd make the format bitfield 64bit no matter what. It doesn't cost much
and it leaves room to extend things without hassle if needed.
> Common control message structure will be explicitly splitted into two
>
> struct virtio_snd_ctl_req {
> le32 type;
le32 stream_id;
> u8 payload[];
> };
> and the SET_FORMAT request will be replaced with the SET_PARAMS
>
> struct virtio_snd_pcm_set_params {
> struct virtio_snd_pcm_hdr hdr;
> le32 channels;
> le32 format;
> le32 rate;
> le32 period_size;
> };
>
> We only need to decide units for the period_size field (frames or bytes).
I'd suggest bytes (and rename the field to period_bytes).
> PCM I/O requests
> --------------------
>
> As we get rid of multiplexing, we don't need a header anymore.
What if a device wants support two playback streams (possibly even with
different features, say one stereo and one 5:1)?
> And a message
> will have only status
>
> struct virtio_snd_pcm_status {
> le32 status;
> le32 actual_length;
> };
Hmm. Why actual_length? Do we want allow short transfers? Why?
Wouldn't it be more useful to just fill the buffer completely?
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-22 7:19 ` Gerd Hoffmann
@ 2019-11-22 13:46 ` Anton Yakovlev
2019-11-25 9:31 ` Gerd Hoffmann
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-22 13:46 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
On 22.11.2019 08:19, Gerd Hoffmann wrote:
>> Feature bits
>> ------------
>>
>> VIRTIO_SND_F_PCM_OUTPUT
>> VIRTIO_SND_F_PCM_INPUT
>>
>> and new ones would be
>>
>> VIRTIO_SND_F_PCM_EVT_PERIOD - support elapsed period notifications
>> VIRTIO_SND_F_PCM_EVT_XRUN - support underrun/overrun notifications
>> VIRTIO_SND_F_PCM_MSG - support message-based data transport
>> VIRTIO_SND_F_PCM_HOST_MEM - support sharing host memory with guest
>> VIRTIO_SND_F_PCM_GUEST_MEM - support sharing guest memory with host
>>
>> Not sure about correct naming here.
>
> Hmm, I think at least parts of this should be device configuration not
> feature bits ...
>
> Instead of PCM_OUTPUT and PCM_INPUT I would put the number of input and
> output streams into the device config space (and explicitly allow count
> being zero).
There are several options for the number of streams:
1. The device can provide one input and/or output stream.
2. The device can provide several input and/or output streams, and all of them
share the same capabilities.
3. The device can provide several input and/or output streams, and they can
have different set of capabilities.
Overall design depends on chosen option. Current draft is based on p.1. But we
have never discussed what is the best solution here.
> PCM_EVT_* probably makes sense to be a per-stream configuration bit.
Yes, it definitely makes sense.
> PCM_MSG -- I would drop the feature bit and make that mandatory, so we
> have a common baseline on which all drivers and devices can agree on.
Then we need to inform device what "transport" will be in use (I assumed it
would be feature negotiation).
>> By the way, Liam mentioned something about using PCI BARs for mapping host
>> memory. Does it mean, that some features will be transport-specific?
>
> In the latest spec revision virtio got support for shared memory
> regions. This should be used for VIRTIO_SND_F_PCM_HOST_MEM.
>
>> What about virtio mmio?
>
> Right now this is defined for the pci transport only.
And now we only declare a feature bit, right?
>> Virtqueues
>> ----------
>>
>> As Gerd explained, we need 4 virtqueues in total:
>>
>> 1. control queue (guest->host: commands)
>> 2. events queue (host->guest: notifications)
>> 3. tx queue (guest->host: playback)
>> 4. rx queue (host->guest: capture)
>>
>> The first one always exists, but all other depend on feature bits. Should we
>> check and assign these in some pre-defined order?
>
> I'd suggest to make them all mandatory.
Yes, it would be the simplest solution.
>> Configuration space
>> -------------------
>>
>> We talked about PCM formats defined in ALSA. In Linux kernel v5.4-rc8 we have
>>
>> Do we really need all of them? And I skipped endianess, but should we care
>> about it as well?
>
> I'd keep the list as short as possible. The formats listed in the patch
> looked like a good start.
It was proposed to have all non-compressed formats.
And... do we care about cross-endian cases?
>> It also affects how large configuration space should be.
>
> I'd make the format bitfield 64bit no matter what. It doesn't cost much
> and it leaves room to extend things without hassle if needed.
Good, then it will be le64.
>> Common control message structure will be explicitly splitted into two
>>
>> struct virtio_snd_ctl_req {
>> le32 type;
> le32 stream_id;
>> u8 payload[];
>> };
>
>> and the SET_FORMAT request will be replaced with the SET_PARAMS
>>
>> struct virtio_snd_pcm_set_params {
>> struct virtio_snd_pcm_hdr hdr;
>> le32 channels;
>> le32 format;
>> le32 rate;
>> le32 period_size;
>> };
>>
>> We only need to decide units for the period_size field (frames or bytes).
>
> I'd suggest bytes (and rename the field to period_bytes).
Okey.
>> PCM I/O requests
>> --------------------
>>
>> As we get rid of multiplexing, we don't need a header anymore.
>
> What if a device wants support two playback streams (possibly even with
> different features, say one stereo and one 5:1)?
We need to make a design choice (see comment above).
>> And a message
>> will have only status
>>
>> struct virtio_snd_pcm_status {
>> le32 status;
>> le32 actual_length;
>> };
>
> Hmm. Why actual_length? Do we want allow short transfers? Why?
> Wouldn't it be more useful to just fill the buffer completely?
In current design we have no buffer size requirements. It means, that in
theory the device and the driver could have buffers with different sizes.
Also, the capture stream is a special case. Now we don't state explicitly
whether read request is blockable or not. And if the device complete it
immediately, the request could contain less frames than was expected by the
driver.
> cheers,
> Gerd
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191122123521.GB6849@sirena.org.uk>
@ 2019-11-22 14:06 ` Anton Yakovlev
[not found] ` <D68AC0D0-851B-4F87-BE0E-F49527B83E24@redhat.com>
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-11-22 14:06 UTC (permalink / raw)
To: Mark Brown
Cc: Gerd Hoffmann, Liam Girdwood, Jean-Philippe Brucker,
Mikhail Golubev, virtio-dev, Takashi Iwai
On 22.11.2019 13:35, Mark Brown wrote:
> On Thu, Nov 21, 2019 at 01:57:51PM +0100, Anton Yakovlev wrote:
>
>> SNDRV_PCM_FORMAT_MPEG
>> SNDRV_PCM_FORMAT_GSM
>> SNDRV_PCM_FORMAT_G723_24
>> SNDRV_PCM_FORMAT_G723_24_1B
>> SNDRV_PCM_FORMAT_G723_40
>> SNDRV_PCM_FORMAT_G723_40_1B
>
>> Do we really need all of them? And I skipped endianess, but should we care
>> about it as well?
>
> The compressed audio formats I quoted above (and possibly some others)
> are mostly redundant and better handled through something like the ALSA
> compressed audio interface but that's definitely a separate thing with
> different requirements.
Then let's include all except these.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-22 13:46 ` Anton Yakovlev
@ 2019-11-25 9:31 ` Gerd Hoffmann
2019-12-02 13:06 ` Anton Yakovlev
0 siblings, 1 reply; 42+ messages in thread
From: Gerd Hoffmann @ 2019-11-25 9:31 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
> > Instead of PCM_OUTPUT and PCM_INPUT I would put the number of input and
> > output streams into the device config space (and explicitly allow count
> > being zero).
>
> There are several options for the number of streams:
>
> 1. The device can provide one input and/or output stream.
> 2. The device can provide several input and/or output streams, and all of them
> share the same capabilities.
> 3. The device can provide several input and/or output streams, and they can
> have different set of capabilities.
>
> Overall design depends on chosen option. Current draft is based on p.1. But we
> have never discussed what is the best solution here.
(3) looks most useful to me.
> > PCM_MSG -- I would drop the feature bit and make that mandatory, so we
> > have a common baseline on which all drivers and devices can agree on.
>
> Then we need to inform device what "transport" will be in use (I assumed it
> would be feature negotiation).
Whenever other transports (i.e. via shared memory) are supported: yes,
that should be a feature bit.
Not sure about choosing the transport. If both msg (i.e. via virtqueue)
and shared memory are available, does it make sense to allow the driver
choose the transport each time it starts a stream?
> > Right now this is defined for the pci transport only.
>
> And now we only declare a feature bit, right?
Not fully sure what you mean here.
Having the first revision define only the virtqueue transport, reserve a
feature bit for the shared memory transport and hash out the details of
that later is possible.
> > > Configuration space
> > > -------------------
> > >
> > > We talked about PCM formats defined in ALSA. In Linux kernel v5.4-rc8 we have
> > >
> > > Do we really need all of them? And I skipped endianess, but should we care
> > > about it as well?
> >
> > I'd keep the list as short as possible. The formats listed in the patch
> > looked like a good start.
>
> It was proposed to have all non-compressed formats.
>
> And... do we care about cross-endian cases?
I'd start with little endian formats only. virtio byte order is little
endian, and the world is moving to little endian anyway. We can add big
endian formats later should this turn out to be needed, but I don't
expect that to be the case.
> > I'd suggest bytes (and rename the field to period_bytes).
> > > struct virtio_snd_pcm_status {
> > > le32 status;
> > > le32 actual_length;
> > > };
> >
> > Hmm. Why actual_length? Do we want allow short transfers? Why?
> > Wouldn't it be more useful to just fill the buffer completely?
>
> In current design we have no buffer size requirements. It means, that in
> theory the device and the driver could have buffers with different sizes.
Should we define that the (virtio) buffer size must be period_bytes then?
> Also, the capture stream is a special case. Now we don't state explicitly
> whether read request is blockable or not.
The concept of "blockable" doesn't exist at that level. The driver
submits buffers to the device, the device fills them and notifies the
driver when the buffer is full. It simply doesn't work like a read(2)
syscall.
Buffers not being filled completely could possibly happen when stopping
a stream with a half-filled buffer, if we want allow that in the spec.
Another possible case would be compressed streams with variable bitrate,
where the number of bytes needed for a millisecond of sound data isn't
fixed.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-20 14:24 ` Anton Yakovlev
2019-11-21 9:04 ` Gerd Hoffmann
@ 2019-11-25 16:50 ` Liam Girdwood
1 sibling, 0 replies; 42+ messages in thread
From: Liam Girdwood @ 2019-11-25 16:50 UTC (permalink / raw)
To: Anton Yakovlev, Jean-Philippe Brucker
Cc: Mikhail Golubev, virtio-dev, Takashi Iwai, Mark Brown
On Wed, 2019-11-20 at 15:24 +0100, Anton Yakovlev wrote:
> this
> regard. We could say something like "you can activate zero-copy mode,
> but it's
> up to your HW, OS and luck", but it does not sound very nicely.
"luck" is a bit unfair here :) But in general it will boil down to
hypervisor, HW and OS config (which will be customised for the
application).
Liam
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <D68AC0D0-851B-4F87-BE0E-F49527B83E24@redhat.com>
@ 2019-11-28 11:42 ` Gerd Hoffmann
2019-12-02 13:28 ` Anton Yakovlev
[not found] ` <20191128121920.GB4210@sirena.org.uk>
2 siblings, 0 replies; 42+ messages in thread
From: Gerd Hoffmann @ 2019-11-28 11:42 UTC (permalink / raw)
To: Christophe de Dinechin
Cc: Anton Yakovlev, Mark Brown, Liam Girdwood, Jean-Philippe Brucker,
Mikhail Golubev, virtio-dev, Takashi Iwai
Hi,
> Is the intent that any virtio device has to support all the same formats?
No.
> So it looks to me like all the formats should be part of the spec, and some
> device capabilities should indicate which ones the device accepts.
There is a per-stream bitmask in device config indicating which formats
are supported.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-11-25 9:31 ` Gerd Hoffmann
@ 2019-12-02 13:06 ` Anton Yakovlev
2019-12-03 9:00 ` Gerd Hoffmann
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-02 13:06 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
Sorry for the late reply, I was not available for a week.
On 25.11.2019 10:31, Gerd Hoffmann wrote:
> Hi,
>
>>> Instead of PCM_OUTPUT and PCM_INPUT I would put the number of input and
>>> output streams into the device config space (and explicitly allow count
>>> being zero).
>>
>> There are several options for the number of streams:
>>
>> 1. The device can provide one input and/or output stream.
>> 2. The device can provide several input and/or output streams, and all of them
>> share the same capabilities.
>> 3. The device can provide several input and/or output streams, and they can
>> have different set of capabilities.
>>
>> Overall design depends on chosen option. Current draft is based on p.1. But we
>> have never discussed what is the best solution here.
>
> (3) looks most useful to me.
Then we need to refactor device configuration. I would propose to put into
configuration space only a total number of all available PCM streams and
introduce special control request to query per-stream configuration. A
response should contain a stream type (input/output) and all its capabilities.
>>> PCM_MSG -- I would drop the feature bit and make that mandatory, so we
>>> have a common baseline on which all drivers and devices can agree on.
>>
>> Then we need to inform device what "transport" will be in use (I assumed it
>> would be feature negotiation).
>
> Whenever other transports (i.e. via shared memory) are supported: yes,
> that should be a feature bit.
>
> Not sure about choosing the transport. If both msg (i.e. via virtqueue)
> and shared memory are available, does it make sense to allow the driver
> choose the transport each time it starts a stream?
Shared memory based transport in any case will require some additional
actions. For HOST_MEM case the driver will need to get an access to host
buffer somehow. In GUEST_MEM case the driver will need to provide a buffer for
the host.
At first sight, we could extend the set_params request with the transport_type
field and some additional information. For example, in case of GUEST_MEM the
request could be followed by a buffer sg-list. This way it will work like you
said: the driver chooses the transport each time it starts a stream.
>>> Right now this is defined for the pci transport only.
>>
>> And now we only declare a feature bit, right?
>
> Not fully sure what you mean here.
>
> Having the first revision define only the virtqueue transport, reserve a
> feature bit for the shared memory transport and hash out the details of
> that later is possible.
Yes, that's what I wanted to clarify.
>>>> Configuration space
>>>> -------------------
>>>>
>>>> We talked about PCM formats defined in ALSA. In Linux kernel v5.4-rc8 we have
>>>>
>>>> Do we really need all of them? And I skipped endianess, but should we care
>>>> about it as well?
>>>
>>> I'd keep the list as short as possible. The formats listed in the patch
>>> looked like a good start.
>>
>> It was proposed to have all non-compressed formats.
>>
>> And... do we care about cross-endian cases?
>
> I'd start with little endian formats only. virtio byte order is little
> endian, and the world is moving to little endian anyway. We can add big
> endian formats later should this turn out to be needed, but I don't
> expect that to be the case.
Great!
>
>>> I'd suggest bytes (and rename the field to period_bytes).
>
>>>> struct virtio_snd_pcm_status {
>>>> le32 status;
>>>> le32 actual_length;
>>>> };
>>>
>>> Hmm. Why actual_length? Do we want allow short transfers? Why?
>>> Wouldn't it be more useful to just fill the buffer completely?
>>
>> In current design we have no buffer size requirements. It means, that in
>> theory the device and the driver could have buffers with different sizes.
>
> Should we define that the (virtio) buffer size must be period_bytes then?
It will have no sense, since the driver chooses period_bytes at runtime. If we
gonna introduce any buffer constrains, it must be set by the device in a
stream configuration.
>> Also, the capture stream is a special case. Now we don't state explicitly
>> whether read request is blockable or not.
>
> The concept of "blockable" doesn't exist at that level. The driver
> submits buffers to the device, the device fills them and notifies the
> driver when the buffer is full. It simply doesn't work like a read(2)
> syscall.
But you described exactly "blockable" case: an I/O request is completed not
immediately but upon some condition (the buffer is full). In case of message-
based transport, both the device and the driver will have its own buffers. And
for capturing these buffers might be filled at different speed. For example,
in order to improve latency, the device could complete requests immediately
and fill in buffers with whatever it has at the moment.
> Buffers not being filled completely could possibly happen when stopping
> a stream with a half-filled buffer, if we want allow that in the spec.
>
> Another possible case would be compressed streams with variable bitrate,
> where the number of bytes needed for a millisecond of sound data isn't
> fixed.
Compressed streams are not part of PCM framework anyway.
> cheers,
> Gerd
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <D68AC0D0-851B-4F87-BE0E-F49527B83E24@redhat.com>
2019-11-28 11:42 ` Gerd Hoffmann
@ 2019-12-02 13:28 ` Anton Yakovlev
[not found] ` <20191128121920.GB4210@sirena.org.uk>
2 siblings, 0 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-02 13:28 UTC (permalink / raw)
To: Christophe de Dinechin
Cc: Mark Brown, Gerd Hoffmann, Liam Girdwood, Jean-Philippe Brucker,
Mikhail Golubev, virtio-dev, Takashi Iwai
Hi Christophe,
On 28.11.2019 10:37, Christophe de Dinechin wrote:
>
>
>> On 22 Nov 2019, at 15:06, Anton Yakovlev <anton.yakovlev@opensynergy.com> wrote:
>>
>> On 22.11.2019 13:35, Mark Brown wrote:
>>> On Thu, Nov 21, 2019 at 01:57:51PM +0100, Anton Yakovlev wrote:
>>>> SNDRV_PCM_FORMAT_MPEG
>>>> SNDRV_PCM_FORMAT_GSM
>>>> SNDRV_PCM_FORMAT_G723_24
>>>> SNDRV_PCM_FORMAT_G723_24_1B
>>>> SNDRV_PCM_FORMAT_G723_40
>>>> SNDRV_PCM_FORMAT_G723_40_1B
>>>> Do we really need all of them? And I skipped endianess, but should we care
>>>> about it as well?
>>> The compressed audio formats I quoted above (and possibly some others)
>>> are mostly redundant and better handled through something like the ALSA
>>> compressed audio interface but that's definitely a separate thing with
>>> different requirements.
>>
>> Then let's include all except these.
>
> Is the intent that any virtio device has to support all the same formats?
>
> If I’m talking to a hardware decoder, then a compressed format makes sense
> (there is no point in decompressing in software if the device’s very purpose
> is to do it in hardware). But then, maybe it supports one format but not another.
>
> So it looks to me like all the formats should be part of the spec, and some
> device capabilities should indicate which ones the device accepts.
We discussed only PCM-related things. Compressed offload could be as a feature
with its own configuration layout, format definitions, request types, maybe
virtqueue(s) and so on.
>
>
> Thanks
> Christophe
>
>>
>>
>> --
>> Anton Yakovlev
>> Senior Software Engineer
>>
>> OpenSynergy GmbH
>> Rotherstr. 20, 10245 Berlin
>>
>> Phone: +49 30 60 98 54 0
>> E-Mail: anton.yakovlev@opensynergy.com
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>>
>
>
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191128121920.GB4210@sirena.org.uk>
@ 2019-12-02 13:30 ` Anton Yakovlev
[not found] ` <20191202135519.GF1998@sirena.org.uk>
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-02 13:30 UTC (permalink / raw)
To: Mark Brown, Christophe de Dinechin
Cc: Gerd Hoffmann, Liam Girdwood, Jean-Philippe Brucker,
Mikhail Golubev, virtio-dev, Takashi Iwai
On 28.11.2019 13:19, Mark Brown wrote:
> On Thu, Nov 28, 2019 at 10:37:17AM +0100, Christophe de Dinechin wrote:
>
>> If I’m talking to a hardware decoder, then a compressed format makes sense
>> (there is no point in decompressing in software if the device’s very purpose
>> is to do it in hardware). But then, maybe it supports one format but not another.
>
> If you're talking about compressed as opposed to PCM audio here (you say
> decompression) I think you probably want a different spec there, the
> flow control stuff is quite different and you often need a lot more
> configuration for compressed formats.
Usually all such kind of things are tight together in one audio subsystem and
in one audio driver. I don't think it has a sense to separate all kind of
streams into different specs.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
www.opensynergy.com
Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
Geschäftsführer/Managing Director: Regis Adjamah
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-12-02 13:06 ` Anton Yakovlev
@ 2019-12-03 9:00 ` Gerd Hoffmann
2019-12-04 11:15 ` Anton Yakovlev
0 siblings, 1 reply; 42+ messages in thread
From: Gerd Hoffmann @ 2019-12-03 9:00 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
On Mon, Dec 02, 2019 at 02:06:53PM +0100, Anton Yakovlev wrote:
> Hi,
>
> Sorry for the late reply, I was not available for a week.
>
>
> On 25.11.2019 10:31, Gerd Hoffmann wrote:
> > Hi,
> >
> > > > Instead of PCM_OUTPUT and PCM_INPUT I would put the number of input and
> > > > output streams into the device config space (and explicitly allow count
> > > > being zero).
> > >
> > > There are several options for the number of streams:
> > >
> > > 1. The device can provide one input and/or output stream.
> > > 2. The device can provide several input and/or output streams, and all of them
> > > share the same capabilities.
> > > 3. The device can provide several input and/or output streams, and they can
> > > have different set of capabilities.
> > >
> > > Overall design depends on chosen option. Current draft is based on p.1. But we
> > > have never discussed what is the best solution here.
> >
> > (3) looks most useful to me.
>
> Then we need to refactor device configuration. I would propose to put into
> configuration space only a total number of all available PCM streams and
> introduce special control request to query per-stream configuration. A
> response should contain a stream type (input/output) and all its capabilities.
Yes, that will fine work too.
> > > > PCM_MSG -- I would drop the feature bit and make that mandatory, so we
> > > > have a common baseline on which all drivers and devices can agree on.
> > >
> > > Then we need to inform device what "transport" will be in use (I assumed it
> > > would be feature negotiation).
> >
> > Whenever other transports (i.e. via shared memory) are supported: yes,
> > that should be a feature bit.
> >
> > Not sure about choosing the transport. If both msg (i.e. via virtqueue)
> > and shared memory are available, does it make sense to allow the driver
> > choose the transport each time it starts a stream?
>
> Shared memory based transport in any case will require some additional
> actions. For HOST_MEM case the driver will need to get an access to host
> buffer somehow. In GUEST_MEM case the driver will need to provide a buffer for
> the host.
>
> At first sight, we could extend the set_params request with the
> transport_type field and some additional information.
Or have a per-transport set_params request command.
> For example, in case
> of GUEST_MEM the request could be followed by a buffer sg-list.
I'm not convinced guest_mem is that useful. host_mem allows to give the
guest access to the buffers used by the hosts sound hardware, which is
probably what you need if the MSG transport can't handle the latency
requirements you have.
> > > > > struct virtio_snd_pcm_status {
> > > > > le32 status;
> > > > > le32 actual_length;
> > > > > };
> > > >
> > > > Hmm. Why actual_length? Do we want allow short transfers? Why?
> > > > Wouldn't it be more useful to just fill the buffer completely?
> > >
> > > In current design we have no buffer size requirements. It means, that in
> > > theory the device and the driver could have buffers with different sizes.
> >
> > Should we define that the (virtio) buffer size must be period_bytes then?
>
> It will have no sense, since the driver chooses period_bytes at runtime.
Yep, the driver will choose period_bytes when starting a stream.
The driver will also queue the buffers (for the MSG transport),
so it should be no problem for the driver to make the two match.
> If we gonna introduce any buffer constrains, it must be set by the
> device in a stream configuration.
If we want allow the device specify min/max period_bytes which it can
handle, then yes, that should go into the stream configuration.
Or we use negotiation: driver asks for period_bytes in set-params, the
driver picks the closest period_bytes value it can handle and returns
that.
> > > Also, the capture stream is a special case. Now we don't state explicitly
> > > whether read request is blockable or not.
> >
> > The concept of "blockable" doesn't exist at that level. The driver
> > submits buffers to the device, the device fills them and notifies the
> > driver when the buffer is full. It simply doesn't work like a read(2)
> > syscall.
>
> But you described exactly "blockable" case: an I/O request is completed not
> immediately but upon some condition (the buffer is full). In case of message-
> based transport, both the device and the driver will have its own buffers.
Well, no. The device doesn't need any buffers, it can use the buffers
submitted by the driver. Typical workflow:
(1) The driver puts a bunch of empty buffers into the rx (record/read)
virtqueue (each being period_bytes in size).
(2) The driver starts recording.
(3) The device fills the first buffer with recorded sound data.
(4) When the buffer is full the device returns it to the driver,
takes the next from the virtqueue to continue recording.
(5) The driver takes the filled buffer and does whatever it wants do
with the data (typically pass on to the userspace app).
(6) The driver submits a new empty buffer to the virtqueue to make
sure the device doesn't run out of buffers.
So, it's not a "here is a buffer, fill it please", "here is the next,
..." ping pong game between driver and device. There is a queue with
multiple buffers instead, and the device fills them one by one.
Note: The device implementation can have additional buffering of course.
When using alsa the library records to its own private buffers, then
goes copy the data to the buffer provided by the application when you
call snd_pcm_readi(). But it is not required. The device
implementation could also pass on the empty guest buffers directly to
the host sound hardware (if host hardware and host api allow that).
> And
> for capturing these buffers might be filled at different speed. For example,
> in order to improve latency, the device could complete requests immediately
> and fill in buffers with whatever it has at the moment.
Latency obviously depends on period_bytes. If the driver cares about
latency it should simply work with lots of small buffers instead of a
few big ones.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
[not found] ` <20191202135519.GF1998@sirena.org.uk>
@ 2019-12-04 8:04 ` Anton Yakovlev
0 siblings, 0 replies; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-04 8:04 UTC (permalink / raw)
To: Mark Brown
Cc: Christophe de Dinechin, Gerd Hoffmann, Liam Girdwood,
Jean-Philippe Brucker, Mikhail Golubev, virtio-dev, Takashi Iwai
On 02.12.2019 14:55, Mark Brown wrote:
> On Mon, Dec 02, 2019 at 02:30:44PM +0100, Anton Yakovlev wrote:
>> On 28.11.2019 13:19, Mark Brown wrote:
>
>>> If you're talking about compressed as opposed to PCM audio here (you say
>>> decompression) I think you probably want a different spec there, the
>>> flow control stuff is quite different and you often need a lot more
>>> configuration for compressed formats.
>
>> Usually all such kind of things are tight together in one audio subsystem and
>> in one audio driver. I don't think it has a sense to separate all kind of
>> streams into different specs.
>
> All the Linux based systems split the data paths, and at the
> hardware level unless you're using a message based system (which
> is basically just software anyway) you need a different interface
> as you can't push data through the system at a constant rate
> which is a central assumption for PCM audio.
Are we going to design separated specs for different audio features? It seems
a little bit of overkill.
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-12-03 9:00 ` Gerd Hoffmann
@ 2019-12-04 11:15 ` Anton Yakovlev
2019-12-04 12:52 ` Gerd Hoffmann
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-04 11:15 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
On 03.12.2019 10:00, Gerd Hoffmann wrote:
...snip...
>>>>> PCM_MSG -- I would drop the feature bit and make that mandatory, so we
>>>>> have a common baseline on which all drivers and devices can agree on.
>>>>
>>>> Then we need to inform device what "transport" will be in use (I assumed it
>>>> would be feature negotiation).
>>>
>>> Whenever other transports (i.e. via shared memory) are supported: yes,
>>> that should be a feature bit.
>>>
>>> Not sure about choosing the transport. If both msg (i.e. via virtqueue)
>>> and shared memory are available, does it make sense to allow the driver
>>> choose the transport each time it starts a stream?
>>
>> Shared memory based transport in any case will require some additional
>> actions. For HOST_MEM case the driver will need to get an access to host
>> buffer somehow. In GUEST_MEM case the driver will need to provide a buffer for
>> the host.
>>
>> At first sight, we could extend the set_params request with the
>> transport_type field and some additional information.
>
> Or have a per-transport set_params request command.
Or, since now we decided to make a message-based transport as a default one,
at the moment we can go without explicit transport selection. Some additional
extensions could be done at the future, when buffer sharing mechanism will be
stabilized.
>> For example, in case
>> of GUEST_MEM the request could be followed by a buffer sg-list.
>
> I'm not convinced guest_mem is that useful. host_mem allows to give the
> guest access to the buffers used by the hosts sound hardware, which is
> probably what you need if the MSG transport can't handle the latency
> requirements you have.
Actually, it might be pretty useful.
If a device is not capable of sharing host memory with a guest but still
capable of using guest shared memory, then there's a good use case for that:
when a buffer is mapped into a user space application. It assumes, that the
driver is not involved in frame transfer at all (thus, it can not queue
buffers into a virtqueue and send notifications to the device). But if that
memory is shared with the device (as long as some piece of memory containing
an application position as well), then it's possible to implement quite
simple poller in the device. And it would be pretty aligned with common
software mixer workflow (like in PA or QEmu), where a mixer invokes client's
callbacks for reading/writing next piece of data. The device will need only
to check an application position and directly read from/write to a shared
buffer.
...snip...
>> If we gonna introduce any buffer constrains, it must be set by the
>> device in a stream configuration.
>
> If we want allow the device specify min/max period_bytes which it can
> handle, then yes, that should go into the stream configuration.
>
> Or we use negotiation: driver asks for period_bytes in set-params, the
> driver picks the closest period_bytes value it can handle and returns
> that.
As I said before, periods are not used everywhere. Also, even in ALSA such
kind of negotiations may be non trivial. I would propose to leave choosing the
period_bytes value up to the driver. We could add yet one mandatory field to
the set_params request - driver's buffer size. (If the driver wants to use
period notification feature, then buffer_bytes % period_bytes must be 0). If
the device has its own intermediate buffer of any kind, it's possible to
adjust this according to the buffer_bytes value (like making it's being no
smaller than the specified size and so on). This way we could resolve original
concerns regarding possible different buffer sizes.
>>>> Also, the capture stream is a special case. Now we don't state explicitly
>>>> whether read request is blockable or not.
>>>
>>> The concept of "blockable" doesn't exist at that level. The driver
>>> submits buffers to the device, the device fills them and notifies the
>>> driver when the buffer is full. It simply doesn't work like a read(2)
>>> syscall.
>>
>> But you described exactly "blockable" case: an I/O request is completed not
>> immediately but upon some condition (the buffer is full). In case of message-
>> based transport, both the device and the driver will have its own buffers.
>
> Well, no. The device doesn't need any buffers, it can use the buffers
> submitted by the driver. Typical workflow:
>
> (1) The driver puts a bunch of empty buffers into the rx (record/read)
> virtqueue (each being period_bytes in size).
> (2) The driver starts recording.
> (3) The device fills the first buffer with recorded sound data.
> (4) When the buffer is full the device returns it to the driver,
> takes the next from the virtqueue to continue recording.
> (5) The driver takes the filled buffer and does whatever it wants do
> with the data (typically pass on to the userspace app).
> (6) The driver submits a new empty buffer to the virtqueue to make
> sure the device doesn't run out of buffers.
>
> So, it's not a "here is a buffer, fill it please", "here is the next,
> ..." ping pong game between driver and device. There is a queue with
> multiple buffers instead, and the device fills them one by one.
Then, should we make this pattern to be mandatory?
...snip...
>
>> And
>> for capturing these buffers might be filled at different speed. For example,
>> in order to improve latency, the device could complete requests immediately
>> and fill in buffers with whatever it has at the moment.
>
> Latency obviously depends on period_bytes. If the driver cares about
> latency it should simply work with lots of small buffers instead of a
> few big ones.
Well, smaller buffers would mean higher rate of hypercalls/traps but better
latency. And larger buffers would mean less amount of hypercalls/traps but
worse latency. There's always a trade-off and an implementer might choose
whatever fits better. I mean, this idea was behind previous design version
(where we could use the actual_length field for supporting all possible
cases).
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-12-04 11:15 ` Anton Yakovlev
@ 2019-12-04 12:52 ` Gerd Hoffmann
2019-12-05 12:06 ` Anton Yakovlev
0 siblings, 1 reply; 42+ messages in thread
From: Gerd Hoffmann @ 2019-12-04 12:52 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
> > > For example, in case
> > > of GUEST_MEM the request could be followed by a buffer sg-list.
> >
> > I'm not convinced guest_mem is that useful. host_mem allows to give the
> > guest access to the buffers used by the hosts sound hardware, which is
> > probably what you need if the MSG transport can't handle the latency
> > requirements you have.
>
> Actually, it might be pretty useful.
>
> If a device is not capable of sharing host memory with a guest but still
> capable of using guest shared memory, then there's a good use case for that:
Note that "hostmem" only means that the buffer is allocated by the host
(i.e. the device). Whenever the allocation is backed by actual host
sound buffers or just normal ram depends on the device implementation
and maybe the host sound hardware capabilities. Qemu would probably use
normal ram due to the extra indirection caused by the sound backend
interface.
I don't think there is a use case which guestmem can handle but hostmem
can not.
> when a buffer is mapped into a user space application. It assumes, that the
> driver is not involved in frame transfer at all (thus, it can not queue
> buffers into a virtqueue and send notifications to the device).
The sound data can actually be in userspace mapped buffers even for the
message transport. Adding a buffer to the virtqueue (which actually
stores pointer(s) to the buffer pages) effectively moves the application
position then. But, yes, userspace must do a syscall for that which is
not needed when updating the position in a shared page.
> But if that
> memory is shared with the device (as long as some piece of memory containing
> an application position as well), then it's possible to implement quite
> simple poller in the device. And it would be pretty aligned with common
> software mixer workflow (like in PA or QEmu), where a mixer invokes client's
> callbacks for reading/writing next piece of data. The device will need only
> to check an application position and directly read from/write to a shared
> buffer.
Note that it is possible to store things in the virtqueue without
notifying the device and expect the device poll the virtqueue instead.
Likewise for the other direction. That allows operating the queue
without vmexits and might work more efficient with lots of small
buffers. Of course device and driver must negotiate whenever they want
use polling or notifications.
> > > If we gonna introduce any buffer constrains, it must be set by the
> > > device in a stream configuration.
> >
> > If we want allow the device specify min/max period_bytes which it can
> > handle, then yes, that should go into the stream configuration.
> >
> > Or we use negotiation: driver asks for period_bytes in set-params, the
> > driver picks the closest period_bytes value it can handle and returns
> > that.
>
> As I said before, periods are not used everywhere. Also, even in ALSA such
> kind of negotiations may be non trivial. I would propose to leave choosing the
> period_bytes value up to the driver. We could add yet one mandatory field to
> the set_params request - driver's buffer size. (If the driver wants to use
> period notification feature, then buffer_bytes % period_bytes must be 0).
Sounds good to me.
> > > But you described exactly "blockable" case: an I/O request is completed not
> > > immediately but upon some condition (the buffer is full). In case of message-
> > > based transport, both the device and the driver will have its own buffers.
> >
> > Well, no. The device doesn't need any buffers, it can use the buffers
> > submitted by the driver. Typical workflow:
> >
> > (1) The driver puts a bunch of empty buffers into the rx (record/read)
> > virtqueue (each being period_bytes in size).
> > (2) The driver starts recording.
> > (3) The device fills the first buffer with recorded sound data.
> > (4) When the buffer is full the device returns it to the driver,
> > takes the next from the virtqueue to continue recording.
> > (5) The driver takes the filled buffer and does whatever it wants do
> > with the data (typically pass on to the userspace app).
> > (6) The driver submits a new empty buffer to the virtqueue to make
> > sure the device doesn't run out of buffers.
> >
> > So, it's not a "here is a buffer, fill it please", "here is the next,
> > ..." ping pong game between driver and device. There is a queue with
> > multiple buffers instead, and the device fills them one by one.
>
> Then, should we make this pattern to be mandatory?
It's pretty standard for virtio. The network receive queue works
fundamentally the same way for example, except that network buffers
actually might be half-filled only because you get network packets
where you don't know the size beforehand instead of a constant stream
of sound samples which you can easily pack into buffers as you like.
But, yes, it probably makes sense to explicitly say so in the specs.
If in doubt be more verbose ;)
> > > And
> > > for capturing these buffers might be filled at different speed. For example,
> > > in order to improve latency, the device could complete requests immediately
> > > and fill in buffers with whatever it has at the moment.
> >
> > Latency obviously depends on period_bytes. If the driver cares about
> > latency it should simply work with lots of small buffers instead of a
> > few big ones.
>
> Well, smaller buffers would mean higher rate of hypercalls/traps but better
> latency. And larger buffers would mean less amount of hypercalls/traps but
> worse latency.
Note that it depends on the use case whenever long latencies are
actually worse. Some use cases don't care much (music playback). For
some use cases it is good enough to know the exact latency so you can
take it into account for synchronization (video playback). Some
usecases are pretty much impossible with long latencies (games, voip).
> There's always a trade-off and an implementer might choose
> whatever fits better.
Sure. But you choose once, when configuring the stream, not for each
individual buffer, right? So the driver will:
(1) set-params (including period_bytes), taking into account what
userspace asked for.
(2) queue buffers for the stream, each being period_bytes in size.
(3) start stream.
> I mean, this idea was behind previous design version
> (where we could use the actual_length field for supporting all possible
> cases).
See above, you can do that without actual_length because the buffer size
is an element of the stream configuration.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-12-04 12:52 ` Gerd Hoffmann
@ 2019-12-05 12:06 ` Anton Yakovlev
2019-12-06 8:31 ` Gerd Hoffmann
0 siblings, 1 reply; 42+ messages in thread
From: Anton Yakovlev @ 2019-12-05 12:06 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
On 04.12.2019 13:52, Gerd Hoffmann wrote:
>>>> For example, in case
>>>> of GUEST_MEM the request could be followed by a buffer sg-list.
>>>
>>> I'm not convinced guest_mem is that useful. host_mem allows to give the
>>> guest access to the buffers used by the hosts sound hardware, which is
>>> probably what you need if the MSG transport can't handle the latency
>>> requirements you have.
>>
>> Actually, it might be pretty useful.
>>
>> If a device is not capable of sharing host memory with a guest but still
>> capable of using guest shared memory, then there's a good use case for that:
>
> Note that "hostmem" only means that the buffer is allocated by the host
> (i.e. the device). Whenever the allocation is backed by actual host
> sound buffers or just normal ram depends on the device implementation
> and maybe the host sound hardware capabilities. Qemu would probably use
> normal ram due to the extra indirection caused by the sound backend
> interface.
>
> I don't think there is a use case which guestmem can handle but hostmem
> can not.
There may be a special security requirement like "a guest must not have any
access to a host memory" (especially in case of type 1 hv).
>> when a buffer is mapped into a user space application. It assumes, that the
>> driver is not involved in frame transfer at all (thus, it can not queue
>> buffers into a virtqueue and send notifications to the device).
>
> The sound data can actually be in userspace mapped buffers even for the
> message transport. Adding a buffer to the virtqueue (which actually
> stores pointer(s) to the buffer pages) effectively moves the application
> position then.
It's how we implemented a PoC for the previous spec version.
> But, yes, userspace must do a syscall for that which is not needed when
> updating the position in a shared page.
Yes, that's the trickiest part.
Also, sharing memory implies sharing of two things: buffer itself and a page
containing position counter. And it goes in both directions (a host shares
buffer and hardware position or a guest shares buffer and application
position). And it seems, that overall solution can benefit if there will be
four independent entities. For example, message-based transport can have
some benefits, if a host will provide its hardware pointer in a shared page.
>> But if that
>> memory is shared with the device (as long as some piece of memory containing
>> an application position as well), then it's possible to implement quite
>> simple poller in the device. And it would be pretty aligned with common
>> software mixer workflow (like in PA or QEmu), where a mixer invokes client's
>> callbacks for reading/writing next piece of data. The device will need only
>> to check an application position and directly read from/write to a shared
>> buffer.
>
> Note that it is possible to store things in the virtqueue without
> notifying the device and expect the device poll the virtqueue instead.
> Likewise for the other direction. That allows operating the queue
> without vmexits and might work more efficient with lots of small
> buffers. Of course device and driver must negotiate whenever they want
> use polling or notifications.
Yes, and a shared memory only makes things simpler here.
>>>> But you described exactly "blockable" case: an I/O request is completed not
>>>> immediately but upon some condition (the buffer is full). In case of message-
>>>> based transport, both the device and the driver will have its own buffers.
>>>
>>> Well, no. The device doesn't need any buffers, it can use the buffers
>>> submitted by the driver. Typical workflow:
>>>
>>> (1) The driver puts a bunch of empty buffers into the rx (record/read)
>>> virtqueue (each being period_bytes in size).
>>> (2) The driver starts recording.
>>> (3) The device fills the first buffer with recorded sound data.
>>> (4) When the buffer is full the device returns it to the driver,
>>> takes the next from the virtqueue to continue recording.
>>> (5) The driver takes the filled buffer and does whatever it wants do
>>> with the data (typically pass on to the userspace app).
>>> (6) The driver submits a new empty buffer to the virtqueue to make
>>> sure the device doesn't run out of buffers.
>>>
>>> So, it's not a "here is a buffer, fill it please", "here is the next,
>>> ..." ping pong game between driver and device. There is a queue with
>>> multiple buffers instead, and the device fills them one by one.
>>
>> Then, should we make this pattern to be mandatory?
>
> It's pretty standard for virtio. The network receive queue works
> fundamentally the same way for example, except that network buffers
> actually might be half-filled only because you get network packets
> where you don't know the size beforehand instead of a constant stream
> of sound samples which you can easily pack into buffers as you like.
Except the end of the capture stream case. Can we use the len field in
virtq_used_elem for storing actual captured size then?
> But, yes, it probably makes sense to explicitly say so in the specs.
> If in doubt be more verbose ;)
I think it would be the best decision here. Such pattern may be a common thing
in virtio world, but people still might have their own interpretations.
Especially, if it's their first experience with implementing something.
>>>> And
>>>> for capturing these buffers might be filled at different speed. For example,
>>>> in order to improve latency, the device could complete requests immediately
>>>> and fill in buffers with whatever it has at the moment.
>>>
>>> Latency obviously depends on period_bytes. If the driver cares about
>>> latency it should simply work with lots of small buffers instead of a
>>> few big ones.
>>
>> Well, smaller buffers would mean higher rate of hypercalls/traps but better
>> latency. And larger buffers would mean less amount of hypercalls/traps but
>> worse latency.
>
> Note that it depends on the use case whenever long latencies are
> actually worse. Some use cases don't care much (music playback). For
> some use cases it is good enough to know the exact latency so you can
> take it into account for synchronization (video playback). Some
> usecases are pretty much impossible with long latencies (games, voip).
By the way, there is a good point here. Maybe it makes sense for the device
somehow reports its current/additionally latency value? Like, as a part of a
response to the set_params request?
>> There's always a trade-off and an implementer might choose
>> whatever fits better.
>
> Sure. But you choose once, when configuring the stream, not for each
> individual buffer, right? So the driver will:
>
> (1) set-params (including period_bytes), taking into account what
> userspace asked for.
> (2) queue buffers for the stream, each being period_bytes in size.
> (3) start stream.
Originally, we introduced period_bytes for enabling period notification
feature (which is optional). At least, it was not supposed to be used for
slicing a buffer into pieces (although, the driver may chose to work in this
way).
--
Anton Yakovlev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
Phone: +49 30 60 98 54 0
E-Mail: anton.yakovlev@opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [virtio-dev] [PATCH] snd: Add virtio sound device specification
2019-12-05 12:06 ` Anton Yakovlev
@ 2019-12-06 8:31 ` Gerd Hoffmann
0 siblings, 0 replies; 42+ messages in thread
From: Gerd Hoffmann @ 2019-12-06 8:31 UTC (permalink / raw)
To: Anton Yakovlev
Cc: Liam Girdwood, Jean-Philippe Brucker, Mikhail Golubev,
virtio-dev, Takashi Iwai, Mark Brown
Hi,
> > I don't think there is a use case which guestmem can handle but hostmem
> > can not.
>
> There may be a special security requirement like "a guest must not have any
> access to a host memory" (especially in case of type 1 hv).
Hmm, ok. Point.
> Also, sharing memory implies sharing of two things: buffer itself and a page
> containing position counter. And it goes in both directions (a host shares
> buffer and hardware position or a guest shares buffer and application
> position). And it seems, that overall solution can benefit if there will be
> four independent entities. For example, message-based transport can have
> some benefits, if a host will provide its hardware pointer in a shared page.
Hmm, I'm not sure mixing things is a good idea. When using shared
memory I would use it for both data and pointer.
> > > > So, it's not a "here is a buffer, fill it please", "here is the next,
> > > > ..." ping pong game between driver and device. There is a queue with
> > > > multiple buffers instead, and the device fills them one by one.
> > >
> > > Then, should we make this pattern to be mandatory?
> >
> > It's pretty standard for virtio. The network receive queue works
> > fundamentally the same way for example, except that network buffers
> > actually might be half-filled only because you get network packets
> > where you don't know the size beforehand instead of a constant stream
> > of sound samples which you can easily pack into buffers as you like.
>
> Except the end of the capture stream case. Can we use the len field in
> virtq_used_elem for storing actual captured size then?
Yes.
> > Note that it depends on the use case whenever long latencies are
> > actually worse. Some use cases don't care much (music playback). For
> > some use cases it is good enough to know the exact latency so you can
> > take it into account for synchronization (video playback). Some
> > usecases are pretty much impossible with long latencies (games, voip).
>
> By the way, there is a good point here. Maybe it makes sense for the device
> somehow reports its current/additionally latency value? Like, as a part of a
> response to the set_params request?
You mean for latencies additional to the ones implied by buffer size?
Yes, that sounds useful.
> > > There's always a trade-off and an implementer might choose
> > > whatever fits better.
> >
> > Sure. But you choose once, when configuring the stream, not for each
> > individual buffer, right? So the driver will:
> >
> > (1) set-params (including period_bytes), taking into account what
> > userspace asked for.
> > (2) queue buffers for the stream, each being period_bytes in size.
> > (3) start stream.
>
> Originally, we introduced period_bytes for enabling period notification
> feature (which is optional). At least, it was not supposed to be used for
> slicing a buffer into pieces (although, the driver may chose to work in this
> way).
Ah, I've missed the detail that you've decoupled buffers and period
notification. Guess we talked past each other because of that. So I'm
trying to outline things again, just to be sure we are on the same page:
The driver picks total buffer size (say 16k for example) and period size
(say 2k). The driver allocates 8 virtio buffers with 2k each.
Allocating one big buffer and slice it works too of course. Then the
driver goes queue up these 8 virtio buffers to the virtqueue. The
device returns the filled virtio buffers to the driver. Then you don't
separate period notification in the first place, you can simply use the
buffer completion notifications for that.
cheers,
Gerd
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 42+ messages in thread
end of thread, other threads:[~2019-12-06 8:32 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-24 14:43 [virtio-dev] [PATCH] snd: Add virtio sound device specification Mikhail Golubev
2019-10-28 16:05 ` Liam Girdwood
2019-10-29 9:42 ` Anton Yakovlev
2019-10-29 10:14 ` Anton Yakovlev
[not found] ` <20191029121810.GB5253@sirena.co.uk>
2019-10-29 13:16 ` Anton Yakovlev
[not found] ` <20191030121137.GC6693@sirena.co.uk>
2019-11-01 13:37 ` Anton Yakovlev
[not found] ` <20191111193903.GE4264@sirena.co.uk>
2019-11-13 12:01 ` Anton Yakovlev
[not found] ` <20191114212940.GC4664@sirena.co.uk>
2019-11-19 9:26 ` Anton Yakovlev
2019-11-11 15:20 ` Liam Girdwood
2019-11-12 11:09 ` Jean-Philippe Brucker
2019-11-12 14:20 ` Liam Girdwood
2019-11-12 16:05 ` Jean-Philippe Brucker
2019-11-12 18:02 ` Liam Girdwood
2019-11-12 18:47 ` Matti Moell
2019-11-19 15:23 ` Liam Girdwood
2019-11-13 9:44 ` Anton Yakovlev
2019-11-19 16:09 ` Liam Girdwood
2019-11-20 14:24 ` Anton Yakovlev
2019-11-21 9:04 ` Gerd Hoffmann
2019-11-21 12:57 ` Anton Yakovlev
2019-11-22 7:19 ` Gerd Hoffmann
2019-11-22 13:46 ` Anton Yakovlev
2019-11-25 9:31 ` Gerd Hoffmann
2019-12-02 13:06 ` Anton Yakovlev
2019-12-03 9:00 ` Gerd Hoffmann
2019-12-04 11:15 ` Anton Yakovlev
2019-12-04 12:52 ` Gerd Hoffmann
2019-12-05 12:06 ` Anton Yakovlev
2019-12-06 8:31 ` Gerd Hoffmann
[not found] ` <20191122123521.GB6849@sirena.org.uk>
2019-11-22 14:06 ` Anton Yakovlev
[not found] ` <D68AC0D0-851B-4F87-BE0E-F49527B83E24@redhat.com>
2019-11-28 11:42 ` Gerd Hoffmann
2019-12-02 13:28 ` Anton Yakovlev
[not found] ` <20191128121920.GB4210@sirena.org.uk>
2019-12-02 13:30 ` Anton Yakovlev
[not found] ` <20191202135519.GF1998@sirena.org.uk>
2019-12-04 8:04 ` Anton Yakovlev
2019-11-25 16:50 ` Liam Girdwood
2019-11-13 7:54 ` Anton Yakovlev
2019-11-12 12:45 ` Anton Yakovlev
2019-11-12 15:16 ` Liam Girdwood
2019-11-13 9:05 ` Anton Yakovlev
2019-11-19 15:49 ` Liam Girdwood
2019-11-20 9:32 ` Gerd Hoffmann
[not found] ` <20191028192952.GI5015@sirena.co.uk>
2019-10-29 10:46 ` Anton Yakovlev
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.