* [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
@ 2022-06-01 7:33 Alvaro Karsz
2022-06-02 15:33 ` Cornelia Huck
2022-06-09 6:50 ` [virtio-comment] " Jason Wang
0 siblings, 2 replies; 7+ messages in thread
From: Alvaro Karsz @ 2022-06-01 7:33 UTC (permalink / raw)
To: virtio-comment, jasowang; +Cc: rabeeh, Alvaro Karsz
Control a network device notifications coalescing parameters using the control virtqueue.
A new control class was added: VIRTIO_NET_CTRL_NOTF_COAL.
This class provides 2 commands:
- VIRTIO_NET_CTRL_NOTF_COAL_TX_SET:
Ask the network device to change the tx_usecs and tx_max_buffers parameters.
- tx_usecs: Time to delay a notification after a TX buffer is used in microseconds.
- tx_max_buffers: Number of TX buffers to delay a notification after a TX buffer is used.
- VIRTIO_NET_CTRL_NOTF_COAL_RX_SET:
Ask the network device to change the rx_usecs and rx_max_buffers parameters.
- rx_usecs: Time to delay a notification after a RX buffer is used in microseconds.
- rx_max_buffers: Number of RX buffers to delay a notification after a RX buffer is used.
--
v2:
- Usage of notification terminology.
- Add a few lines on what device should do when driver asked to
suppress notifications.
v3:
- Remove whitespaces.
- Explain with examples how the device should act.
v4:
- Example of a scenarion when VIRTIO_F_EVENT_IDX is negotiated.
- Usage of separate commands for RX coalescing and TX coalescing.
--
Signed-off-by: Alvaro Karsz <alvaro.karsz@solid-run.com>
---
content.tex | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 88 insertions(+)
diff --git a/content.tex b/content.tex
index 7508dd1..2c75056 100644
--- a/content.tex
+++ b/content.tex
@@ -3084,6 +3084,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
channel.
+\item[VIRTIO_NET_F_NOTF_COAL(55)] Device supports notifications coalescing.
+
\item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
(fragmenting the packet) the USO splits large UDP packet
to several segments when each of these smaller packets has UDP header.
@@ -3129,6 +3131,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
\item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
\end{description}
@@ -4464,6 +4467,91 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
(necessarily when not using the legacy interface) little-endian.
+\paragraph{Notifications Coalescing}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
+
+If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can
+send control commands for dynamically changing the coalescing parameters.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_coal_rx {
+ le32 rx_max_buffers;
+ le32 rx_usecs;
+};
+
+struct virtio_net_ctrl_coal_tx {
+ le32 tx_max_buffers;
+ le32 tx_usecs;
+};
+
+#define VIRTIO_NET_CTRL_NOTF_COAL 6
+ #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0
+ #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1
+\end{lstlisting}
+
+Coalescing parameters:
+\begin{itemize}
+\item rx_usecs: Time to delay a notification after a RX buffer is used in microseconds.
+\item tx_usecs: Time to delay a notification after a TX buffer is used in microseconds.
+\item rx_max_buffers: Number of RX buffers to delay a notification after a RX buffer is used.
+\item tx_max_buffers: Number of TX buffers to delay a notification after a TX buffer is used.
+\end{itemize}
+
+Note: A chain of buffers counts as one, just like it would if this feature is not negotiated.
+
+The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands:
+\begin{itemize}
+\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the tx_usecs and tx_max_buffers parameters.
+\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the rx_usecs and rx_max_buffers parameters.
+\end{itemize}
+
+Note: rx_usecs = 0 or rx_max_buffers = 0 means that a notification should be sent for every used buffer, so if one of these values is 0, the other value is meaningless.
+The same is true for tx_usecs and tx_max_buffers.
+
+\paragraph{RX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Notifications}
+
+If, for example, rx_usecs = 10 and rx_buffers_max = 15.
+
+\begin{itemize}
+\item The device will count used buffers from the RX virtqueue until it accumulates 15, or until 10 usecs elapsed since the usage of the first one.
+\item The device will send a notification to the driver, only if the driver hasn't suppressed notifications.
+\item The device will reset its internal coalescing counters, if a notification was sent.
+\end{itemize}
+
+\paragraph{TX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / TX Notifications}
+
+If, for example, tx_usecs = 10 and tx_buffers_max = 15.
+
+\begin{itemize}
+\item The device will count used buffers from the TX virtqueue until it accumulates 15, or until 10 usecs elapsed since the usage of the first one.
+\item The device will send a notification to the driver, only if the driver hasn't suppressed notifications.
+\item The device will reset its internal coalescing counters, if a notification was sent.
+\end{itemize}
+
+\paragraph{Coalescing and Notifications Suppression}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Coalescing and Notifications Suppression}
+
+If the driver sets the VIRTQ_AVAIL_F_NO_INTERRUPT flag, the device should not send notifications to the driver. \\
+In this case, the device will increase the coalescing counters until the flag is cleared, and only then the device will send a notification and clear the coalescing counters. \\ \\
+
+If the VIRTIO_F_EVENT_IDX feature is negotiated, and used_event field is set, the device will send a notification only after used_event index is reached, even if the coalescing counters expired before.
+
+For example: \\
+rx_usecs = 10, rx_max_buffers = 7, used_event = 5, and current virtqueue location is 0: \\
+If the timer elapses before the \#5 buffer is used, the device will send a notification only after the \#5 buffer is used. \\
+If the \#5 buffer is used before the timer elapses, the device will send a notification after the \#7 buffer is used, or after the timer elapses, whichever occurs first.
+
+\drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
+
+
+If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands.
+
+\devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
+
+A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if was not able to change the parameters.\\ \\
+A device MUST NOT accept tx_buffers_max/rx_buffers_max values bigger than the virtqueue size.\\ \\
+A device SHOULD NOT send notifications to the driver, if the notifications are suppressed.\\ \\
+A device SHOULD initialize all coalescing values to 0, meaning that a notification is sent after every used buffer (if events aren't suppressed). \\ \\
+
+
\subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
Types / Network Device / Legacy Interface: Framing Requirements}
--
2.32.0
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-01 7:33 [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature Alvaro Karsz
@ 2022-06-02 15:33 ` Cornelia Huck
2022-06-02 15:52 ` Alvaro Karsz
2022-06-09 6:50 ` [virtio-comment] " Jason Wang
1 sibling, 1 reply; 7+ messages in thread
From: Cornelia Huck @ 2022-06-02 15:33 UTC (permalink / raw)
To: Alvaro Karsz, virtio-comment, jasowang; +Cc: rabeeh, Alvaro Karsz
On Wed, Jun 01 2022, Alvaro Karsz <alvaro.karsz@solid-run.com> wrote:
> Control a network device notifications coalescing parameters using the control virtqueue.
> A new control class was added: VIRTIO_NET_CTRL_NOTF_COAL.
>
> This class provides 2 commands:
> - VIRTIO_NET_CTRL_NOTF_COAL_TX_SET:
> Ask the network device to change the tx_usecs and tx_max_buffers parameters.
> - tx_usecs: Time to delay a notification after a TX buffer is used in microseconds.
> - tx_max_buffers: Number of TX buffers to delay a notification after a TX buffer is used.
>
> - VIRTIO_NET_CTRL_NOTF_COAL_RX_SET:
> Ask the network device to change the rx_usecs and rx_max_buffers parameters.
> - rx_usecs: Time to delay a notification after a RX buffer is used in microseconds.
> - rx_max_buffers: Number of RX buffers to delay a notification after a RX buffer is used.
> --
> v2:
> - Usage of notification terminology.
> - Add a few lines on what device should do when driver asked to
> suppress notifications.
>
> v3:
> - Remove whitespaces.
> - Explain with examples how the device should act.
>
> v4:
> - Example of a scenarion when VIRTIO_F_EVENT_IDX is negotiated.
> - Usage of separate commands for RX coalescing and TX coalescing.
> --
>
> Signed-off-by: Alvaro Karsz <alvaro.karsz@solid-run.com>
> ---
> content.tex | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 88 insertions(+)
>
(...)
> +\drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
> +
> +
> +If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands.
> +
> +\devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
> +
> +A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if was not able to change the parameters.\\ \\
> +A device MUST NOT accept tx_buffers_max/rx_buffers_max values bigger than the virtqueue size.\\ \\
> +A device SHOULD NOT send notifications to the driver, if the notifications are suppressed.\\ \\
> +A device SHOULD initialize all coalescing values to 0, meaning that a notification is sent after every used buffer (if events aren't suppressed). \\ \\
> +
> +
I think these are missing being referenced in conformance.tex?
[This seems to be easy to miss, I also noticed missing statements while
preparing the changelong for 1.2. Should the build process maybe check
that conformance sections are hooked up properly?]
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-02 15:33 ` Cornelia Huck
@ 2022-06-02 15:52 ` Alvaro Karsz
0 siblings, 0 replies; 7+ messages in thread
From: Alvaro Karsz @ 2022-06-02 15:52 UTC (permalink / raw)
To: Cornelia Huck; +Cc: virtio-comment, Jason Wang, Rabeeh Khoury
> I think these are missing being referenced in conformance.tex?
Sorry, I forgot about conformance.tex.
I'll create a new version next week.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [virtio-comment] Re: [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-01 7:33 [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature Alvaro Karsz
2022-06-02 15:33 ` Cornelia Huck
@ 2022-06-09 6:50 ` Jason Wang
2022-06-09 7:16 ` Alvaro Karsz
1 sibling, 1 reply; 7+ messages in thread
From: Jason Wang @ 2022-06-09 6:50 UTC (permalink / raw)
To: Alvaro Karsz; +Cc: virtio-comment, Rabeeh Khoury
On Wed, Jun 1, 2022 at 3:33 PM Alvaro Karsz <alvaro.karsz@solid-run.com> wrote:
>
> Control a network device notifications coalescing parameters using the control virtqueue.
> A new control class was added: VIRTIO_NET_CTRL_NOTF_COAL.
>
> This class provides 2 commands:
> - VIRTIO_NET_CTRL_NOTF_COAL_TX_SET:
> Ask the network device to change the tx_usecs and tx_max_buffers parameters.
> - tx_usecs: Time to delay a notification after a TX buffer is used in microseconds.
> - tx_max_buffers: Number of TX buffers to delay a notification after a TX buffer is used.
>
> - VIRTIO_NET_CTRL_NOTF_COAL_RX_SET:
> Ask the network device to change the rx_usecs and rx_max_buffers parameters.
> - rx_usecs: Time to delay a notification after a RX buffer is used in microseconds.
> - rx_max_buffers: Number of RX buffers to delay a notification after a RX buffer is used.
> --
> v2:
> - Usage of notification terminology.
> - Add a few lines on what device should do when driver asked to
> suppress notifications.
>
> v3:
> - Remove whitespaces.
> - Explain with examples how the device should act.
>
> v4:
> - Example of a scenarion when VIRTIO_F_EVENT_IDX is negotiated.
> - Usage of separate commands for RX coalescing and TX coalescing.
> --
>
> Signed-off-by: Alvaro Karsz <alvaro.karsz@solid-run.com>
> ---
> content.tex | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 88 insertions(+)
>
> diff --git a/content.tex b/content.tex
> index 7508dd1..2c75056 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_NOTF_COAL(55)] Device supports notifications coalescing.
> +
> \item[VIRTIO_NET_F_HOST_USO (56)] Device can receive USO packets. Unlike UFO
> (fragmenting the packet) the USO splits large UDP packet
> to several segments when each of these smaller packets has UDP header.
> @@ -3129,6 +3131,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_GUEST_ANNOUNCE] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
> @@ -4464,6 +4467,91 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> (necessarily when not using the legacy interface) little-endian.
>
>
> +\paragraph{Notifications Coalescing}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
> +
> +If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can
> +send control commands for dynamically changing the coalescing parameters.
> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_coal_rx {
> + le32 rx_max_buffers;
> + le32 rx_usecs;
> +};
> +
> +struct virtio_net_ctrl_coal_tx {
> + le32 tx_max_buffers;
Considering RX may use multiple buffers per packet, I wonder if it's
better to stick to "packet" here.
> + le32 tx_usecs;
> +};
> +
> +#define VIRTIO_NET_CTRL_NOTF_COAL 6
> + #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0
> + #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1
> +\end{lstlisting}
> +
> +Coalescing parameters:
> +\begin{itemize}
> +\item rx_usecs: Time to delay a notification after a RX buffer is used in microseconds.
> +\item tx_usecs: Time to delay a notification after a TX buffer is used in microseconds.
> +\item rx_max_buffers: Number of RX buffers to delay a notification after a RX buffer is used.
Does this work like: if we set rx_max_buffers to 10, it means after 1
buffers is used, we still need another 10 buffers?
> +\item tx_max_buffers: Number of TX buffers to delay a notification after a TX buffer is used.
> +\end{itemize}
> +
> +Note: A chain of buffers counts as one, just like it would if this feature is not negotiated.
I think we should explicitly say mergeable rx buffers here. And that's
why I think mentioning "packet is sent/received" is probably better
than the buffer used here.
> +
> +The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands:
> +\begin{itemize}
> +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the tx_usecs and tx_max_buffers parameters.
> +\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the rx_usecs and rx_max_buffers parameters.
> +\end{itemize}
> +
> +Note: rx_usecs = 0 or rx_max_buffers = 0 means that a notification should be sent for every used buffer, so if one of these values is 0, the other value is meaningless.
> +The same is true for tx_usecs and tx_max_buffers.
It's better to clarify it in the part of coalescing parameters, or we
can just borrow the comments of kernel's ethtool_coalesce, then it
should explain itself like:
@rx_usecs: How many usecs to delay an RX notification after a packet arrives
@rx_max_buffers: Maximum number of packets to receive before an RX notification
Not a native speaker but then rx_usecs=0 it means the notification is
sent immediately and by saying "maximum number" it means best effort.
> +
> +\paragraph{RX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Notifications}
> +
> +If, for example, rx_usecs = 10 and rx_buffers_max = 15.
> +
> +\begin{itemize}
> +\item The device will count used buffers from the RX virtqueue until it accumulates 15, or until 10 usecs elapsed since the usage of the first one.
> +\item The device will send a notification to the driver, only if the driver hasn't suppressed notifications.
Let's clarify what happens if notification is suppressed.
> +\item The device will reset its internal coalescing counters, if a notification was sent.
I think there's probably no need to mention the implementation details
like counters here. The definition of rx_usecs and rx_max_buffers
should be sufficient.
> +\end{itemize}
> +
> +\paragraph{TX Notifications}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / TX Notifications}
> +
> +If, for example, tx_usecs = 10 and tx_buffers_max = 15.
> +
> +\begin{itemize}
> +\item The device will count used buffers from the TX virtqueue until it accumulates 15, or until 10 usecs elapsed since the usage of the first one.
> +\item The device will send a notification to the driver, only if the driver hasn't suppressed notifications.
> +\item The device will reset its internal coalescing counters, if a notification was sent.
> +\end{itemize}
> +
> +\paragraph{Coalescing and Notifications Suppression}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Coalescing and Notifications Suppression}
> +
> +If the driver sets the VIRTQ_AVAIL_F_NO_INTERRUPT flag, the device should not send notifications to the driver. \\
> +In this case, the device will increase the coalescing counters until the flag is cleared, and only then the device will send a notification and clear the coalescing counters. \\ \\
> +
> +If the VIRTIO_F_EVENT_IDX feature is negotiated, and used_event field is set, the device will send a notification only after used_event index is reached, even if the coalescing counters expired before.
> +
> +For example: \\
> +rx_usecs = 10, rx_max_buffers = 7, used_event = 5, and current virtqueue location is 0: \\
> +If the timer elapses before the \#5 buffer is used, the device will send a notification only after the \#5 buffer is used. \\
> +If the \#5 buffer is used before the timer elapses, the device will send a notification after the \#7 buffer is used, or after the timer elapses, whichever occurs first.
> +
> +\drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
> +
> +
> +If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands.
> +
> +\devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
> +
> +A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if was not able to change the parameters.\\ \\
> +A device MUST NOT accept tx_buffers_max/rx_buffers_max values bigger than the virtqueue size.\\ \\
What's the reason for this?
> +A device SHOULD NOT send notifications to the driver, if the notifications are suppressed.\\ \\
> +A device SHOULD initialize all coalescing values to 0, meaning that a notification is sent after every used buffer (if events aren't suppressed). \\ \\
I think we need use MUST here, something like:
Upon reset, the device must present all zero values ...
Thanks
> +
> +
> \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> Types / Network Device / Legacy Interface: Framing Requirements}
>
> --
> 2.32.0
>
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [virtio-comment] Re: [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-09 6:50 ` [virtio-comment] " Jason Wang
@ 2022-06-09 7:16 ` Alvaro Karsz
2022-06-09 7:30 ` Jason Wang
0 siblings, 1 reply; 7+ messages in thread
From: Alvaro Karsz @ 2022-06-09 7:16 UTC (permalink / raw)
To: Jason Wang; +Cc: virtio-comment, Rabeeh Khoury
> Considering RX may use multiple buffers per packet, I wonder if it's
> better to stick to "packet" here.
Noted.
> Does this work like: if we set rx_max_buffers to 10, it means after 1
> buffers is used, we still need another 10 buffers?
No, if rx_max_buffers=10, we should send a notification on the #10
packet. I will clarify that.
> I think we should explicitly say mergeable rx buffers here. And that's
> why I think mentioning "packet is sent/received" is probably better
> than the buffer used here.
I meant when the host writes a packet in several buffers, and the NEXT
flag is set on the descriptor.
But you're right, I should mention the mergeable rx buffers as well.
> It's better to clarify it in the part of coalescing parameters, or we
> can just borrow the comments of kernel's ethtool_coalesce, then it
> should explain itself like:
>
>
> @rx_usecs: How many usecs to delay an RX notification after a packet arrives
> @rx_max_buffers: Maximum number of packets to receive before an RX notification
>
>
> Not a native speaker but then rx_usecs=0 it means the notification is
> sent immediately and by saying "maximum number" it means best effort.
Noted. I will move the coalescing parameters explanation to this subparagraph.
> Let's clarify what happens if notification is suppressed.
I will add that in this case, the device should not send a notification.
> I think there's probably no need to mention the implementation details
> like counters here. The definition of rx_usecs and rx_max_buffers
> should be sufficient.
Noted.
> What's the reason for this?
Because if (tx/rx)_buffers_max is bigger than the virtqueue size, this
condition will never be met.
Do you think that we should allow this to happen?
> I think we need use MUST here, something like:
>
>
> Upon reset, the device must present all zero values ...
Noted.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [virtio-comment] Re: [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-09 7:16 ` Alvaro Karsz
@ 2022-06-09 7:30 ` Jason Wang
2022-06-09 7:34 ` Alvaro Karsz
0 siblings, 1 reply; 7+ messages in thread
From: Jason Wang @ 2022-06-09 7:30 UTC (permalink / raw)
To: Alvaro Karsz; +Cc: virtio-comment, Rabeeh Khoury
On Thu, Jun 9, 2022 at 3:17 PM Alvaro Karsz <alvaro.karsz@solid-run.com> wrote:
>
> > Considering RX may use multiple buffers per packet, I wonder if it's
> > better to stick to "packet" here.
>
> Noted.
>
> > Does this work like: if we set rx_max_buffers to 10, it means after 1
> > buffers is used, we still need another 10 buffers?
>
> No, if rx_max_buffers=10, we should send a notification on the #10
> packet. I will clarify that.
>
> > I think we should explicitly say mergeable rx buffers here. And that's
> > why I think mentioning "packet is sent/received" is probably better
> > than the buffer used here.
>
> I meant when the host writes a packet in several buffers, and the NEXT
> flag is set on the descriptor.
> But you're right, I should mention the mergeable rx buffers as well.
>
> > It's better to clarify it in the part of coalescing parameters, or we
> > can just borrow the comments of kernel's ethtool_coalesce, then it
> > should explain itself like:
> >
> >
> > @rx_usecs: How many usecs to delay an RX notification after a packet arrives
> > @rx_max_buffers: Maximum number of packets to receive before an RX notification
> >
> >
> > Not a native speaker but then rx_usecs=0 it means the notification is
> > sent immediately and by saying "maximum number" it means best effort.
>
> Noted. I will move the coalescing parameters explanation to this subparagraph.
>
> > Let's clarify what happens if notification is suppressed.
>
> I will add that in this case, the device should not send a notification.
Yes, and when the notification needs to be sent.
>
> > I think there's probably no need to mention the implementation details
> > like counters here. The definition of rx_usecs and rx_max_buffers
> > should be sufficient.
>
> Noted.
>
> > What's the reason for this?
>
> Because if (tx/rx)_buffers_max is bigger than the virtqueue size, this
> condition will never be met.
I don't understand here, the driver can poll for the used buffer actually.
> Do you think that we should allow this to happen?
Not sure but I just have a try with mlx4, it gives me:
# ethtool -g ens3d1
Ring parameters for ens3d1:
Pre-set maximums:
RX: 8192
RX Mini: n/a
RX Jumbo: n/a
TX: 8192
Current hardware settings:
RX: 1024
RX Mini: n/a
RX Jumbo: n/a
TX: 1024
# ethtool -c ens3d1
Coalesce parameters for ens3d1:
Adaptive RX: on TX: n/a
stats-block-usecs: n/a
sample-interval: 0
pkt-rate-low: 400000
pkt-rate-high: 450000
rx-usecs: 16
rx-frames: 2048
rx-usecs-irq: n/a
rx-frames-irq: n/a
tx-usecs: 16
tx-frames: 16
tx-usecs-irq: n/a
tx-frames-irq: 256
rx-usecs-low: 0
rx-frame-low: n/a
tx-usecs-low: n/a
tx-frame-low: n/a
rx-usecs-high: 128
rx-frame-high: n/a
tx-usecs-high: n/a
tx-frame-high: n/a
It means rx-frames could be bigger than RX.
Thanks
>
> > I think we need use MUST here, something like:
> >
> >
> > Upon reset, the device must present all zero values ...
>
> Noted.
>
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [virtio-comment] Re: [PATCH v4] Introduction of Virtio Network device notifications coalescing feature.
2022-06-09 7:30 ` Jason Wang
@ 2022-06-09 7:34 ` Alvaro Karsz
0 siblings, 0 replies; 7+ messages in thread
From: Alvaro Karsz @ 2022-06-09 7:34 UTC (permalink / raw)
To: Jason Wang; +Cc: virtio-comment, Rabeeh Khoury
> It means rx-frames could be bigger than RX.
Noted.
I will include all the changes in the new version.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-06-09 7:35 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-01 7:33 [virtio-comment] [PATCH v4] Introduction of Virtio Network device notifications coalescing feature Alvaro Karsz
2022-06-02 15:33 ` Cornelia Huck
2022-06-02 15:52 ` Alvaro Karsz
2022-06-09 6:50 ` [virtio-comment] " Jason Wang
2022-06-09 7:16 ` Alvaro Karsz
2022-06-09 7:30 ` Jason Wang
2022-06-09 7:34 ` Alvaro Karsz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.