* [PATCH v7] virtio-net: support inner header hash
@ 2023-01-04 7:14 Heng Qi
2023-01-04 12:34 ` [virtio-comment] " Heng Qi
` (5 more replies)
0 siblings, 6 replies; 54+ messages in thread
From: Heng Qi @ 2023-01-04 7:14 UTC (permalink / raw)
To: virtio-comment, virtio-dev
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
If the tunnel is used to encapsulate the packets, the hash calculated
using the outer header of the receive packets is always fixed for the
same flow packets, i.e. they will be steered to the same receive queue.
We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
in \field{hash_types}, which instructs the device to calculate the
hash using the inner headers of tunnel-encapsulated packets. Besides,
values in \field{hash_report_tunnel} are added to report tunnel types.
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
Reviewed-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
v6:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin
v5:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin
v4:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck
v3:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
introduction.tex | 19 +++++
2 files changed, 203 insertions(+), 7 deletions(-)
diff --git a/content.tex b/content.tex
index e863709..7845f6c 100644
--- a/content.tex
+++ b/content.tex
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
+
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
to several segments when each of these smaller packets has UDP header.
\item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
- value and a type of calculated hash.
+ value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
+ is negotiated, an encapsulation packet type.
\item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
\end{description}
\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+ u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
};
\end{lstlisting}
@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
A device attempts to calculate a per-packet hash in the following cases:
\begin{itemize}
\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
\end{itemize}
If the feature VIRTIO_NET_F_RSS was negotiated:
@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
\end{itemize}
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation type is set in \field{hash_types}, the hash for a specific type of
+encapsulated packet is calculated over the inner as opposed to outer header.
+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
+Supported/enabled hash types}.
+
+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
+
+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
+negotiated, the device calculates the hash over the outer header, and
+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
+is no longer valid.
+
\subparagraph{Supported/enabled hash types}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
Hash types applicable for IPv4 packets:
\begin{lstlisting}
#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
\end{lstlisting}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner
+header of the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
+\end{lstlisting}
+
\subparagraph{IPv4 packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
\end{itemize}
+\subparagraph{Inner payload of an encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_types}, the device calculates the
+inner header hash on an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
+
+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
+according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
+ TCPv4 header, the hash is calculated over the following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemsize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
+ inner UDPv4 header, the hash is calculated over the following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \end{itemsize}
+ \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet without an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
+ TCPv6 header, the hash is calculated over the following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemsize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
+ inner UDPv6 header, the hash is calculated over the following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \end{itemsize}
+ \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on the inner IPv6 packet with an extension header
+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
+\begin{itemsize}
+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
+ TCPv6 header, the hash is calculated over the following fields:
+ \begin{itemize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
+ UDPv6 header, the hash is calculated over the following fields:
+ \begin{itemsize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \end{itemize}
+ \item Else skip the inner IPv6 extension header and calculate the inner header hash as
+ defined for an encapsulated packet whose inner payload is an IPv6 packet without
+ an extension header.
+\end{itemsize}
+
\paragraph{Hash reporting for incoming packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
Possible values that the device can report in \field{hash_report} are defined below.
They correspond to supported hash types defined in
@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
\end{lstlisting}
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
+#define VIRTIO_NET_HASH_REPORT_GRE 1
+#define VIRTIO_NET_HASH_REPORT_VXLAN 2
+#define VIRTIO_NET_HASH_REPORT_GENEVE 3
+\end{lstlisting}
+
+They correspond to supported hash types defined in
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
\subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
diff --git a/introduction.tex b/introduction.tex
index 287c5fc..ff01a9b 100644
--- a/introduction.tex
+++ b/introduction.tex
@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
\phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+ \phantomsection\label{intro:GRE}\textbf{[GRE]} &
+ Generic Routing Encapsulation
+ \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+ \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+ Virtual eXtensible Local Area Network
+ \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+ \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+ Generic Network Virtualization Encapsulation
+ \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+ \phantomsection\label{intro:IP}\textbf{[IP]} &
+ INTERNET PROTOCOL
+ \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+ \phantomsection\label{intro:UDP}\textbf{[UDP]} &
+ User Datagram Protocol
+ \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+ \phantomsection\label{intro:TCP}\textbf{[TCP]} &
+ TRANSMISSION CONTROL PROTOCOL
+ \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
+
\end{longtable}
--
2.19.1.6.gb485710b
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [virtio-comment] [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
@ 2023-01-04 12:34 ` Heng Qi
2023-01-04 12:37 ` Michael S. Tsirkin
2023-01-06 5:27 ` Michael S. Tsirkin
` (4 subsequent siblings)
5 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-04 12:34 UTC (permalink / raw)
To: Michael S . Tsirkin, virtio-comment, virtio-dev
Cc: Jason Wang, Yuri Benditovich, Cornelia Huck, Xuan Zhuo
Hi Michael, do you have any other comments?
Thanks.
在 2023/1/4 下午3:14, Heng Qi 写道:
> If the tunnel is used to encapsulate the packets, the hash calculated
> using the outer header of the receive packets is always fixed for the
> same flow packets, i.e. they will be steered to the same receive queue.
>
> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> in \field{hash_types}, which instructs the device to calculate the
> hash using the inner headers of tunnel-encapsulated packets. Besides,
> values in \field{hash_report_tunnel} are added to report tunnel types.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v6:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> introduction.tex | 19 +++++
> 2 files changed, 203 insertions(+), 7 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..7845f6c 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> + is negotiated, an encapsulation packet type.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> +encapsulated packet is calculated over the inner as opposed to outer header.
> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> +Supported/enabled hash types}.
> +
> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> +
> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> +negotiated, the device calculates the hash over the outer header, and
> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> +is no longer valid.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner
> +header of the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payload of an encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_types}, the device calculates the
> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> +
> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> +according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> + TCPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> + inner UDPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet without an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> + inner UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet with an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> + UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> + an extension header.
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +They correspond to supported hash types defined in
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..ff01a9b 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> +
>
> \end{longtable}
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] [PATCH v7] virtio-net: support inner header hash
2023-01-04 12:34 ` [virtio-comment] " Heng Qi
@ 2023-01-04 12:37 ` Michael S. Tsirkin
0 siblings, 0 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-04 12:37 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 04, 2023 at 08:34:56PM +0800, Heng Qi wrote:
>
> Hi Michael, do you have any other comments?
>
> Thanks.
Sit tight pls, you only posted this 5 hours ago.
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
2023-01-04 12:34 ` [virtio-comment] " Heng Qi
@ 2023-01-06 5:27 ` Michael S. Tsirkin
2023-01-06 6:42 ` [virtio-comment] " Heng Qi
2023-01-09 11:36 ` Michael S. Tsirkin
` (3 subsequent siblings)
5 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-06 5:27 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> If the tunnel is used to encapsulate the packets, the hash calculated
> using the outer header of the receive packets is always fixed for the
> same flow packets, i.e. they will be steered to the same receive queue.
>
> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> in \field{hash_types}, which instructs the device to calculate the
> hash using the inner headers of tunnel-encapsulated packets. Besides,
> values in \field{hash_report_tunnel} are added to report tunnel types.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
ok close to being ready. a couple of minor comments.
> ---
> v6:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> introduction.tex | 19 +++++
> 2 files changed, 203 insertions(+), 7 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..7845f6c 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
I would probably drop the list of tunnel types here.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> + is negotiated, an encapsulation packet type.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> +encapsulated packet is calculated over the inner as opposed to outer header.
> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> +Supported/enabled hash types}.
> +
> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> +
> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> +negotiated, the device calculates the hash over the outer header, and
> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> +is no longer valid.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner
> +header of the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payload of an encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_types}, the device calculates the
> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> +
> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> +according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> + TCPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> + inner UDPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet without an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> + inner UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet with an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> + UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> + an extension header.
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
needs to fill -> fills
for consistency
> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +They correspond to supported hash types defined in
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..ff01a9b 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> +
>
> \end{longtable}
>
> --
> 2.19.1.6.gb485710b
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-06 5:27 ` Michael S. Tsirkin
@ 2023-01-06 6:42 ` Heng Qi
2023-01-06 6:59 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-06 6:42 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > If the tunnel is used to encapsulate the packets, the hash calculated
> > using the outer header of the receive packets is always fixed for the
> > same flow packets, i.e. they will be steered to the same receive queue.
> >
> > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > in \field{hash_types}, which instructs the device to calculate the
> > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > values in \field{hash_report_tunnel} are added to report tunnel types.
> >
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> >
> > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>
>
> ok close to being ready. a couple of minor comments.
>
> > ---
> > v6:
> > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > 2. Fix some syntax issues. @Michael S. Tsirkin
> >
> > v5:
> > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > 3. Move the links to introduction section. @Michael S. Tsirkin
> > 4. Clarify some sentences. @Michael S. Tsirkin
> >
> > v4:
> > 1. Clarify some paragraphs. @Cornelia Huck
> > 2. Fix the u8 type. @Cornelia Huck
> >
> > v3:
> > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> >
> > v2:
> > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> >
> > v1:
> > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > 2. Clarify some paragraphs. @Jason Wang
> > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> >
> > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > introduction.tex | 19 +++++
> > 2 files changed, 203 insertions(+), 7 deletions(-)
> >
> > diff --git a/content.tex b/content.tex
> > index e863709..7845f6c 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > channel.
> >
> > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
>
> I would probably drop the list of tunnel types here.
Do you mean to use "Device supports inner header hash for
tunnel-encapsulated packets." instead? Why? We do want to use this
feature bit to indicate that the device supports inner hashing of
GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
>
> > +
> > \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
> >
> > \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> > @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > to several segments when each of these smaller packets has UDP header.
> >
> > \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> > - value and a type of calculated hash.
> > + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> > + is negotiated, an encapsulation packet type.
> >
> > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> > value. Device benefits from knowing the exact header length.
> > @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> > +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> > \end{description}
> >
> > \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> > le16 num_buffers;
> > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> > + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > };
> > \end{lstlisting}
> >
> > @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > A device attempts to calculate a per-packet hash in the following cases:
> > \begin{itemize}
> > \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> > -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> > +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> > +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> > \end{itemize}
> >
> > If the feature VIRTIO_NET_F_RSS was negotiated:
> > @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> > \end{itemize}
> >
> > +\subparagraph{Tunnel/Encapsulated packet}
> > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> > +A tunnel packet is encapsulated from the original packet based on the tunneling
> > +protocol (only a single level of encapsulation is currently supported). The
> > +encapsulated packet contains an outer header and an inner header, and the device
> > +calculates the hash over either the inner header or the outer header.
> > +
> > +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> > +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> > +encapsulated packet is calculated over the inner as opposed to outer header.
> > +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> > +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> > +Supported/enabled hash types}.
> > +
> > +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> > +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> > +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> > +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> > +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> > +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> > +
> > +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> > +negotiated, the device calculates the hash over the outer header, and
> > +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> > +is no longer valid.
> > +
> > \subparagraph{Supported/enabled hash types}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> > +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> > +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> > Hash types applicable for IPv4 packets:
> > \begin{lstlisting}
> > #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> > @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> > \end{lstlisting}
> >
> > +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> > +hash type below indicates that the hash is calculated over the inner
> > +header of the encapsulated packet:
> > +Hash type applicable for inner payload of the gre-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> > +\end{lstlisting}
> > +Hash type applicable for inner payload of the vxlan-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> > +\end{lstlisting}
> > +Hash type applicable for inner payload of the geneve-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> > +\end{lstlisting}
> > +
> > \subparagraph{IPv4 packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> > The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> > @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> > \end{itemize}
> >
> > +\subparagraph{Inner payload of an encapsulated packet}
> > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> > +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> > +encapsulation hash type is set in \field{hash_types}, the device calculates the
> > +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> > +/ Network Device / Device Operation / Processing of Incoming Packets /
> > +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> > +
> > +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> > +according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> > + TCPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> > + inner UDPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet without an extension header
> > +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> > + inner UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet with an extension header
> > +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemsize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> > + UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \end{itemize}
> > + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> > + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> > + an extension header.
> > +\end{itemsize}
> > +
> > \paragraph{Hash reporting for incoming packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
> >
> > -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> > - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> > -and \field{hash_value} with the value of calculated hash.
> > +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> > +hash for the packet, the device fills \field{hash_report} with the report type
> > +of calculated hash, and \field{hash_value} with the value of calculated hash.
> > +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
>
> needs to fill -> fills
>
> for consistency
I agree.
Thanks.
>
> > +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> > +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
> >
> > If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> > -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> > +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> > +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
> >
> > Possible values that the device can report in \field{hash_report} are defined below.
> > They correspond to supported hash types defined in
> > @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> > \end{lstlisting}
> >
> > +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> > +\field{hash_report_tunnel} can report the type of the encapsulated
> > +packet to the driver over the inner header hash calculation.
> > +Possible values that the device can report in \field{hash_report_tunnel}
> > +are defined below:
> > +
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> > +#define VIRTIO_NET_HASH_REPORT_GRE 1
> > +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> > +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> > +\end{lstlisting}
> > +
> > +They correspond to supported hash types defined in
> > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> > +
> > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
> >
> > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> > diff --git a/introduction.tex b/introduction.tex
> > index 287c5fc..ff01a9b 100644
> > --- a/introduction.tex
> > +++ b/introduction.tex
> > @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> > \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> > Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> > \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> > + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> > + Generic Routing Encapsulation
> > + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> > + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> > + Virtual eXtensible Local Area Network
> > + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> > + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> > + Generic Network Virtualization Encapsulation
> > + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> > + \phantomsection\label{intro:IP}\textbf{[IP]} &
> > + INTERNET PROTOCOL
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> > + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> > + User Datagram Protocol
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> > + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> > + TRANSMISSION CONTROL PROTOCOL
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> > +
> >
> > \end{longtable}
> >
> > --
> > 2.19.1.6.gb485710b
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-06 6:42 ` [virtio-comment] " Heng Qi
@ 2023-01-06 6:59 ` Michael S. Tsirkin
2023-01-09 2:43 ` [virtio-comment] Re: [virtio-dev] " Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-06 6:59 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > using the outer header of the receive packets is always fixed for the
> > > same flow packets, i.e. they will be steered to the same receive queue.
> > >
> > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > in \field{hash_types}, which instructs the device to calculate the
> > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > >
> > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > >
> > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >
> >
> > ok close to being ready. a couple of minor comments.
> >
> > > ---
> > > v6:
> > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > >
> > > v5:
> > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > 4. Clarify some sentences. @Michael S. Tsirkin
> > >
> > > v4:
> > > 1. Clarify some paragraphs. @Cornelia Huck
> > > 2. Fix the u8 type. @Cornelia Huck
> > >
> > > v3:
> > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > >
> > > v2:
> > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > >
> > > v1:
> > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > 2. Clarify some paragraphs. @Jason Wang
> > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > >
> > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > introduction.tex | 19 +++++
> > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > >
> > > diff --git a/content.tex b/content.tex
> > > index e863709..7845f6c 100644
> > > --- a/content.tex
> > > +++ b/content.tex
> > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > channel.
> > >
> > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> >
> > I would probably drop the list of tunnel types here.
>
> Do you mean to use "Device supports inner header hash for
> tunnel-encapsulated packets." instead? Why? We do want to use this
> feature bit to indicate that the device supports inner hashing of
> GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
So imagine we add a new tunnel type. Let's say there's VXLAN v2.
why would we need a new feature bit? I think a new hash type
will be sufficient. No?
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-06 6:59 ` Michael S. Tsirkin
@ 2023-01-09 2:43 ` Heng Qi
2023-01-09 8:59 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-09 2:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Fri, Jan 06, 2023 at 01:59:38AM -0500, Michael S. Tsirkin wrote:
> On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> > On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > > using the outer header of the receive packets is always fixed for the
> > > > same flow packets, i.e. they will be steered to the same receive queue.
> > > >
> > > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > > in \field{hash_types}, which instructs the device to calculate the
> > > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > > >
> > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > > >
> > > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > >
> > >
> > > ok close to being ready. a couple of minor comments.
> > >
> > > > ---
> > > > v6:
> > > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > > >
> > > > v5:
> > > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > > 4. Clarify some sentences. @Michael S. Tsirkin
> > > >
> > > > v4:
> > > > 1. Clarify some paragraphs. @Cornelia Huck
> > > > 2. Fix the u8 type. @Cornelia Huck
> > > >
> > > > v3:
> > > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > > >
> > > > v2:
> > > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > > >
> > > > v1:
> > > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > > 2. Clarify some paragraphs. @Jason Wang
> > > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > > >
> > > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > > introduction.tex | 19 +++++
> > > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > > >
> > > > diff --git a/content.tex b/content.tex
> > > > index e863709..7845f6c 100644
> > > > --- a/content.tex
> > > > +++ b/content.tex
> > > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > > channel.
> > > >
> > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > >
> > > I would probably drop the list of tunnel types here.
> >
> > Do you mean to use "Device supports inner header hash for
> > tunnel-encapsulated packets." instead? Why? We do want to use this
> > feature bit to indicate that the device supports inner hashing of
> > GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> > https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> > we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> > VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> > VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
>
> So imagine we add a new tunnel type. Let's say there's VXLAN v2.
> why would we need a new feature bit? I think a new hash type
> will be sufficient. No?
If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.".
Then the following may happen
1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that
device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
2. For other encapsulation protocols such as ip in ip, after device src and device dst have
negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip,
but it is not clear that device dst also supports ip in ip. Especially when migrating, this can
lead to inconsistencies in live migrations.
So, I think it's better to keep the original description:
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
Thanks.
>
> --
> MST
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-09 2:43 ` [virtio-comment] Re: [virtio-dev] " Heng Qi
@ 2023-01-09 8:59 ` Jason Wang
2023-01-09 11:34 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2023-01-09 8:59 UTC (permalink / raw)
To: Heng Qi
Cc: Michael S. Tsirkin, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Mon, Jan 9, 2023 at 10:43 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
> On Fri, Jan 06, 2023 at 01:59:38AM -0500, Michael S. Tsirkin wrote:
> > On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> > > On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > > > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > > > using the outer header of the receive packets is always fixed for the
> > > > > same flow packets, i.e. they will be steered to the same receive queue.
> > > > >
> > > > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > > > in \field{hash_types}, which instructs the device to calculate the
> > > > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > > > >
> > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > > > >
> > > > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > >
> > > >
> > > > ok close to being ready. a couple of minor comments.
> > > >
> > > > > ---
> > > > > v6:
> > > > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > > > >
> > > > > v5:
> > > > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > > > 4. Clarify some sentences. @Michael S. Tsirkin
> > > > >
> > > > > v4:
> > > > > 1. Clarify some paragraphs. @Cornelia Huck
> > > > > 2. Fix the u8 type. @Cornelia Huck
> > > > >
> > > > > v3:
> > > > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > > > >
> > > > > v2:
> > > > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > > > >
> > > > > v1:
> > > > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > > > 2. Clarify some paragraphs. @Jason Wang
> > > > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > > > >
> > > > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > > > introduction.tex | 19 +++++
> > > > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > > > >
> > > > > diff --git a/content.tex b/content.tex
> > > > > index e863709..7845f6c 100644
> > > > > --- a/content.tex
> > > > > +++ b/content.tex
> > > > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > > > channel.
> > > > >
> > > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > >
> > > > I would probably drop the list of tunnel types here.
> > >
> > > Do you mean to use "Device supports inner header hash for
> > > tunnel-encapsulated packets." instead? Why? We do want to use this
> > > feature bit to indicate that the device supports inner hashing of
> > > GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> > > https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> > > we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> > > VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> > > VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
> >
> > So imagine we add a new tunnel type. Let's say there's VXLAN v2.
> > why would we need a new feature bit? I think a new hash type
> > will be sufficient. No?
>
> If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
> "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.".
> Then the following may happen
> 1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that
> device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
> 2. For other encapsulation protocols such as ip in ip, after device src and device dst have
> negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip,
> but it is not clear that device dst also supports ip in ip. Especially when migrating, this can
> lead to inconsistencies in live migrations.
Yes, this looks like the only way that I can think of to keep
migration compatibility in an easy way.
Thanks
> So, I think it's better to keep the original description:
> "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
>
> Thanks.
>
> >
> > --
> > MST
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-09 8:59 ` Jason Wang
@ 2023-01-09 11:34 ` Michael S. Tsirkin
2023-01-10 2:06 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-09 11:34 UTC (permalink / raw)
To: Jason Wang
Cc: Heng Qi, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Mon, Jan 09, 2023 at 04:59:26PM +0800, Jason Wang wrote:
> On Mon, Jan 9, 2023 at 10:43 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
> >
> > On Fri, Jan 06, 2023 at 01:59:38AM -0500, Michael S. Tsirkin wrote:
> > > On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> > > > On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > > > > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > > > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > > > > using the outer header of the receive packets is always fixed for the
> > > > > > same flow packets, i.e. they will be steered to the same receive queue.
> > > > > >
> > > > > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > > > > in \field{hash_types}, which instructs the device to calculate the
> > > > > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > > > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > > > > >
> > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > > > > >
> > > > > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > >
> > > > >
> > > > > ok close to being ready. a couple of minor comments.
> > > > >
> > > > > > ---
> > > > > > v6:
> > > > > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > > > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > > > > >
> > > > > > v5:
> > > > > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > > > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > > > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > > > > 4. Clarify some sentences. @Michael S. Tsirkin
> > > > > >
> > > > > > v4:
> > > > > > 1. Clarify some paragraphs. @Cornelia Huck
> > > > > > 2. Fix the u8 type. @Cornelia Huck
> > > > > >
> > > > > > v3:
> > > > > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > > > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > > > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > > > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > > > > >
> > > > > > v2:
> > > > > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > > > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > > > > >
> > > > > > v1:
> > > > > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > > > > 2. Clarify some paragraphs. @Jason Wang
> > > > > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > > > > >
> > > > > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > > > > introduction.tex | 19 +++++
> > > > > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > > > > >
> > > > > > diff --git a/content.tex b/content.tex
> > > > > > index e863709..7845f6c 100644
> > > > > > --- a/content.tex
> > > > > > +++ b/content.tex
> > > > > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > > > > channel.
> > > > > >
> > > > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > >
> > > > > I would probably drop the list of tunnel types here.
> > > >
> > > > Do you mean to use "Device supports inner header hash for
> > > > tunnel-encapsulated packets." instead? Why? We do want to use this
> > > > feature bit to indicate that the device supports inner hashing of
> > > > GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> > > > https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> > > > we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> > > > VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> > > > VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
> > >
> > > So imagine we add a new tunnel type. Let's say there's VXLAN v2.
> > > why would we need a new feature bit? I think a new hash type
> > > will be sufficient. No?
> >
> > If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
> > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.".
> > Then the following may happen
> > 1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that
> > device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
> > 2. For other encapsulation protocols such as ip in ip, after device src and device dst have
> > negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip,
> > but it is not clear that device dst also supports ip in ip. Especially when migrating, this can
> > lead to inconsistencies in live migrations.
it does not matter. e.g. if device supports GRE but not VXLAN then
feature bit it set but hash types are different.
and this is not different from any other offload we have.
Generally the only reason we even use a new feature bit for this
tunneling is because the header structure is a bit different
so this kind of makes sense.
>
> Yes, this looks like the only way that I can think of to keep
> migration compatibility in an easy way.
>
> Thanks
Jason migration compatibility must also check hash types.
There's no chance with the current hash offload scheme to
only rely on feature bits for migration compatibility.
> > So, I think it's better to keep the original description:
> > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
> >
> > Thanks.
> >
> > >
> > > --
> > > MST
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
> >
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
2023-01-04 12:34 ` [virtio-comment] " Heng Qi
2023-01-06 5:27 ` Michael S. Tsirkin
@ 2023-01-09 11:36 ` Michael S. Tsirkin
2023-01-10 7:46 ` Heng Qi
2023-01-09 11:39 ` Michael S. Tsirkin
` (2 subsequent siblings)
5 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-09 11:36 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> If the tunnel is used to encapsulate the packets, the hash calculated
> using the outer header of the receive packets is always fixed for the
> same flow packets, i.e. they will be steered to the same receive queue.
>
> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> in \field{hash_types}, which instructs the device to calculate the
> hash using the inner headers of tunnel-encapsulated packets. Besides,
> values in \field{hash_report_tunnel} are added to report tunnel types.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v6:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> introduction.tex | 19 +++++
> 2 files changed, 203 insertions(+), 7 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..7845f6c 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> + is negotiated, an encapsulation packet type.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> +encapsulated packet is calculated over the inner as opposed to outer header.
> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> +Supported/enabled hash types}.
> +
> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> +
> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> +negotiated, the device calculates the hash over the outer header, and
> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> +is no longer valid.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner
> +header of the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payload of an encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_types}, the device calculates the
> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> +
> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> +according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> + TCPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> + inner UDPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet without an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> + inner UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet with an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> + UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> + an extension header.
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the encapsulated
> +packet to the driver over the inner header hash calculation.
this sentence is a bit confused btw. so this reports
the type of the encapsulated packet. but what does
"over the inner header hash calculation." mean?
what happens over what?
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +They correspond to supported hash types defined in
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..ff01a9b 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> +
>
> \end{longtable}
>
> --
> 2.19.1.6.gb485710b
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
` (2 preceding siblings ...)
2023-01-09 11:36 ` Michael S. Tsirkin
@ 2023-01-09 11:39 ` Michael S. Tsirkin
2023-01-10 7:47 ` [virtio-comment] " Heng Qi
2023-01-11 9:45 ` Michael S. Tsirkin
2023-01-18 23:45 ` [virtio-dev] " Parav Pandit
5 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-09 11:39 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
Btw this "are defined below" all over the place is just contributing
to making the spec unnecesarily verbose. Simple "are:" will do.
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-09 11:34 ` Michael S. Tsirkin
@ 2023-01-10 2:06 ` Jason Wang
2023-01-10 5:24 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2023-01-10 2:06 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Mon, Jan 9, 2023 at 7:34 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Mon, Jan 09, 2023 at 04:59:26PM +0800, Jason Wang wrote:
> > On Mon, Jan 9, 2023 at 10:43 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
> > >
> > > On Fri, Jan 06, 2023 at 01:59:38AM -0500, Michael S. Tsirkin wrote:
> > > > On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> > > > > On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > > > > > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > > > > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > > > > > using the outer header of the receive packets is always fixed for the
> > > > > > > same flow packets, i.e. they will be steered to the same receive queue.
> > > > > > >
> > > > > > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > > > > > in \field{hash_types}, which instructs the device to calculate the
> > > > > > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > > > > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > > > > > >
> > > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > > > > > >
> > > > > > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > >
> > > > > >
> > > > > > ok close to being ready. a couple of minor comments.
> > > > > >
> > > > > > > ---
> > > > > > > v6:
> > > > > > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > > > > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > > > > > >
> > > > > > > v5:
> > > > > > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > > > > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > > > > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > > > > > 4. Clarify some sentences. @Michael S. Tsirkin
> > > > > > >
> > > > > > > v4:
> > > > > > > 1. Clarify some paragraphs. @Cornelia Huck
> > > > > > > 2. Fix the u8 type. @Cornelia Huck
> > > > > > >
> > > > > > > v3:
> > > > > > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > > > > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > > > > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > > > > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > > > > > >
> > > > > > > v2:
> > > > > > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > > > > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > > > > > >
> > > > > > > v1:
> > > > > > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > > > > > 2. Clarify some paragraphs. @Jason Wang
> > > > > > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > > > > > >
> > > > > > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > > > > > introduction.tex | 19 +++++
> > > > > > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > > > > > >
> > > > > > > diff --git a/content.tex b/content.tex
> > > > > > > index e863709..7845f6c 100644
> > > > > > > --- a/content.tex
> > > > > > > +++ b/content.tex
> > > > > > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > > > > > channel.
> > > > > > >
> > > > > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > > >
> > > > > > I would probably drop the list of tunnel types here.
> > > > >
> > > > > Do you mean to use "Device supports inner header hash for
> > > > > tunnel-encapsulated packets." instead? Why? We do want to use this
> > > > > feature bit to indicate that the device supports inner hashing of
> > > > > GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> > > > > https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> > > > > we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> > > > > VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> > > > > VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
> > > >
> > > > So imagine we add a new tunnel type. Let's say there's VXLAN v2.
> > > > why would we need a new feature bit? I think a new hash type
> > > > will be sufficient. No?
> > >
> > > If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
> > > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.".
> > > Then the following may happen
> > > 1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that
> > > device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
> > > 2. For other encapsulation protocols such as ip in ip, after device src and device dst have
> > > negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip,
> > > but it is not clear that device dst also supports ip in ip. Especially when migrating, this can
> > > lead to inconsistencies in live migrations.
>
> it does not matter. e.g. if device supports GRE but not VXLAN then
> feature bit it set but hash types are different.
But the hash will be used for RSS steering. Consider the steering on
ipip work on src but not on dst it would break application logic.
> and this is not different from any other offload we have.
But we have different feature bits for TCPv4/v6.
>
> Generally the only reason we even use a new feature bit for this
> tunneling is because the header structure is a bit different
> so this kind of makes sense.
>
> >
> > Yes, this looks like the only way that I can think of to keep
> > migration compatibility in an easy way.
> >
> > Thanks
>
> Jason migration compatibility must also check hash types.
> There's no chance with the current hash offload scheme to
> only rely on feature bits for migration compatibility.
This will give extra pressure on the management stack, e.g it requires
the device to have an out of spec way for introspection.
Thanks
>
>
> > > So, I think it's better to keep the original description:
> > > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
> > >
> > > Thanks.
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
> > >
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-10 2:06 ` Jason Wang
@ 2023-01-10 5:24 ` Michael S. Tsirkin
2023-01-10 5:57 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-10 5:24 UTC (permalink / raw)
To: Jason Wang
Cc: Heng Qi, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Tue, Jan 10, 2023 at 10:06:53AM +0800, Jason Wang wrote:
> On Mon, Jan 9, 2023 at 7:34 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Mon, Jan 09, 2023 at 04:59:26PM +0800, Jason Wang wrote:
> > > On Mon, Jan 9, 2023 at 10:43 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
> > > >
> > > > On Fri, Jan 06, 2023 at 01:59:38AM -0500, Michael S. Tsirkin wrote:
> > > > > On Fri, Jan 06, 2023 at 02:42:21PM +0800, Heng Qi wrote:
> > > > > > On Fri, Jan 06, 2023 at 12:27:04AM -0500, Michael S. Tsirkin wrote:
> > > > > > > On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > > > > > > > If the tunnel is used to encapsulate the packets, the hash calculated
> > > > > > > > using the outer header of the receive packets is always fixed for the
> > > > > > > > same flow packets, i.e. they will be steered to the same receive queue.
> > > > > > > >
> > > > > > > > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > > > > > > > in \field{hash_types}, which instructs the device to calculate the
> > > > > > > > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > > > > > > > values in \field{hash_report_tunnel} are added to report tunnel types.
> > > > > > > >
> > > > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> > > > > > > >
> > > > > > > > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > > > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > > >
> > > > > > >
> > > > > > > ok close to being ready. a couple of minor comments.
> > > > > > >
> > > > > > > > ---
> > > > > > > > v6:
> > > > > > > > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > > > > > > > 2. Fix some syntax issues. @Michael S. Tsirkin
> > > > > > > >
> > > > > > > > v5:
> > > > > > > > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > > > > > > > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > > > > > > > 3. Move the links to introduction section. @Michael S. Tsirkin
> > > > > > > > 4. Clarify some sentences. @Michael S. Tsirkin
> > > > > > > >
> > > > > > > > v4:
> > > > > > > > 1. Clarify some paragraphs. @Cornelia Huck
> > > > > > > > 2. Fix the u8 type. @Cornelia Huck
> > > > > > > >
> > > > > > > > v3:
> > > > > > > > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > > > > > > > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > > > > > > > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > > > > > > > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> > > > > > > >
> > > > > > > > v2:
> > > > > > > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > > > > > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > > > > > > >
> > > > > > > > v1:
> > > > > > > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > > > > > > 2. Clarify some paragraphs. @Jason Wang
> > > > > > > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > > > > > > >
> > > > > > > > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > > > > > > > introduction.tex | 19 +++++
> > > > > > > > 2 files changed, 203 insertions(+), 7 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/content.tex b/content.tex
> > > > > > > > index e863709..7845f6c 100644
> > > > > > > > --- a/content.tex
> > > > > > > > +++ b/content.tex
> > > > > > > > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > > > > > > channel.
> > > > > > > >
> > > > > > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > > > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > > > >
> > > > > > > I would probably drop the list of tunnel types here.
> > > > > >
> > > > > > Do you mean to use "Device supports inner header hash for
> > > > > > tunnel-encapsulated packets." instead? Why? We do want to use this
> > > > > > feature bit to indicate that the device supports inner hashing of
> > > > > > GRE, VXLAN and GENEVE encapsulated packets. As in the v3 discussion
> > > > > > https://lists.oasis-open.org/archives/virtio-dev/202212/msg00024.html ,
> > > > > > we discussed using VIRTIO_NET_F_HASH_TUNNEL to replace
> > > > > > VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER and plan to use
> > > > > > VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extensions.
> > > > >
> > > > > So imagine we add a new tunnel type. Let's say there's VXLAN v2.
> > > > > why would we need a new feature bit? I think a new hash type
> > > > > will be sufficient. No?
> > > >
> > > > If the description for VIRTIO_NET_F_HASH_TUNNEL is as follows:
> > > > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.".
> > > > Then the following may happen
> > > > 1. For VXLANv2, if both device src and device dst have negotiated this feature, it is assumed that
> > > > device src supports VXLAN and VXLANv2, but device dst may only support VXLAN, not VXLANv2.
> > > > 2. For other encapsulation protocols such as ip in ip, after device src and device dst have
> > > > negotiated this feature, it is assumed that device src supports GRE, VXLAN, GENEVE and ip in ip,
> > > > but it is not clear that device dst also supports ip in ip. Especially when migrating, this can
> > > > lead to inconsistencies in live migrations.
> >
> > it does not matter. e.g. if device supports GRE but not VXLAN then
> > feature bit it set but hash types are different.
>
> But the hash will be used for RSS steering. Consider the steering on
> ipip work on src but not on dst it would break application logic.
Exactly. And for this reason just making sure feature bits match
is not sufficient for migration. It follows that hash types
must also match, and therefore there is no need to invent
new feature bits for migration if all we are doing is adding new hash types.
> > and this is not different from any other offload we have.
>
> But we have different feature bits for TCPv4/v6.
For segmentation but not for hash. Basically because we
don't have segmentation types analogous to hash types.
> >
> > Generally the only reason we even use a new feature bit for this
> > tunneling is because the header structure is a bit different
> > so this kind of makes sense.
> >
> > >
> > > Yes, this looks like the only way that I can think of to keep
> > > migration compatibility in an easy way.
> > >
> > > Thanks
> >
> > Jason migration compatibility must also check hash types.
> > There's no chance with the current hash offload scheme to
> > only rely on feature bits for migration compatibility.
>
> This will give extra pressure on the management stack, e.g it requires
> the device to have an out of spec way for introspection.
>
> Thanks
As I tried to explain this is already the case. Feature bits do not
describe device capabilities fully, some of them are in config space.
> >
> >
> > > > So, I think it's better to keep the original description:
> > > > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets."
> > > >
> > > > Thanks.
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
> > > >
> >
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-10 5:24 ` Michael S. Tsirkin
@ 2023-01-10 5:57 ` Michael S. Tsirkin
2023-01-10 7:26 ` Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-10 5:57 UTC (permalink / raw)
To: Jason Wang
Cc: Heng Qi, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Tue, Jan 10, 2023 at 12:25:02AM -0500, Michael S. Tsirkin wrote:
> > This will give extra pressure on the management stack, e.g it requires
> > the device to have an out of spec way for introspection.
> >
> > Thanks
>
> As I tried to explain this is already the case. Feature bits do not
> describe device capabilities fully, some of them are in config space.
To be precise, this does not necessarily require introspection, but
it does require management control over config space
such as supported hash types just like it has control over feature bits.
E.g. QEMU currently seems to hard-code these to
#define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
but there's no reason not to give management control over these.
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-10 5:57 ` Michael S. Tsirkin
@ 2023-01-10 7:26 ` Heng Qi
2023-01-11 3:22 ` [virtio-comment] " Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-10 7:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jason Wang, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Tue, Jan 10, 2023 at 12:57:38AM -0500, Michael S. Tsirkin wrote:
> On Tue, Jan 10, 2023 at 12:25:02AM -0500, Michael S. Tsirkin wrote:
> > > This will give extra pressure on the management stack, e.g it requires
> > > the device to have an out of spec way for introspection.
> > >
> > > Thanks
> >
> > As I tried to explain this is already the case. Feature bits do not
> > describe device capabilities fully, some of them are in config space.
>
> To be precise, this does not necessarily require introspection, but
> it does require management control over config space
> such as supported hash types just like it has control over feature bits.
> E.g. QEMU currently seems to hard-code these to
> #define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
> VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
> VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
> VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
> VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
> VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
> VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
> VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
> VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
>
> but there's no reason not to give management control over these.
Yes, QEMU has requirements for live migration: the PCI config space will be
checked in get_pci_config_device(), and if src and dst are inconsistent, it
will prompt that the live migration failed.
In fact, this is also done within our group. Live migration requires that
the two VMs have the same rss configuration, otherwise the migration will fail.
Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into
"[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.",
and use different hash_types to help the migration determine whether it can succeed.
Thanks.
>
> --
> MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH v7] virtio-net: support inner header hash
2023-01-09 11:36 ` Michael S. Tsirkin
@ 2023-01-10 7:46 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-01-10 7:46 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/1/9 下午7:36, Michael S. Tsirkin 写道:
> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
>> If the tunnel is used to encapsulate the packets, the hash calculated
>> using the outer header of the receive packets is always fixed for the
>> same flow packets, i.e. they will be steered to the same receive queue.
>>
>> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
>> in \field{hash_types}, which instructs the device to calculate the
>> hash using the inner headers of tunnel-encapsulated packets. Besides,
>> values in \field{hash_report_tunnel} are added to report tunnel types.
>>
>> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>>
>> Reviewed-by: Jason Wang <jasowang@redhat.com>
>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> v6:
>> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
>> 2. Fix some syntax issues. @Michael S. Tsirkin
>>
>> v5:
>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
>> 3. Move the links to introduction section. @Michael S. Tsirkin
>> 4. Clarify some sentences. @Michael S. Tsirkin
>>
>> v4:
>> 1. Clarify some paragraphs. @Cornelia Huck
>> 2. Fix the u8 type. @Cornelia Huck
>>
>> v3:
>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
>> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>>
>> v2:
>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>>
>> v1:
>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>> 2. Clarify some paragraphs. @Jason Wang
>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>>
>> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
>> introduction.tex | 19 +++++
>> 2 files changed, 203 insertions(+), 7 deletions(-)
>>
>> diff --git a/content.tex b/content.tex
>> index e863709..7845f6c 100644
>> --- a/content.tex
>> +++ b/content.tex
>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>> channel.
>>
>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
>> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
>> +
>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>>
>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>> to several segments when each of these smaller packets has UDP header.
>>
>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>> - value and a type of calculated hash.
>> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
>> + is negotiated, an encapsulation packet type.
>>
>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>> value. Device benefits from knowing the exact header length.
>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
>> \end{description}
>>
>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>> le16 num_buffers;
>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> };
>> \end{lstlisting}
>>
>> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> A device attempts to calculate a per-packet hash in the following cases:
>> \begin{itemize}
>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
>> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
>> \end{itemize}
>>
>> If the feature VIRTIO_NET_F_RSS was negotiated:
>> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
>> \end{itemize}
>>
>> +\subparagraph{Tunnel/Encapsulated packet}
>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
>> +A tunnel packet is encapsulated from the original packet based on the tunneling
>> +protocol (only a single level of encapsulation is currently supported). The
>> +encapsulated packet contains an outer header and an inner header, and the device
>> +calculates the hash over either the inner header or the outer header.
>> +
>> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
>> +encapsulated packet is calculated over the inner as opposed to outer header.
>> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
>> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
>> +Supported/enabled hash types}.
>> +
>> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
>> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
>> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
>> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
>> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
>> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
>> +
>> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
>> +negotiated, the device calculates the hash over the outer header, and
>> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
>> +is no longer valid.
>> +
>> \subparagraph{Supported/enabled hash types}
>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
>> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
>> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
>> Hash types applicable for IPv4 packets:
>> \begin{lstlisting}
>> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
>> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>> \end{lstlisting}
>>
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
>> +hash type below indicates that the hash is calculated over the inner
>> +header of the encapsulated packet:
>> +Hash type applicable for inner payload of the gre-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the vxlan-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the geneve-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>> +\end{lstlisting}
>> +
>> \subparagraph{IPv4 packets}
>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
>> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
>> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
>> \end{itemize}
>>
>> +\subparagraph{Inner payload of an encapsulated packet}
>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>> +encapsulation hash type is set in \field{hash_types}, the device calculates the
>> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
>> +/ Network Device / Device Operation / Processing of Incoming Packets /
>> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
>> +
>> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
>> +according to 'Enabled hash types' bitmask as follows:
>> +\begin{itemize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
>> + TCPv4 header, the hash is calculated over the following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemsize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
>> + inner UDPv4 header, the hash is calculated over the following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \end{itemsize}
>> + \item Else the device does not calculate the hash
>> +\end{itemize}
>> +
>> +The device calculates the hash on the inner IPv6 packet without an extension header
>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>> +\begin{itemize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
>> + TCPv6 header, the hash is calculated over the following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemsize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
>> + inner UDPv6 header, the hash is calculated over the following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \end{itemsize}
>> + \item Else the device does not calculate the hash
>> +\end{itemize}
>> +
>> +The device calculates the hash on the inner IPv6 packet with an extension header
>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>> +\begin{itemsize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
>> + TCPv6 header, the hash is calculated over the following fields:
>> + \begin{itemize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
>> + UDPv6 header, the hash is calculated over the following fields:
>> + \begin{itemsize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \end{itemize}
>> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
>> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
>> + an extension header.
>> +\end{itemsize}
>> +
>> \paragraph{Hash reporting for incoming packets}
>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>>
>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
>> -and \field{hash_value} with the value of calculated hash.
>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
>> +hash for the packet, the device fills \field{hash_report} with the report type
>> +of calculated hash, and \field{hash_value} with the value of calculated hash.
>> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
>> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
>> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>>
>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
>> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
>> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>>
>> Possible values that the device can report in \field{hash_report} are defined below.
>> They correspond to supported hash types defined in
>> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>> \end{lstlisting}
>>
>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>> +\field{hash_report_tunnel} can report the type of the encapsulated
>> +packet to the driver over the inner header hash calculation.
> this sentence is a bit confused btw. so this reports
> the type of the encapsulated packet. but what does
> "over the inner header hash calculation." mean?
> what happens over what?
Just means "After the hash is computed, the encapsulation type can be
reported to the driver".
It means redundant here, I will remove "over the inner header hash
calculation".
Thanks.
>
>
>> +Possible values that the device can report in \field{hash_report_tunnel}
>> +are defined below:
>> +
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>> +\end{lstlisting}
>> +
>> +They correspond to supported hash types defined in
>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>> +
>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>
>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>> diff --git a/introduction.tex b/introduction.tex
>> index 287c5fc..ff01a9b 100644
>> --- a/introduction.tex
>> +++ b/introduction.tex
>> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
>> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
>> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
>> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
>> + Generic Routing Encapsulation
>> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
>> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
>> + Virtual eXtensible Local Area Network
>> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
>> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
>> + Generic Network Virtualization Encapsulation
>> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
>> + \phantomsection\label{intro:IP}\textbf{[IP]} &
>> + INTERNET PROTOCOL
>> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
>> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
>> + User Datagram Protocol
>> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
>> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
>> + TRANSMISSION CONTROL PROTOCOL
>> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
>> +
>>
>> \end{longtable}
>>
>> --
>> 2.19.1.6.gb485710b
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-09 11:39 ` Michael S. Tsirkin
@ 2023-01-10 7:47 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-01-10 7:47 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/1/9 下午7:39, Michael S. Tsirkin 写道:
> Btw this "are defined below" all over the place is just contributing
> to making the spec unnecesarily verbose. Simple "are:" will do.
Sure. I'll fix it in the next version.
Thanks.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-10 7:26 ` Heng Qi
@ 2023-01-11 3:22 ` Heng Qi
2023-01-11 4:45 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-11 3:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jason Wang, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/1/10 下午3:26, Heng Qi 写道:
> On Tue, Jan 10, 2023 at 12:57:38AM -0500, Michael S. Tsirkin wrote:
>> On Tue, Jan 10, 2023 at 12:25:02AM -0500, Michael S. Tsirkin wrote:
>>>> This will give extra pressure on the management stack, e.g it requires
>>>> the device to have an out of spec way for introspection.
>>>>
>>>> Thanks
>>> As I tried to explain this is already the case. Feature bits do not
>>> describe device capabilities fully, some of them are in config space.
>> To be precise, this does not necessarily require introspection, but
>> it does require management control over config space
>> such as supported hash types just like it has control over feature bits.
>> E.g. QEMU currently seems to hard-code these to
>> #define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
>> VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
>> VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
>> VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
>> VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
>> VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
>> VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
>> VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
>> VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
>>
>> but there's no reason not to give management control over these.
> Yes, QEMU has requirements for live migration: the PCI config space will be
> checked in get_pci_config_device(), and if src and dst are inconsistent, it
> will prompt that the live migration failed.
To be clearer, I mean \filed{supported_hash_types} in structure
virtio_net_config.
Thanks.
> In fact, this is also done within our group. Live migration requires that
> the two VMs have the same rss configuration, otherwise the migration will fail.
>
> Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into
> "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.",
> and use different hash_types to help the migration determine whether it can succeed.
>
> Thanks.
>
>> --
>> MST
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-11 3:22 ` [virtio-comment] " Heng Qi
@ 2023-01-11 4:45 ` Jason Wang
2023-01-11 9:49 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2023-01-11 4:45 UTC (permalink / raw)
To: Heng Qi
Cc: Michael S. Tsirkin, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 11, 2023 at 11:23 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
>
>
> 在 2023/1/10 下午3:26, Heng Qi 写道:
> > On Tue, Jan 10, 2023 at 12:57:38AM -0500, Michael S. Tsirkin wrote:
> >> On Tue, Jan 10, 2023 at 12:25:02AM -0500, Michael S. Tsirkin wrote:
> >>>> This will give extra pressure on the management stack, e.g it requires
> >>>> the device to have an out of spec way for introspection.
> >>>>
> >>>> Thanks
> >>> As I tried to explain this is already the case. Feature bits do not
> >>> describe device capabilities fully, some of them are in config space.
Yes.
> >> To be precise, this does not necessarily require introspection, but
> >> it does require management control over config space
> >> such as supported hash types just like it has control over feature bits.
> >> E.g. QEMU currently seems to hard-code these to
> >> #define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
> >> VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
> >> VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
> >> VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
> >>
> >> but there's no reason not to give management control over these.
Note that the management expects the migration compatibility to work
with machine types. So it needs a way to disable some tunnel hash
types to make it work for old machine types.
> > Yes, QEMU has requirements for live migration: the PCI config space will be
> > checked in get_pci_config_device(), and if src and dst are inconsistent, it
> > will prompt that the live migration failed.
It might be too late since it can't work for the second run (unlike subsection).
>
> To be clearer, I mean \filed{supported_hash_types} in structure
> virtio_net_config.
Yes.
Thanks
>
> Thanks.
>
> > In fact, this is also done within our group. Live migration requires that
> > the two VMs have the same rss configuration, otherwise the migration will fail.
> >
> > Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into
> > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.",
> > and use different hash_types to help the migration determine whether it can succeed.
> >
> > Thanks.
> >
> >> --
> >> MST
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
` (3 preceding siblings ...)
2023-01-09 11:39 ` Michael S. Tsirkin
@ 2023-01-11 9:45 ` Michael S. Tsirkin
2023-01-16 8:01 ` [virtio-comment] " Heng Qi
2023-01-18 23:45 ` [virtio-dev] " Parav Pandit
5 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-11 9:45 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> If the tunnel is used to encapsulate the packets, the hash calculated
> using the outer header of the receive packets is always fixed for the
> same flow packets, i.e. they will be steered to the same receive queue.
>
> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> in \field{hash_types}, which instructs the device to calculate the
> hash using the inner headers of tunnel-encapsulated packets. Besides,
> values in \field{hash_report_tunnel} are added to report tunnel types.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v6:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> introduction.tex | 19 +++++
> 2 files changed, 203 insertions(+), 7 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..7845f6c 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> + is negotiated, an encapsulation packet type.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
of -> if?
also let's add: otherwise reserved?
> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> +encapsulated packet is calculated over the inner as opposed to outer header.
> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> +Supported/enabled hash types}.
> +
> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> +
> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> +negotiated, the device calculates the hash over the outer header, and
> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> +is no longer valid.
Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER
must not be set without VIRTIO_NET_F_HASH_TUNNEL?
If yes then we can add conformance statements to this end and then
drop the
"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
all over the place just relying on hash type being set instead.
Also, note that it has to be ok for device to expose hash types without
negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL
just add hash_report_tunnel and have hash types speak for themselves?
It makes things nicely orthogonal in that if one does not care
about knowing encapsulation type (e.g. for RSS) one can disable
VIRTIO_NET_F_HASH_TUNNEL.
Objections? Jason?
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner
> +header of the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payload of an encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_types}, the device calculates the
> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> +
> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> +according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> + TCPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> + inner UDPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet without an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> + inner UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet with an extension header
> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> + UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> + an extension header.
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +They correspond to supported hash types defined in
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..ff01a9b 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> +
>
> \end{longtable}
>
> --
> 2.19.1.6.gb485710b
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-11 4:45 ` Jason Wang
@ 2023-01-11 9:49 ` Michael S. Tsirkin
0 siblings, 0 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-01-11 9:49 UTC (permalink / raw)
To: Jason Wang
Cc: Heng Qi, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 11, 2023 at 12:45:06PM +0800, Jason Wang wrote:
> On Wed, Jan 11, 2023 at 11:23 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
> >
> >
> >
> > 在 2023/1/10 下午3:26, Heng Qi 写道:
> > > On Tue, Jan 10, 2023 at 12:57:38AM -0500, Michael S. Tsirkin wrote:
> > >> On Tue, Jan 10, 2023 at 12:25:02AM -0500, Michael S. Tsirkin wrote:
> > >>>> This will give extra pressure on the management stack, e.g it requires
> > >>>> the device to have an out of spec way for introspection.
> > >>>>
> > >>>> Thanks
> > >>> As I tried to explain this is already the case. Feature bits do not
> > >>> describe device capabilities fully, some of them are in config space.
>
> Yes.
>
> > >> To be precise, this does not necessarily require introspection, but
> > >> it does require management control over config space
> > >> such as supported hash types just like it has control over feature bits.
> > >> E.g. QEMU currently seems to hard-code these to
> > >> #define VIRTIO_NET_RSS_SUPPORTED_HASHES (VIRTIO_NET_RSS_HASH_TYPE_IPv4 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_TCPv4 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_IPv6 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_TCPv6 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_UDPv6 | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_IP_EX | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
> > >> VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
> > >>
> > >> but there's no reason not to give management control over these.
>
> Note that the management expects the migration compatibility to work
> with machine types. So it needs a way to disable some tunnel hash
> types to make it work for old machine types.
yes.
This means qemu will need to create properties for these things
and control through machine type compatibility machinery.
For those not hacking qemu - "machine type" is
a string roughly describing a version of guest/host interface used.
> > > Yes, QEMU has requirements for live migration: the PCI config space will be
> > > checked in get_pci_config_device(), and if src and dst are inconsistent, it
> > > will prompt that the live migration failed.
>
> It might be too late since it can't work for the second run (unlike subsection).
This is really a low level detail of qemu. I'm not sure how important
this is for the spec.
> >
> > To be clearer, I mean \filed{supported_hash_types} in structure
> > virtio_net_config.
>
> Yes.
>
> Thanks
>
> >
> > Thanks.
> >
> > > In fact, this is also done within our group. Live migration requires that
> > > the two VMs have the same rss configuration, otherwise the migration will fail.
> > >
> > > Therefore, it seems that we can regularize the description of VIRTIO_NET_F_HASH_TUNNEL into
> > > "[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash for tunnel-encapsulated packets.",
> > > and use different hash_types to help the migration determine whether it can succeed.
> > >
> > > Thanks.
> > >
> > >> --
> > >> MST
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
> >
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-11 9:45 ` Michael S. Tsirkin
@ 2023-01-16 8:01 ` Heng Qi
2023-01-16 8:18 ` [virtio-dev] " Cornelia Huck
2023-01-16 8:42 ` Jason Wang
0 siblings, 2 replies; 54+ messages in thread
From: Heng Qi @ 2023-01-16 8:01 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> > If the tunnel is used to encapsulate the packets, the hash calculated
> > using the outer header of the receive packets is always fixed for the
> > same flow packets, i.e. they will be steered to the same receive queue.
> >
> > We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> > in \field{hash_types}, which instructs the device to calculate the
> > hash using the inner headers of tunnel-encapsulated packets. Besides,
> > values in \field{hash_report_tunnel} are added to report tunnel types.
> >
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> >
> > Reviewed-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > ---
> > v6:
> > 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> > 2. Fix some syntax issues. @Michael S. Tsirkin
> >
> > v5:
> > 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> > 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> > 3. Move the links to introduction section. @Michael S. Tsirkin
> > 4. Clarify some sentences. @Michael S. Tsirkin
> >
> > v4:
> > 1. Clarify some paragraphs. @Cornelia Huck
> > 2. Fix the u8 type. @Cornelia Huck
> >
> > v3:
> > 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> > 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> > 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> > 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> >
> > v2:
> > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> >
> > v1:
> > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > 2. Clarify some paragraphs. @Jason Wang
> > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> >
> > content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> > introduction.tex | 19 +++++
> > 2 files changed, 203 insertions(+), 7 deletions(-)
> >
> > diff --git a/content.tex b/content.tex
> > index e863709..7845f6c 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > channel.
> >
> > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > +
> > \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
> >
> > \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> > @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > to several segments when each of these smaller packets has UDP header.
> >
> > \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> > - value and a type of calculated hash.
> > + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> > + is negotiated, an encapsulation packet type.
> >
> > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> > value. Device benefits from knowing the exact header length.
> > @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> > +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> > \end{description}
> >
> > \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> > le16 num_buffers;
> > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>
> of -> if?
Sorry for the late reply.
It's Cornelia's suggestion, and 'of' seems to work fine.
>
> also let's add: otherwise reserved?
Sure.
>
>
> > + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > };
> > \end{lstlisting}
> >
> > @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > A device attempts to calculate a per-packet hash in the following cases:
> > \begin{itemize}
> > \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> > -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> > +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> > +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> > \end{itemize}
> >
> > If the feature VIRTIO_NET_F_RSS was negotiated:
> > @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> > \end{itemize}
> >
> > +\subparagraph{Tunnel/Encapsulated packet}
> > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> > +A tunnel packet is encapsulated from the original packet based on the tunneling
> > +protocol (only a single level of encapsulation is currently supported). The
> > +encapsulated packet contains an outer header and an inner header, and the device
> > +calculates the hash over either the inner header or the outer header.
> > +
> > +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> > +encapsulation type is set in \field{hash_types}, the hash for a specific type of
> > +encapsulated packet is calculated over the inner as opposed to outer header.
>
>
> > +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> > +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> > +Supported/enabled hash types}.
> > +
> > +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> > +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> > +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> > +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> > +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> > +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> > +
> > +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> > +negotiated, the device calculates the hash over the outer header, and
> > +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> > +is no longer valid.
>
> Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER
> must not be set without VIRTIO_NET_F_HASH_TUNNEL?
>
Yes.
> If yes then we can add conformance statements to this end and then
> drop the
> "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
> all over the place just relying on hash type being set instead.
Please see the response below.
>
> Also, note that it has to be ok for device to expose hash types without
> negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
> Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL
> just add hash_report_tunnel and have hash types speak for themselves?
> It makes things nicely orthogonal in that if one does not care
> about knowing encapsulation type (e.g. for RSS) one can disable
> VIRTIO_NET_F_HASH_TUNNEL.
This seems to have the following 3 problems:
1. This will break the specification. As described in the specification, any field
in the configuration space only exists when there is a corresponding feature bit:
"Device configuration space is generally used for rarely-changing or initialization-time parameters.
Where configuration fields are optional, their existence is indicated by feature bits: ",
which also applies to supported_hash_types.
2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation
hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case,
we only use encapsulation hash types to provide guidance for hash calculation, but not
necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).
3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL
feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is
negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add
hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash
types are present.
So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used
to declare that the device supports inner header hash and let the corresponding encapsulation
hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the
receive queue. Considering the migration at the same time, if the dst device does not negotiate
VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to
determine whether the migration can succeed.
Thanks.
> Objections? Jason?
>
>
> > +
> > \subparagraph{Supported/enabled hash types}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> > +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> > +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> > Hash types applicable for IPv4 packets:
> > \begin{lstlisting}
> > #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> > @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> > \end{lstlisting}
> >
> > +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> > +hash type below indicates that the hash is calculated over the inner
> > +header of the encapsulated packet:
> > +Hash type applicable for inner payload of the gre-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> > +\end{lstlisting}
> > +Hash type applicable for inner payload of the vxlan-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> > +\end{lstlisting}
> > +Hash type applicable for inner payload of the geneve-encapsulated packet
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> > +\end{lstlisting}
> > +
> > \subparagraph{IPv4 packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> > The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> > @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> > \end{itemize}
> >
> > +\subparagraph{Inner payload of an encapsulated packet}
> > +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> > +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> > +encapsulation hash type is set in \field{hash_types}, the device calculates the
> > +inner header hash on an encapsulated packet (See \ref{sec:Device Types
> > +/ Network Device / Device Operation / Processing of Incoming Packets /
> > +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> > +
> > +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> > +according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> > + TCPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> > + inner UDPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet without an extension header
> > +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> > + inner UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet with an extension header
> > +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemsize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> > + UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> > + following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6 destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \end{itemize}
> > + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> > + defined for an encapsulated packet whose inner payload is an IPv6 packet without
> > + an extension header.
> > +\end{itemsize}
> > +
> > \paragraph{Hash reporting for incoming packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
> >
> > -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> > - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> > -and \field{hash_value} with the value of calculated hash.
> > +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> > +hash for the packet, the device fills \field{hash_report} with the report type
> > +of calculated hash, and \field{hash_value} with the value of calculated hash.
> > +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
> > +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> > +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
> >
> > If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> > -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> > +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> > +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
> >
> > Possible values that the device can report in \field{hash_report} are defined below.
> > They correspond to supported hash types defined in
> > @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> > \end{lstlisting}
> >
> > +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> > +\field{hash_report_tunnel} can report the type of the encapsulated
> > +packet to the driver over the inner header hash calculation.
> > +Possible values that the device can report in \field{hash_report_tunnel}
> > +are defined below:
> > +
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> > +#define VIRTIO_NET_HASH_REPORT_GRE 1
> > +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> > +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> > +\end{lstlisting}
> > +
> > +They correspond to supported hash types defined in
> > +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> > +
> > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
> >
> > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> > diff --git a/introduction.tex b/introduction.tex
> > index 287c5fc..ff01a9b 100644
> > --- a/introduction.tex
> > +++ b/introduction.tex
> > @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> > \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> > Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> > \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> > + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> > + Generic Routing Encapsulation
> > + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> > + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> > + Virtual eXtensible Local Area Network
> > + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> > + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> > + Generic Network Virtualization Encapsulation
> > + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> > + \phantomsection\label{intro:IP}\textbf{[IP]} &
> > + INTERNET PROTOCOL
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> > + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> > + User Datagram Protocol
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> > + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> > + TRANSMISSION CONTROL PROTOCOL
> > + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> > +
> >
> > \end{longtable}
> >
> > --
> > 2.19.1.6.gb485710b
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-16 8:01 ` [virtio-comment] " Heng Qi
@ 2023-01-16 8:18 ` Cornelia Huck
2023-01-31 5:31 ` Heng Qi
2023-01-16 8:42 ` Jason Wang
1 sibling, 1 reply; 54+ messages in thread
From: Cornelia Huck @ 2023-01-16 8:18 UTC (permalink / raw)
To: Heng Qi, Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich, Xuan Zhuo
On Mon, Jan 16 2023, Heng Qi <hengqi@linux.alibaba.com> wrote:
> On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
>> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
>> > @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>> > le16 num_buffers;
>> > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> > - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> > + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>>
>> of -> if?
>
> Sorry for the late reply.
> It's Cornelia's suggestion, and 'of' seems to work fine.
I think that was simply a typo, should have been 'if'.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-16 8:01 ` [virtio-comment] " Heng Qi
2023-01-16 8:18 ` [virtio-dev] " Cornelia Huck
@ 2023-01-16 8:42 ` Jason Wang
2023-01-31 5:28 ` [virtio-dev] " Heng Qi
1 sibling, 1 reply; 54+ messages in thread
From: Jason Wang @ 2023-01-16 8:42 UTC (permalink / raw)
To: Heng Qi, Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/1/16 16:01, Heng Qi 写道:
> On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
>> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
>>> If the tunnel is used to encapsulate the packets, the hash calculated
>>> using the outer header of the receive packets is always fixed for the
>>> same flow packets, i.e. they will be steered to the same receive queue.
>>>
>>> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
>>> in \field{hash_types}, which instructs the device to calculate the
>>> hash using the inner headers of tunnel-encapsulated packets. Besides,
>>> values in \field{hash_report_tunnel} are added to report tunnel types.
>>>
>>> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>>>
>>> Reviewed-by: Jason Wang <jasowang@redhat.com>
>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> v6:
>>> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
>>> 2. Fix some syntax issues. @Michael S. Tsirkin
>>>
>>> v5:
>>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
>>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
>>> 3. Move the links to introduction section. @Michael S. Tsirkin
>>> 4. Clarify some sentences. @Michael S. Tsirkin
>>>
>>> v4:
>>> 1. Clarify some paragraphs. @Cornelia Huck
>>> 2. Fix the u8 type. @Cornelia Huck
>>>
>>> v3:
>>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
>>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
>>> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
>>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>>>
>>> v2:
>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>>>
>>> v1:
>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>> 2. Clarify some paragraphs. @Jason Wang
>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>>>
>>> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
>>> introduction.tex | 19 +++++
>>> 2 files changed, 203 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/content.tex b/content.tex
>>> index e863709..7845f6c 100644
>>> --- a/content.tex
>>> +++ b/content.tex
>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>>> channel.
>>>
>>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
>>> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
>>> +
>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>>>
>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>> to several segments when each of these smaller packets has UDP header.
>>>
>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>>> - value and a type of calculated hash.
>>> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
>>> + is negotiated, an encapsulation packet type.
>>>
>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>>> value. Device benefits from knowing the exact header length.
>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
>>> \end{description}
>>>
>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>>> le16 num_buffers;
>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>> of -> if?
> Sorry for the late reply.
> It's Cornelia's suggestion, and 'of' seems to work fine.
>
>> also let's add: otherwise reserved?
> Sure.
>
>>
>>> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>> };
>>> \end{lstlisting}
>>>
>>> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>> A device attempts to calculate a per-packet hash in the following cases:
>>> \begin{itemize}
>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
>>> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
>>> \end{itemize}
>>>
>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
>>> \end{itemize}
>>>
>>> +\subparagraph{Tunnel/Encapsulated packet}
>>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
>>> +A tunnel packet is encapsulated from the original packet based on the tunneling
>>> +protocol (only a single level of encapsulation is currently supported). The
>>> +encapsulated packet contains an outer header and an inner header, and the device
>>> +calculates the hash over either the inner header or the outer header.
>>> +
>>> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>>> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
>>> +encapsulated packet is calculated over the inner as opposed to outer header.
>>
>>> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
>>> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
>>> +Supported/enabled hash types}.
>>> +
>>> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
>>> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
>>> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
>>> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
>>> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
>>> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
>>> +
>>> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
>>> +negotiated, the device calculates the hash over the outer header, and
>>> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
>>> +is no longer valid.
>> Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER
>> must not be set without VIRTIO_NET_F_HASH_TUNNEL?
>>
> Yes.
>
>> If yes then we can add conformance statements to this end and then
>> drop the
>> "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
>> all over the place just relying on hash type being set instead.
> Please see the response below.
>
>> Also, note that it has to be ok for device to expose hash types without
>> negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
>> Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL
>> just add hash_report_tunnel and have hash types speak for themselves?
>> It makes things nicely orthogonal in that if one does not care
>> about knowing encapsulation type (e.g. for RSS) one can disable
>> VIRTIO_NET_F_HASH_TUNNEL.
> This seems to have the following 3 problems:
>
> 1. This will break the specification. As described in the specification, any field
> in the configuration space only exists when there is a corresponding feature bit:
> "Device configuration space is generally used for rarely-changing or initialization-time parameters.
> Where configuration fields are optional, their existence is indicated by feature bits: ",
> which also applies to supported_hash_types.
>
> 2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
> VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation
> hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case,
> we only use encapsulation hash types to provide guidance for hash calculation, but not
> necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).
>
> 3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL
> feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is
> negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add
> hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash
> types are present.
>
> So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used
> to declare that the device supports inner header hash and let the corresponding encapsulation
> hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the
> receive queue. Considering the migration at the same time, if the dst device does not negotiate
> VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to
> determine whether the migration can succeed.
I think it might be better to have consistency:
1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet header, is
it better to have a new config filed in the config space?
or
2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file, should
we try to reuse hash_report?
1) seems better and cleaner to me.
More below
>
> Thanks.
>
>> Objections? Jason?
>>
>>
>>> +
>>> \subparagraph{Supported/enabled hash types}
>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
>>> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
>>> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
>>> Hash types applicable for IPv4 packets:
>>> \begin{lstlisting}
>>> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
>>> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>>> \end{lstlisting}
>>>
>>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
>>> +hash type below indicates that the hash is calculated over the inner
>>> +header of the encapsulated packet:
>>> +Hash type applicable for inner payload of the gre-encapsulated packet
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>>> +\end{lstlisting}
>>> +Hash type applicable for inner payload of the vxlan-encapsulated packet
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>>> +\end{lstlisting}
>>> +Hash type applicable for inner payload of the geneve-encapsulated packet
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>>> +\end{lstlisting}
Just notice that this seems to break the equation:
VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)
?
Thanks
>>> +
>>> \subparagraph{IPv4 packets}
>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
>>> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
>>> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
>>> \end{itemize}
>>>
>>> +\subparagraph{Inner payload of an encapsulated packet}
>>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
>>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>>> +encapsulation hash type is set in \field{hash_types}, the device calculates the
>>> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
>>> +/ Network Device / Device Operation / Processing of Incoming Packets /
>>> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
>>> +
>>> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
>>> +according to 'Enabled hash types' bitmask as follows:
>>> +\begin{itemize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
>>> + TCPv4 header, the hash is calculated over the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemsize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
>>> + inner UDPv4 header, the hash is calculated over the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \end{itemsize}
>>> + \item Else the device does not calculate the hash
>>> +\end{itemize}
>>> +
>>> +The device calculates the hash on the inner IPv6 packet without an extension header
>>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>>> +\begin{itemize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
>>> + TCPv6 header, the hash is calculated over the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemsize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
>>> + inner UDPv6 header, the hash is calculated over the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \end{itemsize}
>>> + \item Else the device does not calculate the hash
>>> +\end{itemize}
>>> +
>>> +The device calculates the hash on the inner IPv6 packet with an extension header
>>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>>> +\begin{itemsize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
>>> + TCPv6 header, the hash is calculated over the following fields:
>>> + \begin{itemize}
>>> + \item Home address from the home address option in the inner IPv6 destination
>>> + options header. If the inner extension header is not present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
>>> + UDPv6 header, the hash is calculated over the following fields:
>>> + \begin{itemsize}
>>> + \item Home address from the home address option in the inner IPv6 destination
>>> + options header. If the inner extension header is not present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item Home address from the home address option in the inner IPv6 destination
>>> + options header. If the inner extension header is not present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \end{itemize}
>>> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
>>> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
>>> + an extension header.
>>> +\end{itemsize}
>>> +
>>> \paragraph{Hash reporting for incoming packets}
>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>>>
>>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>>> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
>>> -and \field{hash_value} with the value of calculated hash.
>>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
>>> +hash for the packet, the device fills \field{hash_report} with the report type
>>> +of calculated hash, and \field{hash_value} with the value of calculated hash.
>>> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
>>> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
>>> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>>>
>>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>>> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
>>> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
>>> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>>>
>>> Possible values that the device can report in \field{hash_report} are defined below.
>>> They correspond to supported hash types defined in
>>> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>>> \end{lstlisting}
>>>
>>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>>> +\field{hash_report_tunnel} can report the type of the encapsulated
>>> +packet to the driver over the inner header hash calculation.
>>> +Possible values that the device can report in \field{hash_report_tunnel}
>>> +are defined below:
>>> +
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
>>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>>> +\end{lstlisting}
>>> +
>>> +They correspond to supported hash types defined in
>>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>>> +
>>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>>
>>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>>> diff --git a/introduction.tex b/introduction.tex
>>> index 287c5fc..ff01a9b 100644
>>> --- a/introduction.tex
>>> +++ b/introduction.tex
>>> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
>>> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
>>> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
>>> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>>> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
>>> + Generic Routing Encapsulation
>>> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
>>> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
>>> + Virtual eXtensible Local Area Network
>>> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
>>> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
>>> + Generic Network Virtualization Encapsulation
>>> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
>>> + \phantomsection\label{intro:IP}\textbf{[IP]} &
>>> + INTERNET PROTOCOL
>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
>>> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
>>> + User Datagram Protocol
>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
>>> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
>>> + TRANSMISSION CONTROL PROTOCOL
>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
>>> +
>>>
>>> \end{longtable}
>>>
>>> --
>>> 2.19.1.6.gb485710b
>>
>> This publicly archived list offers a means to provide input to the
>> OASIS Virtual I/O Device (VIRTIO) TC.
>>
>> In order to verify user consent to the Feedback License terms and
>> to minimize spam in the list archive, subscription is required
>> before posting.
>>
>> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
>> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
>> List help: virtio-comment-help@lists.oasis-open.org
>> List archive: https://lists.oasis-open.org/archives/virtio-comment/
>> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
>> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
>> Committee: https://www.oasis-open.org/committees/virtio/
>> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
` (4 preceding siblings ...)
2023-01-11 9:45 ` Michael S. Tsirkin
@ 2023-01-18 23:45 ` Parav Pandit
2023-01-31 5:57 ` [virtio-comment] " Heng Qi
5 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-01-18 23:45 UTC (permalink / raw)
To: Heng Qi, virtio-comment, virtio-dev
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org>
> Sent: Wednesday, January 4, 2023 2:14 AM
> If the tunnel is used to encapsulate the packets, the hash calculated using the
> outer header of the receive packets is always fixed for the same flow packets,
> i.e. they will be steered to the same receive queue.
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> +
A device may not support all 3 at the same time.
Please remove mentioning tunneling protocols description from here.
Just say device support inner header hash ...
An additional bit map somewhere else should say supported hash over different tunneling types.
[...]
> +The device calculates the hash on the inner IPv4 packet of an
> +encapsulated packet according to 'Enabled hash types' bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet
> has an inner
> + TCPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated
> packet has an
> + inner UDPv4 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over
> the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash \end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet without an
> +extension header of an encapsulated packet according to 'Enabled hash types'
> bitmask as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet
> has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated
> packet has an
> + inner UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over
> the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash \end{itemize}
> +
> +The device calculates the hash on the inner IPv6 packet with an
> +extension header of an encapsulated packet according to 'Enabled hash types'
> bitmask as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet
> has an inner
> + TCPv6 header, the hash is calculated over the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6
> destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated
> packet has an inner
> + UDPv6 header, the hash is calculated over the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6
> destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated
> over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6
> destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip the inner IPv6 extension header and calculate the inner
> header hash as
> + defined for an encapsulated packet whose inner payload is an IPv6
> packet without
> + an extension header.
> +\end{itemsize}
> +
With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel.
hash tunnel capability only indicates that hashing is done on the inner packets as_before.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-16 8:42 ` Jason Wang
@ 2023-01-31 5:28 ` Heng Qi
2023-02-08 2:30 ` Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-31 5:28 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo, Parav Pandit
On Mon, Jan 16, 2023 at 04:42:11PM +0800, Jason Wang wrote:
>
> 在 2023/1/16 16:01, Heng Qi 写道:
> >On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
> >>On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> >>>If the tunnel is used to encapsulate the packets, the hash calculated
> >>>using the outer header of the receive packets is always fixed for the
> >>>same flow packets, i.e. they will be steered to the same receive queue.
> >>>
> >>>We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
> >>>in \field{hash_types}, which instructs the device to calculate the
> >>>hash using the inner headers of tunnel-encapsulated packets. Besides,
> >>>values in \field{hash_report_tunnel} are added to report tunnel types.
> >>>
> >>>Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
> >>>
> >>>Reviewed-by: Jason Wang <jasowang@redhat.com>
> >>>Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> >>>Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >>>---
> >>>v6:
> >>> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> >>> 2. Fix some syntax issues. @Michael S. Tsirkin
> >>>
> >>>v5:
> >>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> >>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> >>> 3. Move the links to introduction section. @Michael S. Tsirkin
> >>> 4. Clarify some sentences. @Michael S. Tsirkin
> >>>
> >>>v4:
> >>> 1. Clarify some paragraphs. @Cornelia Huck
> >>> 2. Fix the u8 type. @Cornelia Huck
> >>>
> >>>v3:
> >>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> >>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> >>> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> >>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
> >>>
> >>>v2:
> >>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> >>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> >>>
> >>>v1:
> >>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> >>> 2. Clarify some paragraphs. @Jason Wang
> >>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> >>>
> >>> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
> >>> introduction.tex | 19 +++++
> >>> 2 files changed, 203 insertions(+), 7 deletions(-)
> >>>
> >>>diff --git a/content.tex b/content.tex
> >>>index e863709..7845f6c 100644
> >>>--- a/content.tex
> >>>+++ b/content.tex
> >>>@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> >>> channel.
> >>>+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> >>>+ header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> >>>+
> >>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
> >>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> >>>@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >>> to several segments when each of these smaller packets has UDP header.
> >>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> >>>- value and a type of calculated hash.
> >>>+ value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
> >>>+ is negotiated, an encapsulation packet type.
> >>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> >>> value. Device benefits from knowing the exact header length.
> >>>@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> >>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >>>+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>> \end{description}
> >>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> >>>@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> >>> le16 num_buffers;
> >>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >>>- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >>>+ u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> >>of -> if?
> >Sorry for the late reply.
> >It's Cornelia's suggestion, and 'of' seems to work fine.
> >
> >>also let's add: otherwise reserved?
> >Sure.
> >
> >>
> >>>+ u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >>> };
> >>> \end{lstlisting}
> >>>@@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> >>> A device attempts to calculate a per-packet hash in the following cases:
> >>> \begin{itemize}
> >>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> >>>-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> >>>+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
> >>>+VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
> >>> \end{itemize}
> >>> If the feature VIRTIO_NET_F_RSS was negotiated:
> >>>@@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> >>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> >>> \end{itemize}
> >>>+\subparagraph{Tunnel/Encapsulated packet}
> >>>+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> >>>+A tunnel packet is encapsulated from the original packet based on the tunneling
> >>>+protocol (only a single level of encapsulation is currently supported). The
> >>>+encapsulated packet contains an outer header and an inner header, and the device
> >>>+calculates the hash over either the inner header or the outer header.
> >>>+
> >>>+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> >>>+encapsulation type is set in \field{hash_types}, the hash for a specific type of
> >>>+encapsulated packet is calculated over the inner as opposed to outer header.
> >>
> >>>+Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
> >>>+Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
> >>>+Supported/enabled hash types}.
> >>>+
> >>>+If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
> >>>+the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
> >>>+\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
> >>>+encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
> >>>+is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
> >>>+\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
> >>>+
> >>>+If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
> >>>+negotiated, the device calculates the hash over the outer header, and
> >>>+\field{hash_report} reports the hash type. \field{hash_report_tunnel}
> >>>+is no longer valid.
> >>Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER
> >>must not be set without VIRTIO_NET_F_HASH_TUNNEL?
> >>
> >Yes.
> >
> >>If yes then we can add conformance statements to this end and then
> >>drop the
> >>"If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
> >>all over the place just relying on hash type being set instead.
> >Please see the response below.
> >
> >>Also, note that it has to be ok for device to expose hash types without
> >>negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
> >>Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL
> >>just add hash_report_tunnel and have hash types speak for themselves?
> >>It makes things nicely orthogonal in that if one does not care
> >>about knowing encapsulation type (e.g. for RSS) one can disable
> >>VIRTIO_NET_F_HASH_TUNNEL.
> >This seems to have the following 3 problems:
> >
> >1. This will break the specification. As described in the specification, any field
> > in the configuration space only exists when there is a corresponding feature bit:
> > "Device configuration space is generally used for rarely-changing or initialization-time parameters.
> > Where configuration fields are optional, their existence is indicated by feature bits: ",
> > which also applies to supported_hash_types.
> >
> >2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
> > VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation
> > hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case,
> > we only use encapsulation hash types to provide guidance for hash calculation, but not
> > necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).
> >
> >3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL
> > feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is
> > negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add
> > hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash
> > types are present.
> >
> >So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used
> >to declare that the device supports inner header hash and let the corresponding encapsulation
> >hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the
> >receive queue. Considering the migration at the same time, if the dst device does not negotiate
> >VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to
> >determine whether the migration can succeed.
>
>
> I think it might be better to have consistency:
>
> 1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet
> header, is it better to have a new config filed in the config space?
>
> or
>
> 2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file,
> should we try to reuse hash_report?
>
> 1) seems better and cleaner to me.
Sorry for the late reply due to vacation.
Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to
control all information related to inner header hash.
>
> More below
>
>
> >
> >Thanks.
> >
> >>Objections? Jason?
> >>
> >>
> >>>+
> >>> \subparagraph{Supported/enabled hash types}
> >>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> >>>+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> >>>+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> >>> Hash types applicable for IPv4 packets:
> >>> \begin{lstlisting}
> >>> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> >>>@@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> >>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> >>> \end{lstlisting}
> >>>+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> >>>+hash type below indicates that the hash is calculated over the inner
> >>>+header of the encapsulated packet:
> >>>+Hash type applicable for inner payload of the gre-encapsulated packet
> >>>+\begin{lstlisting}
> >>>+#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> >>>+\end{lstlisting}
> >>>+Hash type applicable for inner payload of the vxlan-encapsulated packet
> >>>+\begin{lstlisting}
> >>>+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> >>>+\end{lstlisting}
> >>>+Hash type applicable for inner payload of the geneve-encapsulated packet
> >>>+\begin{lstlisting}
> >>>+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> >>>+\end{lstlisting}
>
>
> Just notice that this seems to break the equation:
>
> VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)
>
> ?
>
Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html ,
as you suggested above, it seems better to use the new config space field.
Thanks.
> Thanks
>
>
> >>>+
> >>> \subparagraph{IPv4 packets}
> >>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> >>> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> >>>@@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> >>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> >>> \end{itemize}
> >>>+\subparagraph{Inner payload of an encapsulated packet}
> >>>+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
> >>>+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> >>>+encapsulation hash type is set in \field{hash_types}, the device calculates the
> >>>+inner header hash on an encapsulated packet (See \ref{sec:Device Types
> >>>+/ Network Device / Device Operation / Processing of Incoming Packets /
> >>>+Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
> >>>+
> >>>+The device calculates the hash on the inner IPv4 packet of an encapsulated packet
> >>>+according to 'Enabled hash types' bitmask as follows:
> >>>+\begin{itemize}
> >>>+ \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
> >>>+ TCPv4 header, the hash is calculated over the following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IP address
> >>>+ \item inner Destination IP address
> >>>+ \item inner Source TCP port
> >>>+ \item inner Destination TCP port
> >>>+ \end{itemsize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
> >>>+ inner UDPv4 header, the hash is calculated over the following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IP address
> >>>+ \item inner Destination IP address
> >>>+ \item inner Source UDP port
> >>>+ \item inner Destination UDP port
> >>>+ \end{itemize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
> >>>+ following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IP address
> >>>+ \item inner Destination IP address
> >>>+ \end{itemsize}
> >>>+ \item Else the device does not calculate the hash
> >>>+\end{itemize}
> >>>+
> >>>+The device calculates the hash on the inner IPv6 packet without an extension header
> >>>+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> >>>+\begin{itemize}
> >>>+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
> >>>+ TCPv6 header, the hash is calculated over the following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IPv6 address
> >>>+ \item inner Destination IPv6 address
> >>>+ \item inner Source TCP port
> >>>+ \item inner Destination TCP port
> >>>+ \end{itemsize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
> >>>+ inner UDPv6 header, the hash is calculated over the following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IPv6 address
> >>>+ \item inner Destination IPv6 address
> >>>+ \item inner Source UDP port
> >>>+ \item inner Destination UDP port
> >>>+ \end{itemize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
> >>>+ following fields:
> >>>+ \begin{itemsize}
> >>>+ \item inner Source IPv6 address
> >>>+ \item inner Destination IPv6 address
> >>>+ \end{itemsize}
> >>>+ \item Else the device does not calculate the hash
> >>>+\end{itemize}
> >>>+
> >>>+The device calculates the hash on the inner IPv6 packet with an extension header
> >>>+of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
> >>>+\begin{itemsize}
> >>>+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
> >>>+ TCPv6 header, the hash is calculated over the following fields:
> >>>+ \begin{itemize}
> >>>+ \item Home address from the home address option in the inner IPv6 destination
> >>>+ options header. If the inner extension header is not present, use the
> >>>+ inner Source IPv6 address.
> >>>+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >>>+ associated inner extension header. If the inner extension header is not
> >>>+ present, use the inner Destination IPv6 address.
> >>>+ \item inner Source TCP port
> >>>+ \item inner Destination TCP port
> >>>+ \end{itemize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
> >>>+ UDPv6 header, the hash is calculated over the following fields:
> >>>+ \begin{itemsize}
> >>>+ \item Home address from the home address option in the inner IPv6 destination
> >>>+ options header. If the inner extension header is not present, use the
> >>>+ inner Source IPv6 address.
> >>>+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >>>+ associated inner extension header. If the inner extension header is not
> >>>+ present, use the inner Destination IPv6 address.
> >>>+ \item inner Source UDP port
> >>>+ \item inner Destination UDP port
> >>>+ \end{itemize}
> >>>+ \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
> >>>+ following fields:
> >>>+ \begin{itemsize}
> >>>+ \item Home address from the home address option in the inner IPv6 destination
> >>>+ options header. If the inner extension header is not present, use the
> >>>+ inner Source IPv6 address.
> >>>+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >>>+ associated inner extension header. If the inner extension header is not
> >>>+ present, use the inner Destination IPv6 address.
> >>>+ \end{itemize}
> >>>+ \item Else skip the inner IPv6 extension header and calculate the inner header hash as
> >>>+ defined for an encapsulated packet whose inner payload is an IPv6 packet without
> >>>+ an extension header.
> >>>+\end{itemsize}
> >>>+
> >>> \paragraph{Hash reporting for incoming packets}
> >>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
> >>>-If VIRTIO_NET_F_HASH_REPORT was negotiated and
> >>>- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> >>>-and \field{hash_value} with the value of calculated hash.
> >>>+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> >>>+hash for the packet, the device fills \field{hash_report} with the report type
> >>>+of calculated hash, and \field{hash_value} with the value of calculated hash.
> >>>+Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
> >>>+\field{hash_report_tunnel} with the report type of the encapsulated packet, and
> >>>+it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
> >>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> >>>-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> >>>+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> >>>+and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
> >>> Possible values that the device can report in \field{hash_report} are defined below.
> >>> They correspond to supported hash types defined in
> >>>@@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> >>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> >>> \end{lstlisting}
> >>>+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> >>>+\field{hash_report_tunnel} can report the type of the encapsulated
> >>>+packet to the driver over the inner header hash calculation.
> >>>+Possible values that the device can report in \field{hash_report_tunnel}
> >>>+are defined below:
> >>>+
> >>>+\begin{lstlisting}
> >>>+#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
> >>>+#define VIRTIO_NET_HASH_REPORT_GRE 1
> >>>+#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> >>>+#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> >>>+\end{lstlisting}
> >>>+
> >>>+They correspond to supported hash types defined in
> >>>+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> >>>+
> >>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
> >>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> >>>diff --git a/introduction.tex b/introduction.tex
> >>>index 287c5fc..ff01a9b 100644
> >>>--- a/introduction.tex
> >>>+++ b/introduction.tex
> >>>@@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
> >>> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
> >>> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> >>> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
> >>>+ \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> >>>+ Generic Routing Encapsulation
> >>>+ \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> >>>+ \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> >>>+ Virtual eXtensible Local Area Network
> >>>+ \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> >>>+ \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> >>>+ Generic Network Virtualization Encapsulation
> >>>+ \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> >>>+ \phantomsection\label{intro:IP}\textbf{[IP]} &
> >>>+ INTERNET PROTOCOL
> >>>+ \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> >>>+ \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> >>>+ User Datagram Protocol
> >>>+ \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> >>>+ \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> >>>+ TRANSMISSION CONTROL PROTOCOL
> >>>+ \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> >>>+
> >>> \end{longtable}
> >>>--
> >>>2.19.1.6.gb485710b
> >>
> >>This publicly archived list offers a means to provide input to the
> >>OASIS Virtual I/O Device (VIRTIO) TC.
> >>
> >>In order to verify user consent to the Feedback License terms and
> >>to minimize spam in the list archive, subscription is required
> >>before posting.
> >>
> >>Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> >>Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> >>List help: virtio-comment-help@lists.oasis-open.org
> >>List archive: https://lists.oasis-open.org/archives/virtio-comment/
> >>Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> >>List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> >>Committee: https://www.oasis-open.org/committees/virtio/
> >>Join OASIS: https://www.oasis-open.org/join/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-16 8:18 ` [virtio-dev] " Cornelia Huck
@ 2023-01-31 5:31 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-01-31 5:31 UTC (permalink / raw)
To: Cornelia Huck
Cc: Jason Wang, Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Xuan Zhuo
On Mon, Jan 16, 2023 at 09:18:16AM +0100, Cornelia Huck wrote:
> On Mon, Jan 16 2023, Heng Qi <hengqi@linux.alibaba.com> wrote:
>
> > On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
> >> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> >> > @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> >> > le16 num_buffers;
> >> > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >> > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >> > - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >> > + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> >>
> >> of -> if?
> >
> > Sorry for the late reply.
> > It's Cornelia's suggestion, and 'of' seems to work fine.
>
> I think that was simply a typo, should have been 'if'.
Sorry for the late reply due to vacation.
It is my mistake, sorry.
Thanks.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-01-18 23:45 ` [virtio-dev] " Parav Pandit
@ 2023-01-31 5:57 ` Heng Qi
2023-02-01 1:51 ` Parav Pandit
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-01-31 5:57 UTC (permalink / raw)
To: Parav Pandit
Cc: Jason Wang, Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Jan 18, 2023 at 11:45:39PM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org>
> > Sent: Wednesday, January 4, 2023 2:14 AM
>
> > If the tunnel is used to encapsulate the packets, the hash calculated using the
> > outer header of the receive packets is always fixed for the same flow packets,
> > i.e. they will be steered to the same receive queue.
>
> > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > +
> A device may not support all 3 at the same time.
> Please remove mentioning tunneling protocols description from here.
> Just say device support inner header hash ...
Sorry for the late reply due to vacation.
Good idea, Michael suggested doing the same. But we also discussed this issue:
Early, we used a feature bit to force devices to support GRE and VXLAN (see
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00183.html ).
Later Jason suggested to use VIRTIO_NET_F_HASH_TUNNEL instead of VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER (see
https://lists.oasis-open.org/archives/virtio-dev/202212/msg00014.html ). Now Michael proposes to remove this list (see
https://lists.oasis-open.org/archives/virtio-dev/202301/msg00079.html ), because the migration is based on the feature
bit and hash types to determine whether the live migration is successful.
>
> An additional bit map somewhere else should say supported hash over different tunneling types.
>
Yes, we use \field{supported_hash_types} to declare supported hash types.
> [...]
>
> > +The device calculates the hash on the inner IPv4 packet of an
> > +encapsulated packet according to 'Enabled hash types' bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet
> > has an inner
> > + TCPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated
> > packet has an
> > + inner UDPv4 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over
> > the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IP address
> > + \item inner Destination IP address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash \end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet without an
> > +extension header of an encapsulated packet according to 'Enabled hash types'
> > bitmask as follows:
> > +\begin{itemize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet
> > has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemsize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated
> > packet has an
> > + inner UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over
> > the
> > + following fields:
> > + \begin{itemsize}
> > + \item inner Source IPv6 address
> > + \item inner Destination IPv6 address
> > + \end{itemsize}
> > + \item Else the device does not calculate the hash \end{itemize}
> > +
> > +The device calculates the hash on the inner IPv6 packet with an
> > +extension header of an encapsulated packet according to 'Enabled hash types'
> > bitmask as follows:
> > +\begin{itemsize}
> > + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet
> > has an inner
> > + TCPv6 header, the hash is calculated over the following fields:
> > + \begin{itemize}
> > + \item Home address from the home address option in the inner IPv6
> > destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> > the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source TCP port
> > + \item inner Destination TCP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated
> > packet has an inner
> > + UDPv6 header, the hash is calculated over the following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6
> > destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> > the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \item inner Source UDP port
> > + \item inner Destination UDP port
> > + \end{itemize}
> > + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated
> > over the
> > + following fields:
> > + \begin{itemsize}
> > + \item Home address from the home address option in the inner IPv6
> > destination
> > + options header. If the inner extension header is not present, use the
> > + inner Source IPv6 address.
> > + \item IPv6 address that is contained in the Routing-Header-Type-2 from
> > the
> > + associated inner extension header. If the inner extension header is not
> > + present, use the inner Destination IPv6 address.
> > + \end{itemize}
> > + \item Else skip the inner IPv6 extension header and calculate the inner
> > header hash as
> > + defined for an encapsulated packet whose inner payload is an IPv6
> > packet without
> > + an extension header.
> > +\end{itemsize}
> > +
>
> With the inclusion of tunnel outer header, it doesn't need to redefine the hashing for inner packets which is exactly same without the tunnel.
> hash tunnel capability only indicates that hashing is done on the inner packets as_before.
>
This seems like a trade-off, and I can get rid of this if the simple statement
"computing the hash for the inner header of the tunnel packet is the same as without tunnel"
is also clear to the reader.
What do you think? Cc Jason and Michael.
Thanks.
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-01-31 5:57 ` [virtio-comment] " Heng Qi
@ 2023-02-01 1:51 ` Parav Pandit
2023-02-01 6:47 ` Michael S. Tsirkin
` (2 more replies)
0 siblings, 3 replies; 54+ messages in thread
From: Parav Pandit @ 2023-02-01 1:51 UTC (permalink / raw)
To: Heng Qi
Cc: Jason Wang, Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Tuesday, January 31, 2023 12:57 AM
>
> On Wed, Jan 18, 2023 at 11:45:39PM +0000, Parav Pandit wrote:
> >
> >
> > > From: virtio-dev@lists.oasis-open.org
> > > <virtio-dev@lists.oasis-open.org>
> > > Sent: Wednesday, January 4, 2023 2:14 AM
> >
> > > If the tunnel is used to encapsulate the packets, the hash
> > > calculated using the outer header of the receive packets is always
> > > fixed for the same flow packets, i.e. they will be steered to the same
> receive queue.
> >
> > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > +
> > A device may not support all 3 at the same time.
> > Please remove mentioning tunneling protocols description from here.
> > Just say device support inner header hash ...
>
> Sorry for the late reply due to vacation.
>
> Good idea, Michael suggested doing the same. But we also discussed this issue:
> Early, we used a feature bit to force devices to support GRE and VXLAN
..
There is no need to force the device.
LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
> > An additional bit map somewhere else should say supported hash over
> different tunneling types.
> >
>
> Yes, we use \field{supported_hash_types} to declare supported hash types.
Nice. So yes, just remove the description from the feature bits.
[..]
> >
> > With the inclusion of tunnel outer header, it doesn't need to redefine the
> hashing for inner packets which is exactly same without the tunnel.
> > hash tunnel capability only indicates that hashing is done on the inner packets
> as_before.
> >
>
> This seems like a trade-off, and I can get rid of this if the simple statement
> "computing the hash for the inner header of the tunnel packet is the same as
> without tunnel"
> is also clear to the reader.
> What do you think? Cc Jason and Michael.
Not sure I follow your comment.
Inclusion of outer header has zero change in inner header hash.
The data structure etc is just fine in the patch you proposed.
So, there is no need to redefine all over it again.
Only description need to be simplified as below.
Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.
Also, this patch is adding two functionalities.
1. Inner header hash calculation of existing already defined hash types
2. outer header hash for new type for GRE,VXLAN,GENEVE.
#1 should be in 1st patch.
#2 should be in 2nd patch.
This is better to review.
Overall useful features. Thanks for adding.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-02-01 1:51 ` Parav Pandit
@ 2023-02-01 6:47 ` Michael S. Tsirkin
2023-02-01 6:56 ` Michael S. Tsirkin
2023-02-01 7:14 ` Heng Qi
2 siblings, 0 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-01 6:47 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 01, 2023 at 01:51:25AM +0000, Parav Pandit wrote:
>
>
> > From: Heng Qi <hengqi@linux.alibaba.com>
> > Sent: Tuesday, January 31, 2023 12:57 AM
> >
> > On Wed, Jan 18, 2023 at 11:45:39PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org>
> > > > Sent: Wednesday, January 4, 2023 2:14 AM
> > >
> > > > If the tunnel is used to encapsulate the packets, the hash
> > > > calculated using the outer header of the receive packets is always
> > > > fixed for the same flow packets, i.e. they will be steered to the same
> > receive queue.
> > >
> > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > +
> > > A device may not support all 3 at the same time.
> > > Please remove mentioning tunneling protocols description from here.
> > > Just say device support inner header hash ...
> >
> > Sorry for the late reply due to vacation.
> >
> > Good idea, Michael suggested doing the same. But we also discussed this issue:
> > Early, we used a feature bit to force devices to support GRE and VXLAN
> ..
> There is no need to force the device.
> LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
>
> > > An additional bit map somewhere else should say supported hash over
> > different tunneling types.
> > >
> >
> > Yes, we use \field{supported_hash_types} to declare supported hash types.
> Nice. So yes, just remove the description from the feature bits.
> [..]
> > >
> > > With the inclusion of tunnel outer header, it doesn't need to redefine the
> > hashing for inner packets which is exactly same without the tunnel.
> > > hash tunnel capability only indicates that hashing is done on the inner packets
> > as_before.
> > >
> >
> > This seems like a trade-off, and I can get rid of this if the simple statement
> > "computing the hash for the inner header of the tunnel packet is the same as
> > without tunnel"
> > is also clear to the reader.
> > What do you think? Cc Jason and Michael.
> Not sure I follow your comment.
> Inclusion of outer header has zero change in inner header hash.
>
> The data structure etc is just fine in the patch you proposed.
> So, there is no need to redefine all over it again.
> Only description need to be simplified as below.
> Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.
>
> Also, this patch is adding two functionalities.
> 1. Inner header hash calculation of existing already defined hash types
> 2. outer header hash for new type for GRE,VXLAN,GENEVE.
> #1 should be in 1st patch.
> #2 should be in 2nd patch.
> This is better to review.
Oh, that's an interesting take. Do you feel we need more
separate hash type bits then?
> Overall useful features. Thanks for adding.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-02-01 1:51 ` Parav Pandit
2023-02-01 6:47 ` Michael S. Tsirkin
@ 2023-02-01 6:56 ` Michael S. Tsirkin
2023-02-02 3:55 ` Parav Pandit
2023-02-01 7:14 ` Heng Qi
2 siblings, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-01 6:56 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 01, 2023 at 01:51:25AM +0000, Parav Pandit wrote:
>
>
> > From: Heng Qi <hengqi@linux.alibaba.com>
> > Sent: Tuesday, January 31, 2023 12:57 AM
> >
> > On Wed, Jan 18, 2023 at 11:45:39PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org>
> > > > Sent: Wednesday, January 4, 2023 2:14 AM
> > >
> > > > If the tunnel is used to encapsulate the packets, the hash
> > > > calculated using the outer header of the receive packets is always
> > > > fixed for the same flow packets, i.e. they will be steered to the same
> > receive queue.
> > >
> > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > +
> > > A device may not support all 3 at the same time.
> > > Please remove mentioning tunneling protocols description from here.
> > > Just say device support inner header hash ...
> >
> > Sorry for the late reply due to vacation.
> >
> > Good idea, Michael suggested doing the same. But we also discussed this issue:
> > Early, we used a feature bit to force devices to support GRE and VXLAN
> ..
> There is no need to force the device.
> LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
>
> > > An additional bit map somewhere else should say supported hash over
> > different tunneling types.
> > >
> >
> > Yes, we use \field{supported_hash_types} to declare supported hash types.
> Nice. So yes, just remove the description from the feature bits.
> [..]
> > >
> > > With the inclusion of tunnel outer header, it doesn't need to redefine the
> > hashing for inner packets which is exactly same without the tunnel.
> > > hash tunnel capability only indicates that hashing is done on the inner packets
> > as_before.
> > >
> >
> > This seems like a trade-off, and I can get rid of this if the simple statement
> > "computing the hash for the inner header of the tunnel packet is the same as
> > without tunnel"
> > is also clear to the reader.
> > What do you think? Cc Jason and Michael.
> Not sure I follow your comment.
> Inclusion of outer header has zero change in inner header hash.
>
> The data structure etc is just fine in the patch you proposed.
> So, there is no need to redefine all over it again.
> Only description need to be simplified as below.
> Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.
>
> Also, this patch is adding two functionalities.
> 1. Inner header hash calculation of existing already defined hash types
> 2. outer header hash for new type for GRE,VXLAN,GENEVE.
> #1 should be in 1st patch.
> #2 should be in 2nd patch.
> This is better to review.
Parav, you come to this discussion pretty late. Asking to split up
the patch when it's v1/v2 is ok. Asking after others have
already reviewed v6 is not you are making review easier
for yourself but re-review harder for others who already
have a mind map of the patch.
In this case unless we really want to enable these separately
(and frankly I don't see a good reason to) then
splitting it up makes review more confusing.
> Overall useful features. Thanks for adding.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-02-01 1:51 ` Parav Pandit
2023-02-01 6:47 ` Michael S. Tsirkin
2023-02-01 6:56 ` Michael S. Tsirkin
@ 2023-02-01 7:14 ` Heng Qi
2 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-02-01 7:14 UTC (permalink / raw)
To: Parav Pandit
Cc: Michael S. Tsirkin, Jason Wang, Cornelia Huck, virtio-comment,
virtio-dev, Xuan Zhuo
On Wed, Feb 01, 2023 at 01:51:25AM +0000, Parav Pandit wrote:
>
>
> > From: Heng Qi <hengqi@linux.alibaba.com>
> > Sent: Tuesday, January 31, 2023 12:57 AM
> >
> > On Wed, Jan 18, 2023 at 11:45:39PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org>
> > > > Sent: Wednesday, January 4, 2023 2:14 AM
> > >
> > > > If the tunnel is used to encapsulate the packets, the hash
> > > > calculated using the outer header of the receive packets is always
> > > > fixed for the same flow packets, i.e. they will be steered to the same
> > receive queue.
> > >
> > > > +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> > > > + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
> > > > +
> > > A device may not support all 3 at the same time.
> > > Please remove mentioning tunneling protocols description from here.
> > > Just say device support inner header hash ...
> >
> > Sorry for the late reply due to vacation.
> >
> > Good idea, Michael suggested doing the same. But we also discussed this issue:
> > Early, we used a feature bit to force devices to support GRE and VXLAN
> ..
> There is no need to force the device.
> LM will work as it will use both he values - the feature bit and the supported/negotiated hash type in the config space.
Yes, that's exactly what we do now.
>
> > > An additional bit map somewhere else should say supported hash over
> > different tunneling types.
> > >
> >
> > Yes, we use \field{supported_hash_types} to declare supported hash types.
> Nice. So yes, just remove the description from the feature bits.
The next version will see this modification.
> [..]
> > >
> > > With the inclusion of tunnel outer header, it doesn't need to redefine the
> > hashing for inner packets which is exactly same without the tunnel.
> > > hash tunnel capability only indicates that hashing is done on the inner packets
> > as_before.
> > >
> >
> > This seems like a trade-off, and I can get rid of this if the simple statement
> > "computing the hash for the inner header of the tunnel packet is the same as
> > without tunnel"
> > is also clear to the reader.
> > What do you think? Cc Jason and Michael.
> Not sure I follow your comment.
> Inclusion of outer header has zero change in inner header hash.
>
> The data structure etc is just fine in the patch you proposed.
> So, there is no need to redefine all over it again.
> Only description need to be simplified as below.
> Please just change the text wording to indicate that when tunnel feature is negotiated, and if the hashing is performed by the device on the inner header, hash_report_tunnel will contain the valid outer tunnel type information.
>
OK, I'll take your advice and make appropriate changes.
Thanks.
> Also, this patch is adding two functionalities.
> 1. Inner header hash calculation of existing already defined hash types
> 2. outer header hash for new type for GRE,VXLAN,GENEVE.
> #1 should be in 1st patch.
> #2 should be in 2nd patch.
> This is better to review.
>
> Overall useful features. Thanks for adding.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-02-01 6:56 ` Michael S. Tsirkin
@ 2023-02-02 3:55 ` Parav Pandit
2023-02-02 6:55 ` Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-02 3:55 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
> open.org> On Behalf Of Michael S. Tsirkin
> Sent: Wednesday, February 1, 2023 1:57 AM
> > Also, this patch is adding two functionalities.
> > 1. Inner header hash calculation of existing already defined hash
> > types 2. outer header hash for new type for GRE,VXLAN,GENEVE.
> > #1 should be in 1st patch.
> > #2 should be in 2nd patch.
> > This is better to review.
>
> Parav, you come to this discussion pretty late. Asking to split up the patch when
> it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are
> making review easier for yourself but re-review harder for others who already
> have a mind map of the patch.
>
> In this case unless we really want to enable these separately (and frankly I don't
> see a good reason to) then splitting it up makes review more confusing.
>
No. There is no need to enable it separately.
It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.
As you say, it has some history, I don't have strong opinion to split.
But going forward in subsequent work, it is better to see logical changes in multiple patches.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] [PATCH v7] virtio-net: support inner header hash
2023-02-02 3:55 ` Parav Pandit
@ 2023-02-02 6:55 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-02-02 6:55 UTC (permalink / raw)
To: Parav Pandit, Michael S. Tsirkin
Cc: Jason Wang, virtio-dev, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo, virtio-comment
在 2023/2/2 上午11:55, Parav Pandit 写道:
>> From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
>> open.org> On Behalf Of Michael S. Tsirkin
>> Sent: Wednesday, February 1, 2023 1:57 AM
>>> Also, this patch is adding two functionalities.
>>> 1. Inner header hash calculation of existing already defined hash
>>> types 2. outer header hash for new type for GRE,VXLAN,GENEVE.
>>> #1 should be in 1st patch.
>>> #2 should be in 2nd patch.
>>> This is better to review.
>> Parav, you come to this discussion pretty late. Asking to split up the patch when
>> it's v1/v2 is ok. Asking after others have already reviewed v6 is not you are
>> making review easier for yourself but re-review harder for others who already
>> have a mind map of the patch.
>>
>> In this case unless we really want to enable these separately (and frankly I don't
>> see a good reason to) then splitting it up makes review more confusing.
>>
> No. There is no need to enable it separately.
> It was hard to parse new inner type decoding addition which has close to zero relation to outer headers.
>
> As you say, it has some history, I don't have strong opinion to split.
> But going forward in subsequent work, it is better to see logical changes in multiple patches.
It seems that we don't need to emphasize the outer header hash, which is the
same behavior as usual when the inner header hash is not raised.
Thanks.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-01-31 5:28 ` [virtio-dev] " Heng Qi
@ 2023-02-08 2:30 ` Heng Qi
2023-02-08 3:19 ` Parav Pandit
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-08 2:30 UTC (permalink / raw)
To: Jason Wang, Parav Pandit
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/1/31 下午1:28, Heng Qi 写道:
> On Mon, Jan 16, 2023 at 04:42:11PM +0800, Jason Wang wrote:
>> 在 2023/1/16 16:01, Heng Qi 写道:
>>> On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
>>>> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
>>>>> If the tunnel is used to encapsulate the packets, the hash calculated
>>>>> using the outer header of the receive packets is always fixed for the
>>>>> same flow packets, i.e. they will be steered to the same receive queue.
>>>>>
>>>>> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related bitmasks
>>>>> in \field{hash_types}, which instructs the device to calculate the
>>>>> hash using the inner headers of tunnel-encapsulated packets. Besides,
>>>>> values in \field{hash_report_tunnel} are added to report tunnel types.
>>>>>
>>>>> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/151
>>>>>
>>>>> Reviewed-by: Jason Wang <jasowang@redhat.com>
>>>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>> ---
>>>>> v6:
>>>>> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
>>>>> 2. Fix some syntax issues. @Michael S. Tsirkin
>>>>>
>>>>> v5:
>>>>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
>>>>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
>>>>> 3. Move the links to introduction section. @Michael S. Tsirkin
>>>>> 4. Clarify some sentences. @Michael S. Tsirkin
>>>>>
>>>>> v4:
>>>>> 1. Clarify some paragraphs. @Cornelia Huck
>>>>> 2. Fix the u8 type. @Cornelia Huck
>>>>>
>>>>> v3:
>>>>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
>>>>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
>>>>> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
>>>>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>>>>>
>>>>> v2:
>>>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>>>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>>>>>
>>>>> v1:
>>>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>>>> 2. Clarify some paragraphs. @Jason Wang
>>>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>>>>>
>>>>> content.tex | 191 +++++++++++++++++++++++++++++++++++++++++++++--
>>>>> introduction.tex | 19 +++++
>>>>> 2 files changed, 203 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/content.tex b/content.tex
>>>>> index e863709..7845f6c 100644
>>>>> --- a/content.tex
>>>>> +++ b/content.tex
>>>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>>>>> channel.
>>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
>>>>> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated packets.
>>>>> +
>>>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>>>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>>>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>>>> to several segments when each of these smaller packets has UDP header.
>>>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>>>>> - value and a type of calculated hash.
>>>>> + value, a type of calculated hash, and, if VIRTIO_NET_F_HASH_TUNNEL
>>>>> + is negotiated, an encapsulation packet type.
>>>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>>>>> value. Device benefits from knowing the exact header length.
>>>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>>>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
>>>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>> \end{description}
>>>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>>>>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>>>>> le16 num_buffers;
>>>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>>> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>>>> of -> if?
>>> Sorry for the late reply.
>>> It's Cornelia's suggestion, and 'of' seems to work fine.
>>>
>>>> also let's add: otherwise reserved?
>>> Sure.
>>>
>>>>> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>>> };
>>>>> \end{lstlisting}
>>>>> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>>> A device attempts to calculate a per-packet hash in the following cases:
>>>>> \begin{itemize}
>>>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
>>>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
>>>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type. If additionally
>>>>> +VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the encapsulation type as well.
>>>>> \end{itemize}
>>>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>>>> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
>>>>> \end{itemize}
>>>>> +\subparagraph{Tunnel/Encapsulated packet}
>>>>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
>>>>> +A tunnel packet is encapsulated from the original packet based on the tunneling
>>>>> +protocol (only a single level of encapsulation is currently supported). The
>>>>> +encapsulated packet contains an outer header and an inner header, and the device
>>>>> +calculates the hash over either the inner header or the outer header.
>>>>> +
>>>>> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>>>>> +encapsulation type is set in \field{hash_types}, the hash for a specific type of
>>>>> +encapsulated packet is calculated over the inner as opposed to outer header.
>>>>> +Supported encapsulation types are listed in \ref{sec:Device Types / Network Device /
>>>>> +Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets /
>>>>> +Supported/enabled hash types}.
>>>>> +
>>>>> +If both VIRTIO_NET_F_HASH_REPORT and VIRTIO_NET_F_HASH_TUNNEL are negotiated,
>>>>> +the device can support inner hash calculation for \hyperref[intro:GRE]{[GRE]},
>>>>> +\hyperref[intro:VXLAN]{[VXLAN]} and \hyperref[intro:GENEVE]{[GENEVE]}
>>>>> +encapsulated packets, and the corresponding encapsulation type in \field{hash_types}
>>>>> +is VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively. The value in
>>>>> +\field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE, GRE, VXLAN, GENEVE} respectively.
>>>>> +
>>>>> +If VIRTIO_NET_F_HASH_REPORT is negotiated but VIRTIO_NET_F_HASH_TUNNEL is not
>>>>> +negotiated, the device calculates the hash over the outer header, and
>>>>> +\field{hash_report} reports the hash type. \field{hash_report_tunnel}
>>>>> +is no longer valid.
>>>> Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER
>>>> must not be set without VIRTIO_NET_F_HASH_TUNNEL?
>>>>
>>> Yes.
>>>
>>>> If yes then we can add conformance statements to this end and then
>>>> drop the
>>>> "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
>>>> all over the place just relying on hash type being set instead.
>>> Please see the response below.
>>>
>>>> Also, note that it has to be ok for device to expose hash types without
>>>> negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
>>>> Idea: do we want to decouple these completely? have VIRTIO_NET_F_HASH_TUNNEL
>>>> just add hash_report_tunnel and have hash types speak for themselves?
>>>> It makes things nicely orthogonal in that if one does not care
>>>> about knowing encapsulation type (e.g. for RSS) one can disable
>>>> VIRTIO_NET_F_HASH_TUNNEL.
>>> This seems to have the following 3 problems:
>>>
>>> 1. This will break the specification. As described in the specification, any field
>>> in the configuration space only exists when there is a corresponding feature bit:
>>> "Device configuration space is generally used for rarely-changing or initialization-time parameters.
>>> Where configuration fields are optional, their existence is indicated by feature bits: ",
>>> which also applies to supported_hash_types.
>>>
>>> 2. If I'm not wrong, this seems to make the encapsulation hash types dependent on
>>> VIRTIO_NET_F_RSS, can the driver read supported_hash_types (including encapsulation
>>> hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be the case,
>>> we only use encapsulation hash types to provide guidance for hash calculation, but not
>>> necessarily use hash values to select receive queues (unless VIRTIO_NET_F_RSS is negotiated).
>>>
>>> 3. If all supported_hash_types speak for themselves, why do we need the VIRTIO_NET_F_HASH_TUNNEL
>>> feature, because as long as the encapsulation hash types exist and VIRTIO_NET_F_HASH_REPORT is
>>> negotiated, hash_report_tunnel exists correspondingly. "have VIRTIO_NET_F_HASH_TUNNEL just add
>>> hash_report_tunnel"? It seems like hash_report_tunnel will only work if the encapsulation hash
>>> types are present.
>>>
>>> So, if I'm not wrong, the reasonable logic should be that VIRTIO_NET_F_HASH_TUNNEL is only used
>>> to declare that the device supports inner header hash and let the corresponding encapsulation
>>> hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this hash value to select the
>>> receive queue. Considering the migration at the same time, if the dst device does not negotiate
>>> VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use encapsulation hash types to
>>> determine whether the migration can succeed.
>>
>> I think it might be better to have consistency:
>>
>> 1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet
>> header, is it better to have a new config filed in the config space?
>>
>> or
>>
>> 2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file,
>> should we try to reuse hash_report?
>>
>> 1) seems better and cleaner to me.
> Sorry for the late reply due to vacation.
>
> Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to
> control all information related to inner header hash.
Hi, Parav.
Do you think we need both hash_types and hash_tunnel_types?
Thanks.
>
>> More below
>>
>>
>>> Thanks.
>>>
>>>> Objections? Jason?
>>>>
>>>>
>>>>> +
>>>>> \subparagraph{Supported/enabled hash types}
>>>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
>>>>> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
>>>>> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
>>>>> Hash types applicable for IPv4 packets:
>>>>> \begin{lstlisting}
>>>>> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
>>>>> @@ -3884,6 +3919,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>>>>> \end{lstlisting}
>>>>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
>>>>> +hash type below indicates that the hash is calculated over the inner
>>>>> +header of the encapsulated packet:
>>>>> +Hash type applicable for inner payload of the gre-encapsulated packet
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>>>>> +\end{lstlisting}
>>>>> +Hash type applicable for inner payload of the vxlan-encapsulated packet
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>>>>> +\end{lstlisting}
>>>>> +Hash type applicable for inner payload of the geneve-encapsulated packet
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>>>>> +\end{lstlisting}
>>
>> Just notice that this seems to break the equation:
>>
>> VIRTIO_NET_HASH_TYPE_XXX = 1 « (VIRTIO_NET_HASH_REPORT_XXX - 1)
>>
>> ?
>>
> Yes, we have discussed it here https://lists.oasis-open.org/archives/virtio-dev/202211/msg00189.html ,
> as you suggested above, it seems better to use the new config space field.
>
> Thanks.
>
>> Thanks
>>
>>
>>>>> +
>>>>> \subparagraph{IPv4 packets}
>>>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
>>>>> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
>>>>> @@ -3975,15 +4026,125 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
>>>>> \end{itemize}
>>>>> +\subparagraph{Inner payload of an encapsulated packet}
>>>>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payload of the encapsulated packet}
>>>>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
>>>>> +encapsulation hash type is set in \field{hash_types}, the device calculates the
>>>>> +inner header hash on an encapsulated packet (See \ref{sec:Device Types
>>>>> +/ Network Device / Device Operation / Processing of Incoming Packets /
>>>>> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}) as follows:
>>>>> +
>>>>> +The device calculates the hash on the inner IPv4 packet of an encapsulated packet
>>>>> +according to 'Enabled hash types' bitmask as follows:
>>>>> +\begin{itemize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set and the encapsulated packet has an inner
>>>>> + TCPv4 header, the hash is calculated over the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemsize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set and the encapsulated packet has an
>>>>> + inner UDPv4 header, the hash is calculated over the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set, the hash is calculated over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \end{itemsize}
>>>>> + \item Else the device does not calculate the hash
>>>>> +\end{itemize}
>>>>> +
>>>>> +The device calculates the hash on the inner IPv6 packet without an extension header
>>>>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>>>>> +\begin{itemize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set and the encapsulated packet has an inner
>>>>> + TCPv6 header, the hash is calculated over the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemsize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set and the encapsulated packet has an
>>>>> + inner UDPv6 header, the hash is calculated over the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set, the hash is calculated over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \end{itemsize}
>>>>> + \item Else the device does not calculate the hash
>>>>> +\end{itemize}
>>>>> +
>>>>> +The device calculates the hash on the inner IPv6 packet with an extension header
>>>>> +of an encapsulated packet according to 'Enabled hash types' bitmask as follows:
>>>>> +\begin{itemsize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set and the encapsulated packet has an inner
>>>>> + TCPv6 header, the hash is calculated over the following fields:
>>>>> + \begin{itemize}
>>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>>> + options header. If the inner extension header is not present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set and the encapsulated packet has an inner
>>>>> + UDPv6 header, the hash is calculated over the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>>> + options header. If the inner extension header is not present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set, the hash is calculated over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>>> + options header. If the inner extension header is not present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \end{itemize}
>>>>> + \item Else skip the inner IPv6 extension header and calculate the inner header hash as
>>>>> + defined for an encapsulated packet whose inner payload is an IPv6 packet without
>>>>> + an extension header.
>>>>> +\end{itemsize}
>>>>> +
>>>>> \paragraph{Hash reporting for incoming packets}
>>>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>>>>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>>>>> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
>>>>> -and \field{hash_value} with the value of calculated hash.
>>>>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
>>>>> +hash for the packet, the device fills \field{hash_report} with the report type
>>>>> +of calculated hash, and \field{hash_value} with the value of calculated hash.
>>>>> +Also, if VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device needs to fill
>>>>> +\field{hash_report_tunnel} with the report type of the encapsulated packet, and
>>>>> +it is set to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE for the unencapsulated packet.
>>>>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>>>>> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
>>>>> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
>>>>> +and sets \field{hash_report_tunnel} to VIRTIO_NET_HASH_REPORT_TUNNEL_NONE.
>>>>> Possible values that the device can report in \field{hash_report} are defined below.
>>>>> They correspond to supported hash types defined in
>>>>> @@ -4005,6 +4166,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>>>>> \end{lstlisting}
>>>>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>>>>> +\field{hash_report_tunnel} can report the type of the encapsulated
>>>>> +packet to the driver over the inner header hash calculation.
>>>>> +Possible values that the device can report in \field{hash_report_tunnel}
>>>>> +are defined below:
>>>>> +
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_REPORT_TUNNEL_NONE 0
>>>>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>>>>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>>>>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>>>>> +\end{lstlisting}
>>>>> +
>>>>> +They correspond to supported hash types defined in
>>>>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>>>>> +
>>>>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>>>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>>>>> diff --git a/introduction.tex b/introduction.tex
>>>>> index 287c5fc..ff01a9b 100644
>>>>> --- a/introduction.tex
>>>>> +++ b/introduction.tex
>>>>> @@ -98,6 +98,25 @@ \section{Normative References}\label{sec:Normative References}
>>>>> \phantomsection\label{intro:SEC1}\textbf{[SEC1]} &
>>>>> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
>>>>> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>>>>> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
>>>>> + Generic Routing Encapsulation
>>>>> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
>>>>> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
>>>>> + Virtual eXtensible Local Area Network
>>>>> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
>>>>> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
>>>>> + Generic Network Virtualization Encapsulation
>>>>> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
>>>>> + \phantomsection\label{intro:IP}\textbf{[IP]} &
>>>>> + INTERNET PROTOCOL
>>>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
>>>>> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
>>>>> + User Datagram Protocol
>>>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
>>>>> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
>>>>> + TRANSMISSION CONTROL PROTOCOL
>>>>> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
>>>>> +
>>>>> \end{longtable}
>>>>> --
>>>>> 2.19.1.6.gb485710b
>>>> This publicly archived list offers a means to provide input to the
>>>> OASIS Virtual I/O Device (VIRTIO) TC.
>>>>
>>>> In order to verify user consent to the Feedback License terms and
>>>> to minimize spam in the list archive, subscription is required
>>>> before posting.
>>>>
>>>> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
>>>> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
>>>> List help: virtio-comment-help@lists.oasis-open.org
>>>> List archive: https://lists.oasis-open.org/archives/virtio-comment/
>>>> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
>>>> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
>>>> Committee: https://www.oasis-open.org/committees/virtio/
>>>> Join OASIS: https://www.oasis-open.org/join/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 2:30 ` Heng Qi
@ 2023-02-08 3:19 ` Parav Pandit
2023-02-08 3:24 ` Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 3:19 UTC (permalink / raw)
To: Heng Qi, Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Tuesday, February 7, 2023 9:31 PM
>
> 在 2023/1/31 下午1:28, Heng Qi 写道:
> > On Mon, Jan 16, 2023 at 04:42:11PM +0800, Jason Wang wrote:
> >> 在 2023/1/16 16:01, Heng Qi 写道:
> >>> On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
> >>>> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
> >>>>> If the tunnel is used to encapsulate the packets, the hash
> >>>>> calculated using the outer header of the receive packets is always
> >>>>> fixed for the same flow packets, i.e. they will be steered to the same
> receive queue.
> >>>>>
> >>>>> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related
> bitmasks
> >>>>> in \field{hash_types}, which instructs the device to calculate the
> >>>>> hash using the inner headers of tunnel-encapsulated packets.
> >>>>> Besides, values in \field{hash_report_tunnel} are added to report tunnel
> types.
> >>>>>
> >>>>> Fixes:
> >>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> >>>>> github.com%2Foasis-tcs%2Fvirtio-
> spec%2Fissues%2F151&data=05%7C01%7
> >>>>>
> Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15
> 7
> >>>>>
> 27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown
> %7CT
> >>>>>
> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> >>>>>
> XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC
> OH4v9
> >>>>> %2BnLomXklNqs%3D&reserved=0
> >>>>>
> >>>>> Reviewed-by: Jason Wang <jasowang@redhat.com>
> >>>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> >>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >>>>> ---
> >>>>> v6:
> >>>>> 1. Modify the wording of some sentences for clarity. @Michael S.
> Tsirkin
> >>>>> 2. Fix some syntax issues. @Michael S. Tsirkin
> >>>>>
> >>>>> v5:
> >>>>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> >>>>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> >>>>> 3. Move the links to introduction section. @Michael S. Tsirkin
> >>>>> 4. Clarify some sentences. @Michael S. Tsirkin
> >>>>>
> >>>>> v4:
> >>>>> 1. Clarify some paragraphs. @Cornelia Huck
> >>>>> 2. Fix the u8 type. @Cornelia Huck
> >>>>>
> >>>>> v3:
> >>>>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to
> VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> >>>>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> >>>>> 3. Keep the possibility to use inner hash for automatic receive steering.
> @Jason Wang
> >>>>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
> >>>>> etc. many times. @Michael S. Tsirkin
> >>>>>
> >>>>> v2:
> >>>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> >>>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
> >>>>> @Jason Wang, @Michael S. Tsirkin
> >>>>>
> >>>>> v1:
> >>>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> >>>>> 2. Clarify some paragraphs. @Jason Wang
> >>>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.
> @Yuri
> >>>>> Benditovich
> >>>>>
> >>>>> content.tex | 191
> +++++++++++++++++++++++++++++++++++++++++++++--
> >>>>> introduction.tex | 19 +++++
> >>>>> 2 files changed, 203 insertions(+), 7 deletions(-)
> >>>>>
> >>>>> diff --git a/content.tex b/content.tex index e863709..7845f6c
> >>>>> 100644
> >>>>> --- a/content.tex
> >>>>> +++ b/content.tex
> >>>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device
> Types / Network Device / Feature bits
> >>>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through
> control
> >>>>> channel.
> >>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
> >>>>> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated
> packets.
> >>>>> +
> >>>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
> coalescing.
> >>>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4
> packets.
> >>>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device
> Types / Network Device / Feature bits
> >>>>> to several segments when each of these smaller packets has UDP
> header.
> >>>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet
> hash
> >>>>> - value and a type of calculated hash.
> >>>>> + value, a type of calculated hash, and, if
> VIRTIO_NET_F_HASH_TUNNEL
> >>>>> + is negotiated, an encapsulation packet type.
> >>>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
> \field{hdr_len}
> >>>>> value. Device benefits from knowing the exact header length.
> >>>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit
> requirements}\label{sec:Device Types / Network Device
> >>>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
> VIRTIO_NET_F_HOST_TSO6.
> >>>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires
> VIRTIO_NET_F_CTRL_VQ.
> >>>>> \end{description}
> >>>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
> >>>>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device
> Types / Network Device / Device O
> >>>>> le16 num_buffers;
> >>>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT
> negotiated)
> >>>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT
> negotiated)
> >>>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
> negotiated)
> >>>>> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
> >>>>> + negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
> >>>> of -> if?
> >>> Sorry for the late reply.
> >>> It's Cornelia's suggestion, and 'of' seems to work fine.
> >>>
> >>>> also let's add: otherwise reserved?
> >>> Sure.
> >>>
> >>>>> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
> negotiated)
> >>>>> };
> >>>>> \end{lstlisting}
> >>>>> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming
> Packets}\label{sec:Device Types / Network
> >>>>> A device attempts to calculate a per-packet hash in the following cases:
> >>>>> \begin{itemize}
> >>>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
> the hash to determine the receive virtqueue to place incoming packets.
> >>>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
> device reports the hash value and the hash type with the packet.
> >>>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
> >>>>> +device reports the hash value and the hash type. If additionally
> VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the
> encapsulation type as well.
> >>>>> \end{itemize}
> >>>>> If the feature VIRTIO_NET_F_RSS was negotiated:
> >>>>> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming
> Packets}\label{sec:Device Types / Network
> >>>>> \ref{sec:Device Types / Network Device / Device Operation / Control
> Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> >>>>> \end{itemize}
> >>>>> +\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types
> >>>>> +/ Network Device / Device Operation / Processing of Incoming
> >>>>> +Packets / Hash calculation for incoming packets /
> >>>>> +Tunnel/Encapsulated packet} A tunnel packet is encapsulated from
> >>>>> +the original packet based on the tunneling protocol (only a
> >>>>> +single level of encapsulation is currently supported). The encapsulated
> packet contains an outer header and an inner header, and the device calculates
> the hash over either the inner header or the outer header.
> >>>>> +
> >>>>> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
> >>>>> +corresponding encapsulation type is set in \field{hash_types},
> >>>>> +the hash for a specific type of encapsulated packet is calculated over
> the inner as opposed to outer header.
> >>>>> +Supported encapsulation types are listed in \ref{sec:Device Types
> >>>>> +/ Network Device / Device Operation / Processing of Incoming
> >>>>> +Packets / Hash calculation for incoming packets / Supported/enabled
> hash types}.
> >>>>> +
> >>>>> +If both VIRTIO_NET_F_HASH_REPORT and
> VIRTIO_NET_F_HASH_TUNNEL are
> >>>>> +negotiated, the device can support inner hash calculation for
> >>>>> +\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and
> >>>>> +\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the
> >>>>> +corresponding encapsulation type in \field{hash_types} is
> >>>>> +VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.
> The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE,
> GRE, VXLAN, GENEVE} respectively.
> >>>>> +
> >>>>> +If VIRTIO_NET_F_HASH_REPORT is negotiated but
> >>>>> +VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates
> >>>>> +the hash over the outer header, and \field{hash_report} reports
> >>>>> +the hash type. \field{hash_report_tunnel} is no longer valid.
> >>>> Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,
> GENEVE}_INNER
> >>>> must not be set without VIRTIO_NET_F_HASH_TUNNEL?
> >>>>
> >>> Yes.
> >>>
> >>>> If yes then we can add conformance statements to this end and then
> >>>> drop the "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
> >>>> all over the place just relying on hash type being set instead.
> >>> Please see the response below.
> >>>
> >>>> Also, note that it has to be ok for device to expose hash types without
> >>>> negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
> >>>> Idea: do we want to decouple these completely? have
> VIRTIO_NET_F_HASH_TUNNEL
> >>>> just add hash_report_tunnel and have hash types speak for themselves?
> >>>> It makes things nicely orthogonal in that if one does not care
> >>>> about knowing encapsulation type (e.g. for RSS) one can disable
> >>>> VIRTIO_NET_F_HASH_TUNNEL.
> >>> This seems to have the following 3 problems:
> >>>
> >>> 1. This will break the specification. As described in the specification, any
> field
> >>> in the configuration space only exists when there is a corresponding
> feature bit:
> >>> "Device configuration space is generally used for rarely-changing or
> initialization-time parameters.
> >>> Where configuration fields are optional, their existence is indicated by
> feature bits: ",
> >>> which also applies to supported_hash_types.
> >>>
> >>> 2. If I'm not wrong, this seems to make the encapsulation hash types
> dependent on
> >>> VIRTIO_NET_F_RSS, can the driver read supported_hash_types
> (including encapsulation
> >>> hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be
> the case,
> >>> we only use encapsulation hash types to provide guidance for hash
> calculation, but not
> >>> necessarily use hash values to select receive queues (unless
> VIRTIO_NET_F_RSS is negotiated).
> >>>
> >>> 3. If all supported_hash_types speak for themselves, why do we need the
> VIRTIO_NET_F_HASH_TUNNEL
> >>> feature, because as long as the encapsulation hash types exist and
> VIRTIO_NET_F_HASH_REPORT is
> >>> negotiated, hash_report_tunnel exists correspondingly. "have
> VIRTIO_NET_F_HASH_TUNNEL just add
> >>> hash_report_tunnel"? It seems like hash_report_tunnel will only work if
> the encapsulation hash
> >>> types are present.
> >>>
> >>> So, if I'm not wrong, the reasonable logic should be that
> VIRTIO_NET_F_HASH_TUNNEL is only used
> >>> to declare that the device supports inner header hash and let the
> corresponding encapsulation
> >>> hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this
> hash value to select the
> >>> receive queue. Considering the migration at the same time, if the dst device
> does not negotiate
> >>> VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use
> encapsulation hash types to
> >>> determine whether the migration can succeed.
> >>
> >> I think it might be better to have consistency:
> >>
> >> 1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet
> >> header, is it better to have a new config filed in the config space?
> >>
> >> or
> >>
> >> 2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file,
> >> should we try to reuse hash_report?
> >>
> >> 1) seems better and cleaner to me.
> > Sorry for the late reply due to vacation.
> >
> > Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to
> > control all information related to inner header hash.
>
> Hi, Parav.
>
> Do you think we need both hash_types and hash_tunnel_types?
In struct virtio_net_config we need two fields.
a. supported_hash_types (already exists)
b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is supported.
In struct virtio_net_hdr we need two fields.
a. hash_report (already exists)
b. hash_tunnel_type 8 bits -> absolute value indicating which outer header exists when inner header hash calculated.
You already have it in your patch named as hash_report_tunnel.
May be better to name as hash_report_tunnel_type to make it clearer that its type.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 3:19 ` Parav Pandit
@ 2023-02-08 3:24 ` Heng Qi
2023-02-08 5:18 ` Parav Pandit
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-08 3:24 UTC (permalink / raw)
To: Parav Pandit, Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/2/8 上午11:19, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Tuesday, February 7, 2023 9:31 PM
>>
>> 在 2023/1/31 下午1:28, Heng Qi 写道:
>>> On Mon, Jan 16, 2023 at 04:42:11PM +0800, Jason Wang wrote:
>>>> 在 2023/1/16 16:01, Heng Qi 写道:
>>>>> On Wed, Jan 11, 2023 at 04:45:12AM -0500, Michael S. Tsirkin wrote:
>>>>>> On Wed, Jan 04, 2023 at 03:14:01PM +0800, Heng Qi wrote:
>>>>>>> If the tunnel is used to encapsulate the packets, the hash
>>>>>>> calculated using the outer header of the receive packets is always
>>>>>>> fixed for the same flow packets, i.e. they will be steered to the same
>> receive queue.
>>>>>>> We add a feature bit VIRTIO_NET_F_HASH_TUNNEL and related
>> bitmasks
>>>>>>> in \field{hash_types}, which instructs the device to calculate the
>>>>>>> hash using the inner headers of tunnel-encapsulated packets.
>>>>>>> Besides, values in \field{hash_report_tunnel} are added to report tunnel
>> types.
>>>>>>> Fixes:
>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
>>>>>>> github.com%2Foasis-tcs%2Fvirtio-
>> spec%2Fissues%2F151&data=05%7C01%7
>> Cparav%40nvidia.com%7C12be6c51b3c04860c01608db097c796a%7C43083d15
>> 7
>> 27340c1b7db39efd9ccc17a%7C0%7C0%7C638114202991221369%7CUnknown
>> %7CT
>> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
>> XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6feFgy94LMCYfG0NKPlHCr1AvkC
>> OH4v9
>>>>>>> %2BnLomXklNqs%3D&reserved=0
>>>>>>>
>>>>>>> Reviewed-by: Jason Wang <jasowang@redhat.com>
>>>>>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>>>> ---
>>>>>>> v6:
>>>>>>> 1. Modify the wording of some sentences for clarity. @Michael S.
>> Tsirkin
>>>>>>> 2. Fix some syntax issues. @Michael S. Tsirkin
>>>>>>>
>>>>>>> v5:
>>>>>>> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
>>>>>>> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
>>>>>>> 3. Move the links to introduction section. @Michael S. Tsirkin
>>>>>>> 4. Clarify some sentences. @Michael S. Tsirkin
>>>>>>>
>>>>>>> v4:
>>>>>>> 1. Clarify some paragraphs. @Cornelia Huck
>>>>>>> 2. Fix the u8 type. @Cornelia Huck
>>>>>>>
>>>>>>> v3:
>>>>>>> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to
>> VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
>>>>>>> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
>>>>>>> 3. Keep the possibility to use inner hash for automatic receive steering.
>> @Jason Wang
>>>>>>> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE
>>>>>>> etc. many times. @Michael S. Tsirkin
>>>>>>>
>>>>>>> v2:
>>>>>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>>>>>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
>>>>>>> @Jason Wang, @Michael S. Tsirkin
>>>>>>>
>>>>>>> v1:
>>>>>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>>>>>> 2. Clarify some paragraphs. @Jason Wang
>>>>>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.
>> @Yuri
>>>>>>> Benditovich
>>>>>>>
>>>>>>> content.tex | 191
>> +++++++++++++++++++++++++++++++++++++++++++++--
>>>>>>> introduction.tex | 19 +++++
>>>>>>> 2 files changed, 203 insertions(+), 7 deletions(-)
>>>>>>>
>>>>>>> diff --git a/content.tex b/content.tex index e863709..7845f6c
>>>>>>> 100644
>>>>>>> --- a/content.tex
>>>>>>> +++ b/content.tex
>>>>>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device
>> Types / Network Device / Feature bits
>>>>>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through
>> control
>>>>>>> channel.
>>>>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner
>>>>>>> + header hash for GRE, VXLAN and GENEVE tunnel-encapsulated
>> packets.
>>>>>>> +
>>>>>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
>> coalescing.
>>>>>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4
>> packets.
>>>>>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device
>> Types / Network Device / Feature bits
>>>>>>> to several segments when each of these smaller packets has UDP
>> header.
>>>>>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet
>> hash
>>>>>>> - value and a type of calculated hash.
>>>>>>> + value, a type of calculated hash, and, if
>> VIRTIO_NET_F_HASH_TUNNEL
>>>>>>> + is negotiated, an encapsulation packet type.
>>>>>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
>> \field{hdr_len}
>>>>>>> value. Device benefits from knowing the exact header length.
>>>>>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit
>> requirements}\label{sec:Device Types / Network Device
>>>>>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
>> VIRTIO_NET_F_HOST_TSO6.
>>>>>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires
>> VIRTIO_NET_F_CTRL_VQ.
>>>>>>> \end{description}
>>>>>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
>>>>>>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device
>> Types / Network Device / Device O
>>>>>>> le16 num_buffers;
>>>>>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT
>> negotiated)
>>>>>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT
>> negotiated)
>>>>>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>> negotiated)
>>>>>>> + u8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>>>> + negotiated, only valid of VIRTIO_NET_F_HASH_TUNNEL negotiated)
>>>>>> of -> if?
>>>>> Sorry for the late reply.
>>>>> It's Cornelia's suggestion, and 'of' seems to work fine.
>>>>>
>>>>>> also let's add: otherwise reserved?
>>>>> Sure.
>>>>>
>>>>>>> + u8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>> negotiated)
>>>>>>> };
>>>>>>> \end{lstlisting}
>>>>>>> @@ -3837,7 +3843,8 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>>>>>>> A device attempts to calculate a per-packet hash in the following cases:
>>>>>>> \begin{itemize}
>>>>>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
>> the hash to determine the receive virtqueue to place incoming packets.
>>>>>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>> device reports the hash value and the hash type with the packet.
>>>>>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>>>>>>> +device reports the hash value and the hash type. If additionally
>> VIRTIO_NET_F_HASH_TUNNEL was negotiated, the device reports the
>> encapsulation type as well.
>>>>>>> \end{itemize}
>>>>>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>>>>>> @@ -3863,8 +3870,36 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>>>>>>> \ref{sec:Device Types / Network Device / Device Operation / Control
>> Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
>>>>>>> \end{itemize}
>>>>>>> +\subparagraph{Tunnel/Encapsulated packet} \label{sec:Device Types
>>>>>>> +/ Network Device / Device Operation / Processing of Incoming
>>>>>>> +Packets / Hash calculation for incoming packets /
>>>>>>> +Tunnel/Encapsulated packet} A tunnel packet is encapsulated from
>>>>>>> +the original packet based on the tunneling protocol (only a
>>>>>>> +single level of encapsulation is currently supported). The encapsulated
>> packet contains an outer header and an inner header, and the device calculates
>> the hash over either the inner header or the outer header.
>>>>>>> +
>>>>>>> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
>>>>>>> +corresponding encapsulation type is set in \field{hash_types},
>>>>>>> +the hash for a specific type of encapsulated packet is calculated over
>> the inner as opposed to outer header.
>>>>>>> +Supported encapsulation types are listed in \ref{sec:Device Types
>>>>>>> +/ Network Device / Device Operation / Processing of Incoming
>>>>>>> +Packets / Hash calculation for incoming packets / Supported/enabled
>> hash types}.
>>>>>>> +
>>>>>>> +If both VIRTIO_NET_F_HASH_REPORT and
>> VIRTIO_NET_F_HASH_TUNNEL are
>>>>>>> +negotiated, the device can support inner hash calculation for
>>>>>>> +\hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]} and
>>>>>>> +\hyperref[intro:GENEVE]{[GENEVE]} encapsulated packets, and the
>>>>>>> +corresponding encapsulation type in \field{hash_types} is
>>>>>>> +VIRTIO_NET_HASH_TYPE_{GRE, VXLAN, GENEVE}_INNER respectively.
>> The value in \field{hash_report_tunnel} is VIRTIO_NET_HASH_REPORT_{NONE,
>> GRE, VXLAN, GENEVE} respectively.
>>>>>>> +
>>>>>>> +If VIRTIO_NET_F_HASH_REPORT is negotiated but
>>>>>>> +VIRTIO_NET_F_HASH_TUNNEL is not negotiated, the device calculates
>>>>>>> +the hash over the outer header, and \field{hash_report} reports
>>>>>>> +the hash type. \field{hash_report_tunnel} is no longer valid.
>>>>>> Does this mean that VIRTIO_NET_HASH_TYPE_{GRE, VXLAN,
>> GENEVE}_INNER
>>>>>> must not be set without VIRTIO_NET_F_HASH_TUNNEL?
>>>>>>
>>>>> Yes.
>>>>>
>>>>>> If yes then we can add conformance statements to this end and then
>>>>>> drop the "If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated"
>>>>>> all over the place just relying on hash type being set instead.
>>>>> Please see the response below.
>>>>>
>>>>>> Also, note that it has to be ok for device to expose hash types without
>>>>>> negotiating VIRTIO_NET_F_HASH_TUNNEL since they are in config space.
>>>>>> Idea: do we want to decouple these completely? have
>> VIRTIO_NET_F_HASH_TUNNEL
>>>>>> just add hash_report_tunnel and have hash types speak for themselves?
>>>>>> It makes things nicely orthogonal in that if one does not care
>>>>>> about knowing encapsulation type (e.g. for RSS) one can disable
>>>>>> VIRTIO_NET_F_HASH_TUNNEL.
>>>>> This seems to have the following 3 problems:
>>>>>
>>>>> 1. This will break the specification. As described in the specification, any
>> field
>>>>> in the configuration space only exists when there is a corresponding
>> feature bit:
>>>>> "Device configuration space is generally used for rarely-changing or
>> initialization-time parameters.
>>>>> Where configuration fields are optional, their existence is indicated by
>> feature bits: ",
>>>>> which also applies to supported_hash_types.
>>>>>
>>>>> 2. If I'm not wrong, this seems to make the encapsulation hash types
>> dependent on
>>>>> VIRTIO_NET_F_RSS, can the driver read supported_hash_types
>> (including encapsulation
>>>>> hash types) if VIRTIO_NET_F_RSS is not negotiated? But that shouldn't be
>> the case,
>>>>> we only use encapsulation hash types to provide guidance for hash
>> calculation, but not
>>>>> necessarily use hash values to select receive queues (unless
>> VIRTIO_NET_F_RSS is negotiated).
>>>>> 3. If all supported_hash_types speak for themselves, why do we need the
>> VIRTIO_NET_F_HASH_TUNNEL
>>>>> feature, because as long as the encapsulation hash types exist and
>> VIRTIO_NET_F_HASH_REPORT is
>>>>> negotiated, hash_report_tunnel exists correspondingly. "have
>> VIRTIO_NET_F_HASH_TUNNEL just add
>>>>> hash_report_tunnel"? It seems like hash_report_tunnel will only work if
>> the encapsulation hash
>>>>> types are present.
>>>>>
>>>>> So, if I'm not wrong, the reasonable logic should be that
>> VIRTIO_NET_F_HASH_TUNNEL is only used
>>>>> to declare that the device supports inner header hash and let the
>> corresponding encapsulation
>>>>> hash type exist, when VIRTIO_NET_F_RSS is negotiated, we can use this
>> hash value to select the
>>>>> receive queue. Considering the migration at the same time, if the dst device
>> does not negotiate
>>>>> VIRTIO_NET_F_HASH_TUNNEL, the migration will fail; otherwise, use
>> encapsulation hash types to
>>>>> determine whether the migration can succeed.
>>>> I think it might be better to have consistency:
>>>>
>>>> 1) If VIRTIO_NET_F_HASH_TUNNEL introduces a new field in vnet
>>>> header, is it better to have a new config filed in the config space?
>>>>
>>>> or
>>>>
>>>> 2) If VIRTIO_NET_F_HASH_TUNNEL doesn't introduce new config file,
>>>> should we try to reuse hash_report?
>>>>
>>>> 1) seems better and cleaner to me.
>>> Sorry for the late reply due to vacation.
>>>
>>> Good point, it's clearer to use the VIRTIO_NET_F_HASH_TUNNEL feature to
>>> control all information related to inner header hash.
>> Hi, Parav.
>>
>> Do you think we need both hash_types and hash_tunnel_types?
> In struct virtio_net_config we need two fields.
> a. supported_hash_types (already exists)
> b. supported_hash_tunnel_type -> bitmap indicating for which outer headers, inner hash calculation is supported.
Thanks for the suggestion, we seem to have reached an agreement.
>
> In struct virtio_net_hdr we need two fields.
> a. hash_report (already exists)
> b. hash_tunnel_type 8 bits -> absolute value indicating which outer header exists when inner header hash calculated.
> You already have it in your patch named as hash_report_tunnel.
> May be better to name as hash_report_tunnel_type to make it clearer that its type.
Sure.
Thanks for your reply.
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 3:24 ` Heng Qi
@ 2023-02-08 5:18 ` Parav Pandit
2023-02-08 6:11 ` Heng Qi
2023-02-08 13:31 ` [virtio-comment] " Michael S. Tsirkin
0 siblings, 2 replies; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 5:18 UTC (permalink / raw)
To: Heng Qi, Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Tuesday, February 7, 2023 10:25 PM
[..]
> >>
> >> Do you think we need both hash_types and hash_tunnel_types?
> > In struct virtio_net_config we need two fields.
> > a. supported_hash_types (already exists) b. supported_hash_tunnel_type
> > -> bitmap indicating for which outer headers, inner hash calculation is
> supported.
>
> Thanks for the suggestion, we seem to have reached an agreement.
>
> >
> > In struct virtio_net_hdr we need two fields.
> > a. hash_report (already exists)
> > b. hash_tunnel_type 8 bits -> absolute value indicating which outer header
> exists when inner header hash calculated.
> > You already have it in your patch named as hash_report_tunnel.
> > May be better to name as hash_report_tunnel_type to make it clearer that its
> type.
>
> Sure.
>
> Thanks for your reply.
I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr?
Is this for debug? Or is there a use case that will process this value?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 5:18 ` Parav Pandit
@ 2023-02-08 6:11 ` Heng Qi
2023-02-08 12:21 ` Parav Pandit
2023-02-08 13:31 ` [virtio-comment] " Michael S. Tsirkin
1 sibling, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-08 6:11 UTC (permalink / raw)
To: Parav Pandit, Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/2/8 下午1:18, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Tuesday, February 7, 2023 10:25 PM
> [..]
>>>> Do you think we need both hash_types and hash_tunnel_types?
>>> In struct virtio_net_config we need two fields.
>>> a. supported_hash_types (already exists) b. supported_hash_tunnel_type
>>> -> bitmap indicating for which outer headers, inner hash calculation is
>> supported.
>>
>> Thanks for the suggestion, we seem to have reached an agreement.
>>
>>> In struct virtio_net_hdr we need two fields.
>>> a. hash_report (already exists)
>>> b. hash_tunnel_type 8 bits -> absolute value indicating which outer header
>> exists when inner header hash calculated.
>>> You already have it in your patch named as hash_report_tunnel.
>>> May be better to name as hash_report_tunnel_type to make it clearer that its
>> type.
>>
>> Sure.
>>
>> Thanks for your reply.
> I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> Is this for debug? Or is there a use case that will process this value?
The driver may use it to do some statistical information, or do some rx
classification
based on the rx hash, and we'd better not hide information from the driver.
Thanks.
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 6:11 ` Heng Qi
@ 2023-02-08 12:21 ` Parav Pandit
2023-02-09 5:20 ` [virtio-comment] " Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 12:21 UTC (permalink / raw)
To: Heng Qi, Jason Wang
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Wednesday, February 8, 2023 1:11 AM
>
>
> 在 2023/2/8 下午1:18, Parav Pandit 写道:
> >> From: Heng Qi <hengqi@linux.alibaba.com>
> >> Sent: Tuesday, February 7, 2023 10:25 PM
> > [..]
> >>>> Do you think we need both hash_types and hash_tunnel_types?
> >>> In struct virtio_net_config we need two fields.
> >>> a. supported_hash_types (already exists) b.
> >>> supported_hash_tunnel_type
> >>> -> bitmap indicating for which outer headers, inner hash calculation
> >>> -> is
> >> supported.
> >>
> >> Thanks for the suggestion, we seem to have reached an agreement.
> >>
> >>> In struct virtio_net_hdr we need two fields.
> >>> a. hash_report (already exists)
> >>> b. hash_tunnel_type 8 bits -> absolute value indicating which outer
> >>> header
> >> exists when inner header hash calculated.
> >>> You already have it in your patch named as hash_report_tunnel.
> >>> May be better to name as hash_report_tunnel_type to make it clearer
> >>> that its
> >> type.
> >>
> >> Sure.
> >>
> >> Thanks for your reply.
> > I had one last question. Why do we need to inform the
> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > Is this for debug? Or is there a use case that will process this value?
>
> The driver may use it to do some statistical information, or do some rx
> classification based on the rx hash, and we'd better not hide information from
> the driver.
>
Statistical information is better gathered via stats, instead of adding such code in driver data path.
It is not about hiding.
It has onus on the data path to detect the tunnel type and place that virtio_net_hdr.
So lets find one use of it in data path which sw entity will use it.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 5:18 ` Parav Pandit
2023-02-08 6:11 ` Heng Qi
@ 2023-02-08 13:31 ` Michael S. Tsirkin
2023-02-08 13:38 ` Parav Pandit
1 sibling, 1 reply; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-08 13:31 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > From: Heng Qi <hengqi@linux.alibaba.com>
> > Sent: Tuesday, February 7, 2023 10:25 PM
>
> [..]
> > >>
> > >> Do you think we need both hash_types and hash_tunnel_types?
> > > In struct virtio_net_config we need two fields.
> > > a. supported_hash_types (already exists) b. supported_hash_tunnel_type
> > > -> bitmap indicating for which outer headers, inner hash calculation is
> > supported.
> >
> > Thanks for the suggestion, we seem to have reached an agreement.
> >
> > >
> > > In struct virtio_net_hdr we need two fields.
> > > a. hash_report (already exists)
> > > b. hash_tunnel_type 8 bits -> absolute value indicating which outer header
> > exists when inner header hash calculated.
> > > You already have it in your patch named as hash_report_tunnel.
> > > May be better to name as hash_report_tunnel_type to make it clearer that its
> > type.
> >
> > Sure.
> >
> > Thanks for your reply.
>
> I had one last question. Why do we need to inform the hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> Is this for debug? Or is there a use case that will process this value?
Well we have hash_report which is kind of similar (and also kind of
pointless but I think it's there because WHQL wants it). Maybe we can steal
some bits from there instead of a new field?
I have a follow up question though: are we only hashing the inner header
or both inner and outer header? Somewhat confused on this.
In fact, CC Yuri for thoughts and suggestions from windows side of
things.
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 13:31 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-02-08 13:38 ` Parav Pandit
2023-02-08 13:52 ` Michael S. Tsirkin
0 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 13:38 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, February 8, 2023 8:32 AM
>
> On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > Sent: Tuesday, February 7, 2023 10:25 PM
> >
> > [..]
> > > >>
> > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > In struct virtio_net_config we need two fields.
> > > > a. supported_hash_types (already exists) b.
> > > > supported_hash_tunnel_type
> > > > -> bitmap indicating for which outer headers, inner hash
> > > > -> calculation is
> > > supported.
> > >
> > > Thanks for the suggestion, we seem to have reached an agreement.
> > >
> > > >
> > > > In struct virtio_net_hdr we need two fields.
> > > > a. hash_report (already exists)
> > > > b. hash_tunnel_type 8 bits -> absolute value indicating which
> > > > outer header
> > > exists when inner header hash calculated.
> > > > You already have it in your patch named as hash_report_tunnel.
> > > > May be better to name as hash_report_tunnel_type to make it
> > > > clearer that its
> > > type.
> > >
> > > Sure.
> > >
> > > Thanks for your reply.
> >
> > I had one last question. Why do we need to inform the
> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > Is this for debug? Or is there a use case that will process this value?
>
> Well we have hash_report which is kind of similar (and also kind of pointless
> but I think it's there because WHQL wants it).
Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4 udp etc).
OS can use this value to find tcp connection in a given namespace.
> Maybe we can steal some bits
> from there instead of a new field?
>
I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful.
If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.
>
> I have a follow up question though: are we only hashing the inner header or
> both inner and outer header? Somewhat confused on this.
>
I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 13:38 ` Parav Pandit
@ 2023-02-08 13:52 ` Michael S. Tsirkin
2023-02-08 14:00 ` Parav Pandit
2023-02-08 14:05 ` Parav Pandit
0 siblings, 2 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-08 13:52 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, February 8, 2023 8:32 AM
> >
> > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > >
> > > [..]
> > > > >>
> > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > In struct virtio_net_config we need two fields.
> > > > > a. supported_hash_types (already exists) b.
> > > > > supported_hash_tunnel_type
> > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > -> calculation is
> > > > supported.
> > > >
> > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > >
> > > > >
> > > > > In struct virtio_net_hdr we need two fields.
> > > > > a. hash_report (already exists)
> > > > > b. hash_tunnel_type 8 bits -> absolute value indicating which
> > > > > outer header
> > > > exists when inner header hash calculated.
> > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > clearer that its
> > > > type.
> > > >
> > > > Sure.
> > > >
> > > > Thanks for your reply.
> > >
> > > I had one last question. Why do we need to inform the
> > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > Is this for debug? Or is there a use case that will process this value?
> >
> > Well we have hash_report which is kind of similar (and also kind of pointless
> > but I think it's there because WHQL wants it).
> Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4 udp etc).
> OS can use this value to find tcp connection in a given namespace.
>
> > Maybe we can steal some bits
> > from there instead of a new field?
> >
> I do not have problem adding extra bits. I just don't find that just telling that its vxlan or nvgre to the OS is useful.
> If OS needs to know about outer header details, it needs to know the VNI information than just telling vxlan.
This does make sense.
> >
> > I have a follow up question though: are we only hashing the inner header or
> > both inner and outer header? Somewhat confused on this.
> >
> I understood as inner header. But worth to describe it. May be there. Need to read v8 patch.
Hmm. I just realized that there's a security problem with hashing
just the inner header: it allow users inside the tunnel control queueing outside.
By observing packet loss some information leaks between tunnels.
Ideas for solving this they all involve hashing both inner and outer
header:
1- report two sets of hashes. overkill?
2- hash both headers together
2- add salt. can come from driver or device itself
More ideas?
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 13:52 ` Michael S. Tsirkin
@ 2023-02-08 14:00 ` Parav Pandit
2023-02-08 14:09 ` Michael S. Tsirkin
2023-02-08 14:05 ` Parav Pandit
1 sibling, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 14:00 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, February 8, 2023 8:52 AM
>
> On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, February 8, 2023 8:32 AM
> > >
> > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > >
> > > > [..]
> > > > > >>
> > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > In struct virtio_net_config we need two fields.
> > > > > > a. supported_hash_types (already exists) b.
> > > > > > supported_hash_tunnel_type
> > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > -> calculation is
> > > > > supported.
> > > > >
> > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > >
> > > > > >
> > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > absolute value indicating which outer header
> > > > > exists when inner header hash calculated.
> > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > clearer that its
> > > > > type.
> > > > >
> > > > > Sure.
> > > > >
> > > > > Thanks for your reply.
> > > >
> > > > I had one last question. Why do we need to inform the
> > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > Is this for debug? Or is there a use case that will process this value?
> > >
> > > Well we have hash_report which is kind of similar (and also kind of
> > > pointless but I think it's there because WHQL wants it).
> > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> udp etc).
> > OS can use this value to find tcp connection in a given namespace.
> >
> > > Maybe we can steal some bits
> > > from there instead of a new field?
> > >
> > I do not have problem adding extra bits. I just don't find that just telling that
> its vxlan or nvgre to the OS is useful.
> > If OS needs to know about outer header details, it needs to know the VNI
> information than just telling vxlan.
>
> This does make sense.
>
>
> > >
> > > I have a follow up question though: are we only hashing the inner
> > > header or both inner and outer header? Somewhat confused on this.
> > >
> > I understood as inner header. But worth to describe it. May be there. Need to
> read v8 patch.
>
> Hmm. I just realized that there's a security problem with hashing just the inner
> header: it allow users inside the tunnel control queueing outside.
> By observing packet loss some information leaks between tunnels.
>
I likely didn't understand. Can you please explain?
Queuing is always done on the inner header with/without encapsulation.
Hash is always reported for inner header.
It is only adding the ability to hash even when outer header exists.
If queuing to be decided based on outer header (hash), then that is different.
Hashing both inner and outer in a flat q structure unlikely works, right?
Because both hashes can result in different q selection.
>
> Ideas for solving this they all involve hashing both inner and outer
> header:
> 1- report two sets of hashes. overkill?
> 2- hash both headers together
> 2- add salt. can come from driver or device itself
>
> More ideas?
>
> --
> MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 13:52 ` Michael S. Tsirkin
2023-02-08 14:00 ` Parav Pandit
@ 2023-02-08 14:05 ` Parav Pandit
2023-02-08 14:10 ` Michael S. Tsirkin
1 sibling, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 14:05 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, February 8, 2023 8:52 AM
>
> On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, February 8, 2023 8:32 AM
> > >
> > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > >
> > > > [..]
> > > > > >>
> > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > In struct virtio_net_config we need two fields.
> > > > > > a. supported_hash_types (already exists) b.
> > > > > > supported_hash_tunnel_type
> > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > -> calculation is
> > > > > supported.
> > > > >
> > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > >
> > > > > >
> > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > absolute value indicating which outer header
> > > > > exists when inner header hash calculated.
> > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > clearer that its
> > > > > type.
> > > > >
> > > > > Sure.
> > > > >
> > > > > Thanks for your reply.
> > > >
> > > > I had one last question. Why do we need to inform the
> > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > Is this for debug? Or is there a use case that will process this value?
> > >
> > > Well we have hash_report which is kind of similar (and also kind of
> > > pointless but I think it's there because WHQL wants it).
> > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> udp etc).
> > OS can use this value to find tcp connection in a given namespace.
> >
> > > Maybe we can steal some bits
> > > from there instead of a new field?
> > >
> > I do not have problem adding extra bits. I just don't find that just telling that
> its vxlan or nvgre to the OS is useful.
> > If OS needs to know about outer header details, it needs to know the VNI
> information than just telling vxlan.
>
> This does make sense.
>
>
> > >
> > > I have a follow up question though: are we only hashing the inner
> > > header or both inner and outer header? Somewhat confused on this.
> > >
> > I understood as inner header. But worth to describe it. May be there. Need to
> read v8 patch.
>
> Hmm. I just realized that there's a security problem with hashing just the inner
> header: it allow users inside the tunnel control queueing outside.
> By observing packet loss some information leaks between tunnels.
>
Ah I know now.
We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.
I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 14:00 ` Parav Pandit
@ 2023-02-08 14:09 ` Michael S. Tsirkin
2023-02-08 14:29 ` Parav Pandit
2023-02-09 5:12 ` Heng Qi
0 siblings, 2 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-08 14:09 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 08, 2023 at 02:00:14PM +0000, Parav Pandit wrote:
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, February 8, 2023 8:52 AM
> >
> > On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, February 8, 2023 8:32 AM
> > > >
> > > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > > >
> > > > > [..]
> > > > > > >>
> > > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > > In struct virtio_net_config we need two fields.
> > > > > > > a. supported_hash_types (already exists) b.
> > > > > > > supported_hash_tunnel_type
> > > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > > -> calculation is
> > > > > > supported.
> > > > > >
> > > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > > >
> > > > > > >
> > > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > > absolute value indicating which outer header
> > > > > > exists when inner header hash calculated.
> > > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > > clearer that its
> > > > > > type.
> > > > > >
> > > > > > Sure.
> > > > > >
> > > > > > Thanks for your reply.
> > > > >
> > > > > I had one last question. Why do we need to inform the
> > > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > > Is this for debug? Or is there a use case that will process this value?
> > > >
> > > > Well we have hash_report which is kind of similar (and also kind of
> > > > pointless but I think it's there because WHQL wants it).
> > > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> > udp etc).
> > > OS can use this value to find tcp connection in a given namespace.
> > >
> > > > Maybe we can steal some bits
> > > > from there instead of a new field?
> > > >
> > > I do not have problem adding extra bits. I just don't find that just telling that
> > its vxlan or nvgre to the OS is useful.
> > > If OS needs to know about outer header details, it needs to know the VNI
> > information than just telling vxlan.
> >
> > This does make sense.
> >
> >
> > > >
> > > > I have a follow up question though: are we only hashing the inner
> > > > header or both inner and outer header? Somewhat confused on this.
> > > >
> > > I understood as inner header. But worth to describe it. May be there. Need to
> > read v8 patch.
> >
> > Hmm. I just realized that there's a security problem with hashing just the inner
> > header: it allow users inside the tunnel control queueing outside.
> > By observing packet loss some information leaks between tunnels.
> >
> I likely didn't understand. Can you please explain?
>
> Queuing is always done on the inner header with/without encapsulation.
> Hash is always reported for inner header.
> It is only adding the ability to hash even when outer header exists.
If hashing just on outer header (currently the only option) then
a given tunnel all lands in a given queue.
Just keep that queue separate and users of this tunnel can not
learn whether other queues are overflowing, and can not overflow
other queues.
If you hash inner header then user can flood device with
packets of a given connection and the same connection in a different
tunnel hashes to the same queue. Now one tunnel can
- cause DoS for another tunnel
- cause packet loss or latency triggering possible security bugs within guest
- detect that another tunnel is using the connection by
detecting its own packet loss or increased latency
> If queuing to be decided based on outer header (hash), then that is different.
> Hashing both inner and outer in a flat q structure unlikely works, right?
> Because both hashes can result in different q selection.
That's the point.
Is there any precedent in OSes for configuring things like this
that we can look at?
> >
> > Ideas for solving this they all involve hashing both inner and outer
> > header:
> > 1- report two sets of hashes. overkill?
> > 2- hash both headers together
> > 2- add salt. can come from driver or device itself
> >
> > More ideas?
> >
> > --
> > MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 14:05 ` Parav Pandit
@ 2023-02-08 14:10 ` Michael S. Tsirkin
0 siblings, 0 replies; 54+ messages in thread
From: Michael S. Tsirkin @ 2023-02-08 14:10 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Feb 08, 2023 at 02:05:52PM +0000, Parav Pandit wrote:
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, February 8, 2023 8:52 AM
> >
> > On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, February 8, 2023 8:32 AM
> > > >
> > > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > > >
> > > > > [..]
> > > > > > >>
> > > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > > In struct virtio_net_config we need two fields.
> > > > > > > a. supported_hash_types (already exists) b.
> > > > > > > supported_hash_tunnel_type
> > > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > > -> calculation is
> > > > > > supported.
> > > > > >
> > > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > > >
> > > > > > >
> > > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > > absolute value indicating which outer header
> > > > > > exists when inner header hash calculated.
> > > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > > clearer that its
> > > > > > type.
> > > > > >
> > > > > > Sure.
> > > > > >
> > > > > > Thanks for your reply.
> > > > >
> > > > > I had one last question. Why do we need to inform the
> > > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > > Is this for debug? Or is there a use case that will process this value?
> > > >
> > > > Well we have hash_report which is kind of similar (and also kind of
> > > > pointless but I think it's there because WHQL wants it).
> > > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> > udp etc).
> > > OS can use this value to find tcp connection in a given namespace.
> > >
> > > > Maybe we can steal some bits
> > > > from there instead of a new field?
> > > >
> > > I do not have problem adding extra bits. I just don't find that just telling that
> > its vxlan or nvgre to the OS is useful.
> > > If OS needs to know about outer header details, it needs to know the VNI
> > information than just telling vxlan.
> >
> > This does make sense.
> >
> >
> > > >
> > > > I have a follow up question though: are we only hashing the inner
> > > > header or both inner and outer header? Somewhat confused on this.
> > > >
> > > I understood as inner header. But worth to describe it. May be there. Need to
> > read v8 patch.
> >
> > Hmm. I just realized that there's a security problem with hashing just the inner
> > header: it allow users inside the tunnel control queueing outside.
> > By observing packet loss some information leaks between tunnels.
> >
> Ah I know now.
> We are leaking outer header information inside the virtio net hdr, and outer header might be already stripped off by a different entity.
>
> I think the use case here is it's the same sw entity that owns the virtio net device does the encap/decap too.
No not exactly, we are leaking info between encap tunnels.
--
MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 14:09 ` Michael S. Tsirkin
@ 2023-02-08 14:29 ` Parav Pandit
2023-02-09 5:12 ` Heng Qi
1 sibling, 0 replies; 54+ messages in thread
From: Parav Pandit @ 2023-02-08 14:29 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, February 8, 2023 9:09 AM
> > > header: it allow users inside the tunnel control queueing outside.
> > > By observing packet loss some information leaks between tunnels.
> > >
> > I likely didn't understand. Can you please explain?
> >
> > Queuing is always done on the inner header with/without encapsulation.
> > Hash is always reported for inner header.
> > It is only adding the ability to hash even when outer header exists.
>
>
> If hashing just on outer header (currently the only option) then a given tunnel
> all lands in a given queue.
> Just keep that queue separate and users of this tunnel can not learn whether
> other queues are overflowing, and can not overflow other queues.
>
>
> If you hash inner header then user can flood device with packets of a given
> connection and the same connection in a different tunnel hashes to the same
> queue. Now one tunnel can
> - cause DoS for another tunnel
> - cause packet loss or latency triggering possible security bugs within guest
> - detect that another tunnel is using the connection by
> detecting its own packet loss or increased latency
>
Yes. It can lead to above issues.
Steering on inner is on best effort based sw implementations running on top of net device.
To avoid above issues, a hierarchical model is needed.
I am not aware of any.
To my knowledge, usually who care for above issues end up using a different net device for each VNI and achieve the desired hierarchy.
>
> > If queuing to be decided based on outer header (hash), then that is different.
> > Hashing both inner and outer in a flat q structure unlikely works, right?
> > Because both hashes can result in different q selection.
>
>
> That's the point.
>
> Is there any precedent in OSes for configuring things like this that we can look
> at?
>
ethtool -N (not yet part of virtio) is the closest match that can steer based on inner and outer both, but it is not hierarchical, and it is orthogonal to this feature.
>
> > >
> > > Ideas for solving this they all involve hashing both inner and outer
> > > header:
> > > 1- report two sets of hashes. overkill?
> > > 2- hash both headers together
> > > 2- add salt. can come from driver or device itself
> > >
> > > More ideas?
> > >
> > > --
> > > MST
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 14:09 ` Michael S. Tsirkin
2023-02-08 14:29 ` Parav Pandit
@ 2023-02-09 5:12 ` Heng Qi
2023-02-09 6:05 ` [virtio-dev] " Heng Qi
1 sibling, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-09 5:12 UTC (permalink / raw)
To: Michael S. Tsirkin, Parav Pandit
Cc: Jason Wang, virtio-comment, virtio-dev, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Wed, Feb 08, 2023 at 09:09:15AM -0500, Michael S. Tsirkin wrote:
> On Wed, Feb 08, 2023 at 02:00:14PM +0000, Parav Pandit wrote:
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, February 8, 2023 8:52 AM
> > >
> > > On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Wednesday, February 8, 2023 8:32 AM
> > > > >
> > > > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > > > >
> > > > > > [..]
> > > > > > > >>
> > > > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > > > In struct virtio_net_config we need two fields.
> > > > > > > > a. supported_hash_types (already exists) b.
> > > > > > > > supported_hash_tunnel_type
> > > > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > > > -> calculation is
> > > > > > > supported.
> > > > > > >
> > > > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > > > >
> > > > > > > >
> > > > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > > > absolute value indicating which outer header
> > > > > > > exists when inner header hash calculated.
> > > > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > > > clearer that its
> > > > > > > type.
> > > > > > >
> > > > > > > Sure.
> > > > > > >
> > > > > > > Thanks for your reply.
> > > > > >
> > > > > > I had one last question. Why do we need to inform the
> > > > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > > > Is this for debug? Or is there a use case that will process this value?
> > > > >
> > > > > Well we have hash_report which is kind of similar (and also kind of
> > > > > pointless but I think it's there because WHQL wants it).
> > > > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> > > udp etc).
> > > > OS can use this value to find tcp connection in a given namespace.
> > > >
> > > > > Maybe we can steal some bits
> > > > > from there instead of a new field?
> > > > >
> > > > I do not have problem adding extra bits. I just don't find that just telling that
> > > its vxlan or nvgre to the OS is useful.
> > > > If OS needs to know about outer header details, it needs to know the VNI
> > > information than just telling vxlan.
> > >
> > > This does make sense.
> > >
> > >
> > > > >
> > > > > I have a follow up question though: are we only hashing the inner
> > > > > header or both inner and outer header? Somewhat confused on this.
> > > > >
> > > > I understood as inner header. But worth to describe it. May be there. Need to
> > > read v8 patch.
> > >
> > > Hmm. I just realized that there's a security problem with hashing just the inner
> > > header: it allow users inside the tunnel control queueing outside.
> > > By observing packet loss some information leaks between tunnels.
> > >
> > I likely didn't understand. Can you please explain?
> >
> > Queuing is always done on the inner header with/without encapsulation.
> > Hash is always reported for inner header.
> > It is only adding the ability to hash even when outer header exists.
>
>
> If hashing just on outer header (currently the only option) then
> a given tunnel all lands in a given queue.
> Just keep that queue separate and users of this tunnel can not
> learn whether other queues are overflowing, and can not overflow
> other queues.
>
This is not a problem with the inner hash, it is a general problem with the outer hash.
I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash),
and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.
For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.
-----------
Tunnel t1 | |
VTEP_1 <=======================> (vni id1) |
| |
| VTEP0 | ======================> VM
Tunnel t2 | |
VTEP_2 <=======================> (vni id2) |
-----------
At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM
through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow.
Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or
switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.
Thanks.
>
> If you hash inner header then user can flood device with
> packets of a given connection and the same connection in a different
> tunnel hashes to the same queue. Now one tunnel can
> - cause DoS for another tunnel
> - cause packet loss or latency triggering possible security bugs within guest
> - detect that another tunnel is using the connection by
> detecting its own packet loss or increased latency
>
>
>
>
> > If queuing to be decided based on outer header (hash), then that is different.
> > Hashing both inner and outer in a flat q structure unlikely works, right?
> > Because both hashes can result in different q selection.
>
>
> That's the point.
>
> Is there any precedent in OSes for configuring things like this
> that we can look at?
>
>
> > >
> > > Ideas for solving this they all involve hashing both inner and outer
> > > header:
> > > 1- report two sets of hashes. overkill?
> > > 2- hash both headers together
> > > 2- add salt. can come from driver or device itself
> > >
> > > More ideas?
> > >
> > > --
> > > MST
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-08 12:21 ` Parav Pandit
@ 2023-02-09 5:20 ` Heng Qi
2023-02-09 5:34 ` Parav Pandit
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-09 5:20 UTC (permalink / raw)
To: Parav Pandit, Yuri Benditovich
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev, Cornelia Huck,
Xuan Zhuo, Jason Wang
在 2023/2/8 下午8:21, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Wednesday, February 8, 2023 1:11 AM
>>
>>
>> 在 2023/2/8 下午1:18, Parav Pandit 写道:
>>>> From: Heng Qi <hengqi@linux.alibaba.com>
>>>> Sent: Tuesday, February 7, 2023 10:25 PM
>>> [..]
>>>>>> Do you think we need both hash_types and hash_tunnel_types?
>>>>> In struct virtio_net_config we need two fields.
>>>>> a. supported_hash_types (already exists) b.
>>>>> supported_hash_tunnel_type
>>>>> -> bitmap indicating for which outer headers, inner hash calculation
>>>>> -> is
>>>> supported.
>>>>
>>>> Thanks for the suggestion, we seem to have reached an agreement.
>>>>
>>>>> In struct virtio_net_hdr we need two fields.
>>>>> a. hash_report (already exists)
>>>>> b. hash_tunnel_type 8 bits -> absolute value indicating which outer
>>>>> header
>>>> exists when inner header hash calculated.
>>>>> You already have it in your patch named as hash_report_tunnel.
>>>>> May be better to name as hash_report_tunnel_type to make it clearer
>>>>> that its
>>>> type.
>>>>
>>>> Sure.
>>>>
>>>> Thanks for your reply.
>>> I had one last question. Why do we need to inform the
>> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
>>> Is this for debug? Or is there a use case that will process this value?
>> The driver may use it to do some statistical information, or do some rx
>> classification based on the rx hash, and we'd better not hide information from
>> the driver.
>>
> Statistical information is better gathered via stats, instead of adding such code in driver data path.
A cautionary note : the source of hash_report_tunnel comes from the
discussion here
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html .
If I understand correctly, if virtio-net-hdr also has
hash_report_tunnel, how does the driver do statistics?
CC Yuri for some opinions.
Thanks.
>
> It is not about hiding.
> It has onus on the data path to detect the tunnel type and place that virtio_net_hdr.
> So lets find one use of it in data path which sw entity will use it.
>
>
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
* RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-09 5:20 ` [virtio-comment] " Heng Qi
@ 2023-02-09 5:34 ` Parav Pandit
2023-02-09 9:57 ` Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Parav Pandit @ 2023-02-09 5:34 UTC (permalink / raw)
To: Heng Qi, Yuri Benditovich
Cc: Michael S . Tsirkin, virtio-comment, virtio-dev, Cornelia Huck,
Xuan Zhuo, Jason Wang
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Thursday, February 9, 2023 12:21 AM
> >>>> Thanks for your reply.
> >>> I had one last question. Why do we need to inform the
> >> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> >>> Is this for debug? Or is there a use case that will process this value?
> >> The driver may use it to do some statistical information, or do some
> >> rx classification based on the rx hash, and we'd better not hide
> >> information from the driver.
> >>
> > Statistical information is better gathered via stats, instead of adding such code
> in driver data path.
>
> A cautionary note : the source of hash_report_tunnel comes from the
> discussion here
> https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
> If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how
> does the driver do statistics?
I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.
If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.
Below [1] says about driver doing some rx classification. It is unclear on inner or outer.
The response is not very clear to me, about how this field is useful.
[1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] RE: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-09 5:12 ` Heng Qi
@ 2023-02-09 6:05 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-02-09 6:05 UTC (permalink / raw)
To: Michael S. Tsirkin, Parav Pandit
Cc: Jason Wang, virtio-comment, virtio-dev, yuri.benditovich,
Cornelia Huck, Xuan Zhuo
On Thu, Feb 09, 2023 at 01:12:52PM +0800, Heng Qi wrote:
> On Wed, Feb 08, 2023 at 09:09:15AM -0500, Michael S. Tsirkin wrote:
> > On Wed, Feb 08, 2023 at 02:00:14PM +0000, Parav Pandit wrote:
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, February 8, 2023 8:52 AM
> > > >
> > > > On Wed, Feb 08, 2023 at 01:38:36PM +0000, Parav Pandit wrote:
> > > > >
> > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Sent: Wednesday, February 8, 2023 8:32 AM
> > > > > >
> > > > > > On Wed, Feb 08, 2023 at 05:18:32AM +0000, Parav Pandit wrote:
> > > > > > > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > > Sent: Tuesday, February 7, 2023 10:25 PM
> > > > > > >
> > > > > > > [..]
> > > > > > > > >>
> > > > > > > > >> Do you think we need both hash_types and hash_tunnel_types?
> > > > > > > > > In struct virtio_net_config we need two fields.
> > > > > > > > > a. supported_hash_types (already exists) b.
> > > > > > > > > supported_hash_tunnel_type
> > > > > > > > > -> bitmap indicating for which outer headers, inner hash
> > > > > > > > > -> calculation is
> > > > > > > > supported.
> > > > > > > >
> > > > > > > > Thanks for the suggestion, we seem to have reached an agreement.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > In struct virtio_net_hdr we need two fields.
> > > > > > > > > a. hash_report (already exists) b. hash_tunnel_type 8 bits ->
> > > > > > > > > absolute value indicating which outer header
> > > > > > > > exists when inner header hash calculated.
> > > > > > > > > You already have it in your patch named as hash_report_tunnel.
> > > > > > > > > May be better to name as hash_report_tunnel_type to make it
> > > > > > > > > clearer that its
> > > > > > > > type.
> > > > > > > >
> > > > > > > > Sure.
> > > > > > > >
> > > > > > > > Thanks for your reply.
> > > > > > >
> > > > > > > I had one last question. Why do we need to inform the
> > > > > > hash_report_tunnel_type of the outer header in the virtio_net_hdr?
> > > > > > > Is this for debug? Or is there a use case that will process this value?
> > > > > >
> > > > > > Well we have hash_report which is kind of similar (and also kind of
> > > > > > pointless but I think it's there because WHQL wants it).
> > > > > Hash_report is useful. It tells hash_value is in which namespace (ipv4-tcp/ipv4
> > > > udp etc).
> > > > > OS can use this value to find tcp connection in a given namespace.
> > > > >
> > > > > > Maybe we can steal some bits
> > > > > > from there instead of a new field?
> > > > > >
> > > > > I do not have problem adding extra bits. I just don't find that just telling that
> > > > its vxlan or nvgre to the OS is useful.
> > > > > If OS needs to know about outer header details, it needs to know the VNI
> > > > information than just telling vxlan.
> > > >
> > > > This does make sense.
> > > >
> > > >
> > > > > >
> > > > > > I have a follow up question though: are we only hashing the inner
> > > > > > header or both inner and outer header? Somewhat confused on this.
> > > > > >
> > > > > I understood as inner header. But worth to describe it. May be there. Need to
> > > > read v8 patch.
> > > >
> > > > Hmm. I just realized that there's a security problem with hashing just the inner
> > > > header: it allow users inside the tunnel control queueing outside.
> > > > By observing packet loss some information leaks between tunnels.
> > > >
> > > I likely didn't understand. Can you please explain?
> > >
> > > Queuing is always done on the inner header with/without encapsulation.
> > > Hash is always reported for inner header.
> > > It is only adding the ability to hash even when outer header exists.
> >
> >
> > If hashing just on outer header (currently the only option) then
> > a given tunnel all lands in a given queue.
> > Just keep that queue separate and users of this tunnel can not
> > learn whether other queues are overflowing, and can not overflow
> > other queues.
> >
>
>
> This is not a problem with the inner hash, it is a general problem with the outer hash.
> I communicated with our people who are doing cloud security (they are also one of the demanders of inner hash),
> and it is a common problem for one tunnel to attack another tunnel, which has nothing to do with inner hash or outer hash.
>
> For example, there is a tunnel t1; a tunnel t2; a tunnel endpoint VTEP0, and the vni id of t1 is id1, and the vni id of v2 is id2; a VM.
>
> -----------
> Tunnel t1 | |
> VTEP_1 <=======================> (vni id1) |
> | |
> | VTEP0 | ======================> VM
> Tunnel t2 | |
> VTEP_2 <=======================> (vni id2) |
> -----------
The description graph should look like this:
-----------
Tunnel t1 | |
VTEP_1 <=======================> (vni id1) |
| |
| VTEP | ======================> VM
Tunnel t2 | |
VTEP_2 <=======================> (vni id2) |
-----------
>
> At this time, regardless of the inner hash or the outer hash, the traffic of tunnel t1 and tunnel t2 will reach the VM
> through VTEP0 (whether it is a single queue or multiple queues), and may be placed on the same queue to cause queue overflow.
> Some current forwarding tools such as DPDK have good forwarding performance, and it is difficult to fill up the queue; or
> switch the attack traffic to the attack clusters; or connect the traffic of different tunnels to different network card ports or network devices.
>
> Thanks.
>
> >
> > If you hash inner header then user can flood device with
> > packets of a given connection and the same connection in a different
> > tunnel hashes to the same queue. Now one tunnel can
> > - cause DoS for another tunnel
> > - cause packet loss or latency triggering possible security bugs within guest
> > - detect that another tunnel is using the connection by
> > detecting its own packet loss or increased latency
> >
> >
> >
> >
> > > If queuing to be decided based on outer header (hash), then that is different.
> > > Hashing both inner and outer in a flat q structure unlikely works, right?
> > > Because both hashes can result in different q selection.
> >
> >
> > That's the point.
> >
> > Is there any precedent in OSes for configuring things like this
> > that we can look at?
> >
> >
> > > >
> > > > Ideas for solving this they all involve hashing both inner and outer
> > > > header:
> > > > 1- report two sets of hashes. overkill?
> > > > 2- hash both headers together
> > > > 2- add salt. can come from driver or device itself
> > > >
> > > > More ideas?
> > > >
> > > > --
> > > > MST
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-09 5:34 ` Parav Pandit
@ 2023-02-09 9:57 ` Heng Qi
2023-02-11 2:08 ` [virtio-comment] " Heng Qi
0 siblings, 1 reply; 54+ messages in thread
From: Heng Qi @ 2023-02-09 9:57 UTC (permalink / raw)
To: Parav Pandit, Michael S . Tsirkin
Cc: virtio-comment, virtio-dev, Cornelia Huck, Xuan Zhuo, Jason Wang,
Yuri Benditovich
在 2023/2/9 下午1:34, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Thursday, February 9, 2023 12:21 AM
>>>>>> Thanks for your reply.
>>>>> I had one last question. Why do we need to inform the
>>>> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
>>>>> Is this for debug? Or is there a use case that will process this value?
>>>> The driver may use it to do some statistical information, or do some
>>>> rx classification based on the rx hash, and we'd better not hide
>>>> information from the driver.
>>>>
>>> Statistical information is better gathered via stats, instead of adding such code
>> in driver data path.
>>
>> A cautionary note : the source of hash_report_tunnel comes from the
>> discussion here
>> https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
>> If I understand correctly, if virtio-net-hdr also has hash_report_tunnel, how
>> does the driver do statistics?
> I am asking to avoid having hash_report_tunnel in virtio_net_hdr if it is only for the statistics purpose.
Parav, I don't seem to follow this.
Note that the hash_report(le16) and padding reserved(le16) fields only
exist when VIRTIO_NET_F_HASH_REPORT is negotiated,
and hash_report_tunnel only uses the first 8 bits of padding_reserved,
which does not add any overhead.
Whether to reuse hash_report or use the new hash_report_tunnel initially
has been discussed:
https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .
> If it has more use than just statistics, I like to understand its use in rx packet processing flow, if there is any and to describe in commit log too.
I think there are still some important points, such as the kernel or
driver forwards the packet,
and the hash value is used when selecting the queue, but at this time
they want to know whether
the hash value comes from the inner or outer header.
What if it doesn't want to forward based on the hash calculated by the
inner header, but only uses the hash calculated by the outer header?
Thanks.
> Below [1] says about driver doing some rx classification. It is unclear on inner or outer.
> The response is not very clear to me, about how this field is useful.
>
> [1] https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
>
> If driver needs to know the statistics of it, it can query them from the device by introducing statistics cmd.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v7] virtio-net: support inner header hash
2023-02-09 9:57 ` Heng Qi
@ 2023-02-11 2:08 ` Heng Qi
0 siblings, 0 replies; 54+ messages in thread
From: Heng Qi @ 2023-02-11 2:08 UTC (permalink / raw)
To: Parav Pandit, Michael S . Tsirkin
Cc: virtio-comment, virtio-dev, Cornelia Huck, Xuan Zhuo, Jason Wang,
Yuri Benditovich
Any more discussions or comments? Please let me know if this is OK with you.
and the v8 version is released at
[1] https://lists.oasis-open.org/archives/virtio-dev/202302/msg00094.html
Thanks.
在 2023/2/9 下午5:57, Heng Qi 写道:
>
>
> 在 2023/2/9 下午1:34, Parav Pandit 写道:
>>> From: Heng Qi <hengqi@linux.alibaba.com>
>>> Sent: Thursday, February 9, 2023 12:21 AM
>>>>>>> Thanks for your reply.
>>>>>> I had one last question. Why do we need to inform the
>>>>> hash_report_tunnel_type of the outer header in the virtio_net_hdr?
>>>>>> Is this for debug? Or is there a use case that will process this
>>>>>> value?
>>>>> The driver may use it to do some statistical information, or do some
>>>>> rx classification based on the rx hash, and we'd better not hide
>>>>> information from the driver.
>>>>>
>>>> Statistical information is better gathered via stats, instead of
>>>> adding such code
>>> in driver data path.
>>>
>>> A cautionary note : the source of hash_report_tunnel comes from the
>>> discussion here
>>> https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
>>> If I understand correctly, if virtio-net-hdr also has
>>> hash_report_tunnel, how
>>> does the driver do statistics?
>> I am asking to avoid having hash_report_tunnel in virtio_net_hdr if
>> it is only for the statistics purpose.
>
> Parav, I don't seem to follow this.
>
> Note that the hash_report(le16) and padding reserved(le16) fields only
> exist when VIRTIO_NET_F_HASH_REPORT is negotiated,
> and hash_report_tunnel only uses the first 8 bits of padding_reserved,
> which does not add any overhead.
>
> Whether to reuse hash_report or use the new hash_report_tunnel
> initially has been discussed:
> https://lists.oasis-open.org/archives/virtio-dev/202211/msg00065.html .
>
>> If it has more use than just statistics, I like to understand its use
>> in rx packet processing flow, if there is any and to describe in
>> commit log too.
>
> I think there are still some important points, such as the kernel or
> driver forwards the packet,
> and the hash value is used when selecting the queue, but at this time
> they want to know whether
> the hash value comes from the inner or outer header.
> What if it doesn't want to forward based on the hash calculated by the
> inner header, but only uses the hash calculated by the outer header?
>
> Thanks.
>
>> Below [1] says about driver doing some rx classification. It is
>> unclear on inner or outer.
>> The response is not very clear to me, about how this field is useful.
>>
>> [1]
>> https://lists.oasis-open.org/archives/virtio-dev/202211/msg00063.html
>>
>> If driver needs to know the statistics of it, it can query them from
>> the device by introducing statistics cmd.
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines:
> https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 54+ messages in thread
end of thread, other threads:[~2023-02-11 2:08 UTC | newest]
Thread overview: 54+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-04 7:14 [PATCH v7] virtio-net: support inner header hash Heng Qi
2023-01-04 12:34 ` [virtio-comment] " Heng Qi
2023-01-04 12:37 ` Michael S. Tsirkin
2023-01-06 5:27 ` Michael S. Tsirkin
2023-01-06 6:42 ` [virtio-comment] " Heng Qi
2023-01-06 6:59 ` Michael S. Tsirkin
2023-01-09 2:43 ` [virtio-comment] Re: [virtio-dev] " Heng Qi
2023-01-09 8:59 ` Jason Wang
2023-01-09 11:34 ` Michael S. Tsirkin
2023-01-10 2:06 ` Jason Wang
2023-01-10 5:24 ` Michael S. Tsirkin
2023-01-10 5:57 ` Michael S. Tsirkin
2023-01-10 7:26 ` Heng Qi
2023-01-11 3:22 ` [virtio-comment] " Heng Qi
2023-01-11 4:45 ` Jason Wang
2023-01-11 9:49 ` Michael S. Tsirkin
2023-01-09 11:36 ` Michael S. Tsirkin
2023-01-10 7:46 ` Heng Qi
2023-01-09 11:39 ` Michael S. Tsirkin
2023-01-10 7:47 ` [virtio-comment] " Heng Qi
2023-01-11 9:45 ` Michael S. Tsirkin
2023-01-16 8:01 ` [virtio-comment] " Heng Qi
2023-01-16 8:18 ` [virtio-dev] " Cornelia Huck
2023-01-31 5:31 ` Heng Qi
2023-01-16 8:42 ` Jason Wang
2023-01-31 5:28 ` [virtio-dev] " Heng Qi
2023-02-08 2:30 ` Heng Qi
2023-02-08 3:19 ` Parav Pandit
2023-02-08 3:24 ` Heng Qi
2023-02-08 5:18 ` Parav Pandit
2023-02-08 6:11 ` Heng Qi
2023-02-08 12:21 ` Parav Pandit
2023-02-09 5:20 ` [virtio-comment] " Heng Qi
2023-02-09 5:34 ` Parav Pandit
2023-02-09 9:57 ` Heng Qi
2023-02-11 2:08 ` [virtio-comment] " Heng Qi
2023-02-08 13:31 ` [virtio-comment] " Michael S. Tsirkin
2023-02-08 13:38 ` Parav Pandit
2023-02-08 13:52 ` Michael S. Tsirkin
2023-02-08 14:00 ` Parav Pandit
2023-02-08 14:09 ` Michael S. Tsirkin
2023-02-08 14:29 ` Parav Pandit
2023-02-09 5:12 ` Heng Qi
2023-02-09 6:05 ` [virtio-dev] " Heng Qi
2023-02-08 14:05 ` Parav Pandit
2023-02-08 14:10 ` Michael S. Tsirkin
2023-01-18 23:45 ` [virtio-dev] " Parav Pandit
2023-01-31 5:57 ` [virtio-comment] " Heng Qi
2023-02-01 1:51 ` Parav Pandit
2023-02-01 6:47 ` Michael S. Tsirkin
2023-02-01 6:56 ` Michael S. Tsirkin
2023-02-02 3:55 ` Parav Pandit
2023-02-02 6:55 ` Heng Qi
2023-02-01 7:14 ` Heng Qi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.