* [PATCH v3] virtio_net: support inner header hash
@ 2022-12-05 6:36 Heng Qi
2022-12-09 6:09 ` Heng Qi
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Heng Qi @ 2022-12-05 6:36 UTC (permalink / raw)
To: virtio-comment, virtio-dev
Cc: Jason Wang, Michael S . Tsirkin, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
encapsulate the packets, the hash calculated using the outer header
of the receive packets is always fixed for the same flow packets,
i.e. they will be steered to the same receive queue.
We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
related bitmasks in \field{hash_types}, which instructs the device
to calculate the hash using the inner headers of GRE, VXLAN or
GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
added to report packet type when calculating hash over the inner header.
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
v2:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
v1:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 181 insertions(+), 9 deletions(-)
diff --git a/content.tex b/content.tex
index e863709..675da87 100644
--- a/content.tex
+++ b/content.tex
@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
channel.
+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
+ header hash for GRE, VXLAN and GENEVE-encapsulated packets.
+
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
to several segments when each of these smaller packets has UDP header.
\item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
- value and a type of calculated hash.
+ value, a type of calculated hash and a tunnel packet type if
+ VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
\item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
value. Device benefits from knowing the exact header length.
@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
\end{description}
\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
le16 num_buffers;
le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+ le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
+ le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
};
\end{lstlisting}
@@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
A device attempts to calculate a per-packet hash in the following cases:
\begin{itemize}
\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
\end{itemize}
If the feature VIRTIO_NET_F_RSS was negotiated:
\begin{itemize}
-\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
+\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
+but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
+\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
\item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
\end{itemize}
If the feature VIRTIO_NET_F_RSS was not negotiated:
\begin{itemize}
-\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
+\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
+except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
\item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
\end{itemize}
@@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
#define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
\end{lstlisting}
+Hash types applicable to inner payloads of GRE-encapsulated packets
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
+\end{lstlisting}
+Hash types applicable to inner payloads of VXLAN-encapsulated packets
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
+\end{lstlisting}
+Hash types applicable to inner payloads of GENEVE-encapsulated packets
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
+\end{lstlisting}
\subparagraph{IPv4 packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
@@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
\end{itemize}
+\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
+VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
+VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
+the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
+
+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
+payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner TCPv4 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemsize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner UDPv4 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item inner Source IP address
+ \item inner Destination IP address
+ \end{itemsize}
+ \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
+payloads are IPv6 packets without extension headers according to 'Enabled hash types'
+bitmasks as follows:
+\begin{itemize}
+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner TCPv6 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemsize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner UDPv6 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item inner Source IPv6 address
+ \item inner Destination IPv6 address
+ \end{itemsize}
+ \item Else the device does not calculate the hash
+\end{itemize}
+
+The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
+payloads are IPv6 packets with extension headers according to 'Enabled hash types'
+bitmasks as follows:
+\begin{itemsize}
+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner TCPv6 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \item inner Source TCP port
+ \item inner Destination TCP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
+ packet has an inner UDPv6 header in its payload, the hash is calculated over
+ the following fields:
+ \begin{itemsize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \item inner Source UDP port
+ \item inner Destination UDP port
+ \end{itemize}
+ \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
+ following fields:
+ \begin{itemsize}
+ \item Home address from the home address option in the inner IPv6 destination
+ options header. If the inner extension header is not present, use the
+ inner Source IPv6 address.
+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
+ associated inner extension header. If the inner extension header is not
+ present, use the inner Destination IPv6 address.
+ \end{itemize}
+ \item Else skip inner IPv6 extension headers and calculate the hash as defined
+ for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
+ packet without extension headers
+\end{itemsize}
+
\paragraph{Hash reporting for incoming packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
-If VIRTIO_NET_F_HASH_REPORT was negotiated and
- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
-and \field{hash_value} with the value of calculated hash.
+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
+hash for the packet, the device fills \field{hash_report} with the report type
+of calculated hash, and \field{hash_value} with the value of calculated hash.
+And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
+needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
+and \field{hash_report_tunnel} is no longer valid.
Possible values that the device can report in \field{hash_report} are defined below.
They correspond to supported hash types defined in
@@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
#define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
\end{lstlisting}
+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
+\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
+packet to the driver over the inner header hash calculation.
+Possible values that the device can report in \field{hash_report_tunnel}
+are defined below:
+
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_REPORT_GRE 1
+#define VIRTIO_NET_HASH_REPORT_VXLAN 2
+#define VIRTIO_NET_HASH_REPORT_GENEVE 3
+\end{lstlisting}
+
+The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
+VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
+VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
+of supported hash types defined in respectively
+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
+
\subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
--
2.19.1.6.gb485710b
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3] virtio_net: support inner header hash
2022-12-05 6:36 [PATCH v3] virtio_net: support inner header hash Heng Qi
@ 2022-12-09 6:09 ` Heng Qi
2022-12-14 9:01 ` [virtio-dev] " Heng Qi
2022-12-16 6:33 ` Jason Wang
2022-12-16 12:49 ` Michael S. Tsirkin
2 siblings, 1 reply; 15+ messages in thread
From: Heng Qi @ 2022-12-09 6:09 UTC (permalink / raw)
To: virtio-comment, virtio-dev
Cc: Jason Wang, Michael S . Tsirkin, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
Do you have any comments on this new version?
Thanks.
在 2022/12/5 下午2:36, Heng Qi 写道:
> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
> encapsulate the packets, the hash calculated using the outer header
> of the receive packets is always fixed for the same flow packets,
> i.e. they will be steered to the same receive queue.
>
> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
> related bitmasks in \field{hash_types}, which instructs the device
> to calculate the hash using the inner headers of GRE, VXLAN or
> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
> added to report packet type when calculating hash over the inner header.
>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 181 insertions(+), 9 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..675da87 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash and a tunnel packet type if
> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> -\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
> +but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
> +\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> -\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
> +except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
> +Hash types applicable to inner payloads of GRE-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
>
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
> +
> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> +payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv4 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv4 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> +payloads are IPv6 packets without extension headers according to 'Enabled hash types'
> +bitmasks as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
> +payloads are IPv6 packets with extension headers according to 'Enabled hash types'
> +bitmasks as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip inner IPv6 extension headers and calculate the hash as defined
> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
> + packet without extension headers
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
> +needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and \field{hash_report_tunnel} is no longer valid.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> +of supported hash types defined in respectively
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-09 6:09 ` Heng Qi
@ 2022-12-14 9:01 ` Heng Qi
2022-12-14 9:12 ` Jason Wang
0 siblings, 1 reply; 15+ messages in thread
From: Heng Qi @ 2022-12-14 9:01 UTC (permalink / raw)
To: virtio-comment, virtio-dev
Cc: Jason Wang, Michael S . Tsirkin, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
>
> Do you have any comments on this new version?
>
If the reviewers do not have more comments on
"[v3] virtio_net: support inner header hash",
then I submit an issue in the virtio-spec repository.
Please vote on whether this new feature is added to
the virtio specification.
The link is https://github.com/oasis-tcs/virtio-spec/issues/151.
Thanks.
> Thanks.
>
> 在 2022/12/5 下午2:36, Heng Qi 写道:
> >When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
> >encapsulate the packets, the hash calculated using the outer header
> >of the receive packets is always fixed for the same flow packets,
> >i.e. they will be steered to the same receive queue.
> >
> >We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
> >related bitmasks in \field{hash_types}, which instructs the device
> >to calculate the hash using the inner headers of GRE, VXLAN or
> >GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
> >added to report packet type when calculating hash over the inner header.
> >
> >Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> >Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >---
> >v2:
> > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> >
> >v1:
> > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > 2. Clarify some paragraphs. @Jason Wang
> > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> >
> > content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 181 insertions(+), 9 deletions(-)
> >
> >diff --git a/content.tex b/content.tex
> >index e863709..675da87 100644
> >--- a/content.tex
> >+++ b/content.tex
> >@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > channel.
> >+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
> >+ header hash for GRE, VXLAN and GENEVE-encapsulated packets.
> >+
> > \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
> > \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> >@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > to several segments when each of these smaller packets has UDP header.
> > \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> >- value and a type of calculated hash.
> >+ value, a type of calculated hash and a tunnel packet type if
> >+ VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> > value. Device benefits from knowing the exact header length.
> >@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > \end{description}
> > \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> >@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> > le16 num_buffers;
> > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >+ le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> >+ le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > };
> > \end{lstlisting}
> >@@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > A device attempts to calculate a per-packet hash in the following cases:
> > \begin{itemize}
> > \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> >-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> >+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
> > \end{itemize}
> > If the feature VIRTIO_NET_F_RSS was negotiated:
> > \begin{itemize}
> >-\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> >+\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
> >+but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
> >+\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> > \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> > \end{itemize}
> > If the feature VIRTIO_NET_F_RSS was not negotiated:
> > \begin{itemize}
> >-\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> >+\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
> >+except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
> > \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> > \end{itemize}
> >@@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
> > #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> > \end{lstlisting}
> >+Hash types applicable to inner payloads of GRE-encapsulated packets
> >+\begin{lstlisting}
> >+#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> >+\end{lstlisting}
> >+Hash types applicable to inner payloads of VXLAN-encapsulated packets
> >+\begin{lstlisting}
> >+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> >+\end{lstlisting}
> >+Hash types applicable to inner payloads of GENEVE-encapsulated packets
> >+\begin{lstlisting}
> >+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> >+\end{lstlisting}
> > \subparagraph{IPv4 packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> >@@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> > \end{itemize}
> >+\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
> >+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
> >+VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
> >+VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
> >+the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
> >+
> >+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> >+payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
> >+\begin{itemize}
> >+ \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner TCPv4 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IP address
> >+ \item inner Destination IP address
> >+ \item inner Source TCP port
> >+ \item inner Destination TCP port
> >+ \end{itemsize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner UDPv4 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IP address
> >+ \item inner Destination IP address
> >+ \item inner Source UDP port
> >+ \item inner Destination UDP port
> >+ \end{itemize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> >+ following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IP address
> >+ \item inner Destination IP address
> >+ \end{itemsize}
> >+ \item Else the device does not calculate the hash
> >+\end{itemize}
> >+
> >+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> >+payloads are IPv6 packets without extension headers according to 'Enabled hash types'
> >+bitmasks as follows:
> >+\begin{itemize}
> >+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner TCPv6 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IPv6 address
> >+ \item inner Destination IPv6 address
> >+ \item inner Source TCP port
> >+ \item inner Destination TCP port
> >+ \end{itemsize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner UDPv6 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IPv6 address
> >+ \item inner Destination IPv6 address
> >+ \item inner Source UDP port
> >+ \item inner Destination UDP port
> >+ \end{itemize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> >+ following fields:
> >+ \begin{itemsize}
> >+ \item inner Source IPv6 address
> >+ \item inner Destination IPv6 address
> >+ \end{itemsize}
> >+ \item Else the device does not calculate the hash
> >+\end{itemize}
> >+
> >+The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
> >+payloads are IPv6 packets with extension headers according to 'Enabled hash types'
> >+bitmasks as follows:
> >+\begin{itemsize}
> >+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner TCPv6 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemize}
> >+ \item Home address from the home address option in the inner IPv6 destination
> >+ options header. If the inner extension header is not present, use the
> >+ inner Source IPv6 address.
> >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >+ associated inner extension header. If the inner extension header is not
> >+ present, use the inner Destination IPv6 address.
> >+ \item inner Source TCP port
> >+ \item inner Destination TCP port
> >+ \end{itemize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> >+ packet has an inner UDPv6 header in its payload, the hash is calculated over
> >+ the following fields:
> >+ \begin{itemsize}
> >+ \item Home address from the home address option in the inner IPv6 destination
> >+ options header. If the inner extension header is not present, use the
> >+ inner Source IPv6 address.
> >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >+ associated inner extension header. If the inner extension header is not
> >+ present, use the inner Destination IPv6 address.
> >+ \item inner Source UDP port
> >+ \item inner Destination UDP port
> >+ \end{itemize}
> >+ \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
> >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> >+ following fields:
> >+ \begin{itemsize}
> >+ \item Home address from the home address option in the inner IPv6 destination
> >+ options header. If the inner extension header is not present, use the
> >+ inner Source IPv6 address.
> >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> >+ associated inner extension header. If the inner extension header is not
> >+ present, use the inner Destination IPv6 address.
> >+ \end{itemize}
> >+ \item Else skip inner IPv6 extension headers and calculate the hash as defined
> >+ for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
> >+ packet without extension headers
> >+\end{itemsize}
> >+
> > \paragraph{Hash reporting for incoming packets}
> > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
> >-If VIRTIO_NET_F_HASH_REPORT was negotiated and
> >- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> >-and \field{hash_value} with the value of calculated hash.
> >+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> >+hash for the packet, the device fills \field{hash_report} with the report type
> >+of calculated hash, and \field{hash_value} with the value of calculated hash.
> >+And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
> >+needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
> > If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> >-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> >+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> >+and \field{hash_report_tunnel} is no longer valid.
> > Possible values that the device can report in \field{hash_report} are defined below.
> > They correspond to supported hash types defined in
> >@@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> > \end{lstlisting}
> >+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> >+\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
> >+packet to the driver over the inner header hash calculation.
> >+Possible values that the device can report in \field{hash_report_tunnel}
> >+are defined below:
> >+
> >+\begin{lstlisting}
> >+#define VIRTIO_NET_HASH_REPORT_GRE 1
> >+#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> >+#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> >+\end{lstlisting}
> >+
> >+The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
> >+VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >+VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> >+of supported hash types defined in respectively
> >+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> >+
> > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
> > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-14 9:01 ` [virtio-dev] " Heng Qi
@ 2022-12-14 9:12 ` Jason Wang
2022-12-14 9:16 ` Heng Qi
0 siblings, 1 reply; 15+ messages in thread
From: Jason Wang @ 2022-12-14 9:12 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Dec 14, 2022 at 5:01 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
> On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
> >
> > Do you have any comments on this new version?
> >
>
> If the reviewers do not have more comments on
I plan to review this no later than the end of this week.
Thanks
> "[v3] virtio_net: support inner header hash",
> then I submit an issue in the virtio-spec repository.
> Please vote on whether this new feature is added to
> the virtio specification.
>
> The link is https://github.com/oasis-tcs/virtio-spec/issues/151.
>
> Thanks.
>
> > Thanks.
> >
> > 在 2022/12/5 下午2:36, Heng Qi 写道:
> > >When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
> > >encapsulate the packets, the hash calculated using the outer header
> > >of the receive packets is always fixed for the same flow packets,
> > >i.e. they will be steered to the same receive queue.
> > >
> > >We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
> > >related bitmasks in \field{hash_types}, which instructs the device
> > >to calculate the hash using the inner headers of GRE, VXLAN or
> > >GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
> > >added to report packet type when calculating hash over the inner header.
> > >
> > >Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > >Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > >---
> > >v2:
> > > 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> > > 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
> > >
> > >v1:
> > > 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> > > 2. Clarify some paragraphs. @Jason Wang
> > > 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
> > >
> > > content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
> > > 1 file changed, 181 insertions(+), 9 deletions(-)
> > >
> > >diff --git a/content.tex b/content.tex
> > >index e863709..675da87 100644
> > >--- a/content.tex
> > >+++ b/content.tex
> > >@@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> > > channel.
> > >+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
> > >+ header hash for GRE, VXLAN and GENEVE-encapsulated packets.
> > >+
> > > \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
> > > \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> > >@@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > to several segments when each of these smaller packets has UDP header.
> > > \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> > >- value and a type of calculated hash.
> > >+ value, a type of calculated hash and a tunnel packet type if
> > >+ VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> > > \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> > > value. Device benefits from knowing the exact header length.
> > >@@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > > \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> > >+\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > \end{description}
> > > \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > >@@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> > > le16 num_buffers;
> > > le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > > le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > >- le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > >+ le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > >+ le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> > > };
> > > \end{lstlisting}
> > >@@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > > A device attempts to calculate a per-packet hash in the following cases:
> > > \begin{itemize}
> > > \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> > >-\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> > >+\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
> > > \end{itemize}
> > > If the feature VIRTIO_NET_F_RSS was negotiated:
> > > \begin{itemize}
> > >-\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> > >+\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
> > >+but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
> > >+\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> > > \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> > > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> > > \end{itemize}
> > > If the feature VIRTIO_NET_F_RSS was not negotiated:
> > > \begin{itemize}
> > >-\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> > >+\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
> > >+except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
> > > \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> > > \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> > > \end{itemize}
> > >@@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > > #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
> > > #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> > > \end{lstlisting}
> > >+Hash types applicable to inner payloads of GRE-encapsulated packets
> > >+\begin{lstlisting}
> > >+#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> > >+\end{lstlisting}
> > >+Hash types applicable to inner payloads of VXLAN-encapsulated packets
> > >+\begin{lstlisting}
> > >+#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> > >+\end{lstlisting}
> > >+Hash types applicable to inner payloads of GENEVE-encapsulated packets
> > >+\begin{lstlisting}
> > >+#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> > >+\end{lstlisting}
> > > \subparagraph{IPv4 packets}
> > > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> > >@@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > > (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> > > \end{itemize}
> > >+\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
> > >+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
> > >+VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
> > >+VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
> > >+the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
> > >+
> > >+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> > >+payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
> > >+\begin{itemize}
> > >+ \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner TCPv4 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IP address
> > >+ \item inner Destination IP address
> > >+ \item inner Source TCP port
> > >+ \item inner Destination TCP port
> > >+ \end{itemsize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner UDPv4 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IP address
> > >+ \item inner Destination IP address
> > >+ \item inner Source UDP port
> > >+ \item inner Destination UDP port
> > >+ \end{itemize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> > >+ following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IP address
> > >+ \item inner Destination IP address
> > >+ \end{itemsize}
> > >+ \item Else the device does not calculate the hash
> > >+\end{itemize}
> > >+
> > >+The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> > >+payloads are IPv6 packets without extension headers according to 'Enabled hash types'
> > >+bitmasks as follows:
> > >+\begin{itemize}
> > >+ \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner TCPv6 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IPv6 address
> > >+ \item inner Destination IPv6 address
> > >+ \item inner Source TCP port
> > >+ \item inner Destination TCP port
> > >+ \end{itemsize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner UDPv6 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IPv6 address
> > >+ \item inner Destination IPv6 address
> > >+ \item inner Source UDP port
> > >+ \item inner Destination UDP port
> > >+ \end{itemize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> > >+ following fields:
> > >+ \begin{itemsize}
> > >+ \item inner Source IPv6 address
> > >+ \item inner Destination IPv6 address
> > >+ \end{itemsize}
> > >+ \item Else the device does not calculate the hash
> > >+\end{itemize}
> > >+
> > >+The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
> > >+payloads are IPv6 packets with extension headers according to 'Enabled hash types'
> > >+bitmasks as follows:
> > >+\begin{itemsize}
> > >+ \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner TCPv6 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemize}
> > >+ \item Home address from the home address option in the inner IPv6 destination
> > >+ options header. If the inner extension header is not present, use the
> > >+ inner Source IPv6 address.
> > >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > >+ associated inner extension header. If the inner extension header is not
> > >+ present, use the inner Destination IPv6 address.
> > >+ \item inner Source TCP port
> > >+ \item inner Destination TCP port
> > >+ \end{itemize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> > >+ packet has an inner UDPv6 header in its payload, the hash is calculated over
> > >+ the following fields:
> > >+ \begin{itemsize}
> > >+ \item Home address from the home address option in the inner IPv6 destination
> > >+ options header. If the inner extension header is not present, use the
> > >+ inner Source IPv6 address.
> > >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > >+ associated inner extension header. If the inner extension header is not
> > >+ present, use the inner Destination IPv6 address.
> > >+ \item inner Source UDP port
> > >+ \item inner Destination UDP port
> > >+ \end{itemize}
> > >+ \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
> > >+ VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> > >+ VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> > >+ following fields:
> > >+ \begin{itemsize}
> > >+ \item Home address from the home address option in the inner IPv6 destination
> > >+ options header. If the inner extension header is not present, use the
> > >+ inner Source IPv6 address.
> > >+ \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> > >+ associated inner extension header. If the inner extension header is not
> > >+ present, use the inner Destination IPv6 address.
> > >+ \end{itemize}
> > >+ \item Else skip inner IPv6 extension headers and calculate the hash as defined
> > >+ for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
> > >+ packet without extension headers
> > >+\end{itemsize}
> > >+
> > > \paragraph{Hash reporting for incoming packets}
> > > \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
> > >-If VIRTIO_NET_F_HASH_REPORT was negotiated and
> > >- the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> > >-and \field{hash_value} with the value of calculated hash.
> > >+If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> > >+hash for the packet, the device fills \field{hash_report} with the report type
> > >+of calculated hash, and \field{hash_value} with the value of calculated hash.
> > >+And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
> > >+needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
> > > If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> > >-hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> > >+hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> > >+and \field{hash_report_tunnel} is no longer valid.
> > > Possible values that the device can report in \field{hash_report} are defined below.
> > > They correspond to supported hash types defined in
> > >@@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> > > #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> > > \end{lstlisting}
> > >+If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> > >+\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
> > >+packet to the driver over the inner header hash calculation.
> > >+Possible values that the device can report in \field{hash_report_tunnel}
> > >+are defined below:
> > >+
> > >+\begin{lstlisting}
> > >+#define VIRTIO_NET_HASH_REPORT_GRE 1
> > >+#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> > >+#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> > >+\end{lstlisting}
> > >+
> > >+The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
> > >+VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
> > >+VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> > >+of supported hash types defined in respectively
> > >+\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> > >+
> > > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
> > > The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-14 9:12 ` Jason Wang
@ 2022-12-14 9:16 ` Heng Qi
2022-12-14 9:30 ` [virtio-comment] " Cornelia Huck
0 siblings, 1 reply; 15+ messages in thread
From: Heng Qi @ 2022-12-14 9:16 UTC (permalink / raw)
To: Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2022/12/14 下午5:12, Jason Wang 写道:
> On Wed, Dec 14, 2022 at 5:01 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>> On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
>>> Do you have any comments on this new version?
>>>
>> If the reviewers do not have more comments on
> I plan to review this no later than the end of this week.
Okay, I see.
@Cornelia Huck, Although I have submitted the voting issue, please wait
for a while for @Jason Wang to review, thank you.
> Thanks
>
>> "[v3] virtio_net: support inner header hash",
>> then I submit an issue in the virtio-spec repository.
>> Please vote on whether this new feature is added to
>> the virtio specification.
>>
>> The link is https://github.com/oasis-tcs/virtio-spec/issues/151.
>>
>> Thanks.
>>
>>> Thanks.
>>>
>>> 在 2022/12/5 下午2:36, Heng Qi 写道:
>>>> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
>>>> encapsulate the packets, the hash calculated using the outer header
>>>> of the receive packets is always fixed for the same flow packets,
>>>> i.e. they will be steered to the same receive queue.
>>>>
>>>> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
>>>> related bitmasks in \field{hash_types}, which instructs the device
>>>> to calculate the hash using the inner headers of GRE, VXLAN or
>>>> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
>>>> added to report packet type when calculating hash over the inner header.
>>>>
>>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>> ---
>>>> v2:
>>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>>>>
>>>> v1:
>>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>>> 2. Clarify some paragraphs. @Jason Wang
>>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>>>>
>>>> content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
>>>> 1 file changed, 181 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/content.tex b/content.tex
>>>> index e863709..675da87 100644
>>>> --- a/content.tex
>>>> +++ b/content.tex
>>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>>>> channel.
>>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
>>>> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
>>>> +
>>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>>>> to several segments when each of these smaller packets has UDP header.
>>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>>>> - value and a type of calculated hash.
>>>> + value, a type of calculated hash and a tunnel packet type if
>>>> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>>>> value. Device benefits from knowing the exact header length.
>>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
>>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
>>>> \end{description}
>>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>>>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>>>> le16 num_buffers;
>>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>>>> };
>>>> \end{lstlisting}
>>>> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>> A device attempts to calculate a per-packet hash in the following cases:
>>>> \begin{itemize}
>>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
>>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
>>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
>>>> \end{itemize}
>>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>>> \begin{itemize}
>>>> -\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
>>>> +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
>>>> +but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
>>>> +\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>>> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
>>>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
>>>> \end{itemize}
>>>> If the feature VIRTIO_NET_F_RSS was not negotiated:
>>>> \begin{itemize}
>>>> -\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
>>>> +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
>>>> +except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
>>>> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
>>>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
>>>> \end{itemize}
>>>> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
>>>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>>>> \end{lstlisting}
>>>> +Hash types applicable to inner payloads of GRE-encapsulated packets
>>>> +\begin{lstlisting}
>>>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>>>> +\end{lstlisting}
>>>> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
>>>> +\begin{lstlisting}
>>>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>>>> +\end{lstlisting}
>>>> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
>>>> +\begin{lstlisting}
>>>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>>>> +\end{lstlisting}
>>>> \subparagraph{IPv4 packets}
>>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
>>>> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
>>>> \end{itemize}
>>>> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
>>>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
>>>> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
>>>> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
>>>> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
>>>> +
>>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
>>>> +payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
>>>> +\begin{itemize}
>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner TCPv4 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IP address
>>>> + \item inner Destination IP address
>>>> + \item inner Source TCP port
>>>> + \item inner Destination TCP port
>>>> + \end{itemsize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner UDPv4 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IP address
>>>> + \item inner Destination IP address
>>>> + \item inner Source UDP port
>>>> + \item inner Destination UDP port
>>>> + \end{itemize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>>>> + following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IP address
>>>> + \item inner Destination IP address
>>>> + \end{itemsize}
>>>> + \item Else the device does not calculate the hash
>>>> +\end{itemize}
>>>> +
>>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
>>>> +payloads are IPv6 packets without extension headers according to 'Enabled hash types'
>>>> +bitmasks as follows:
>>>> +\begin{itemize}
>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner TCPv6 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IPv6 address
>>>> + \item inner Destination IPv6 address
>>>> + \item inner Source TCP port
>>>> + \item inner Destination TCP port
>>>> + \end{itemsize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner UDPv6 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IPv6 address
>>>> + \item inner Destination IPv6 address
>>>> + \item inner Source UDP port
>>>> + \item inner Destination UDP port
>>>> + \end{itemize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>>>> + following fields:
>>>> + \begin{itemsize}
>>>> + \item inner Source IPv6 address
>>>> + \item inner Destination IPv6 address
>>>> + \end{itemsize}
>>>> + \item Else the device does not calculate the hash
>>>> +\end{itemize}
>>>> +
>>>> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
>>>> +payloads are IPv6 packets with extension headers according to 'Enabled hash types'
>>>> +bitmasks as follows:
>>>> +\begin{itemsize}
>>>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner TCPv6 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemize}
>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>> + options header. If the inner extension header is not present, use the
>>>> + inner Source IPv6 address.
>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>> + associated inner extension header. If the inner extension header is not
>>>> + present, use the inner Destination IPv6 address.
>>>> + \item inner Source TCP port
>>>> + \item inner Destination TCP port
>>>> + \end{itemize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>>>> + packet has an inner UDPv6 header in its payload, the hash is calculated over
>>>> + the following fields:
>>>> + \begin{itemsize}
>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>> + options header. If the inner extension header is not present, use the
>>>> + inner Source IPv6 address.
>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>> + associated inner extension header. If the inner extension header is not
>>>> + present, use the inner Destination IPv6 address.
>>>> + \item inner Source UDP port
>>>> + \item inner Destination UDP port
>>>> + \end{itemize}
>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>>>> + following fields:
>>>> + \begin{itemsize}
>>>> + \item Home address from the home address option in the inner IPv6 destination
>>>> + options header. If the inner extension header is not present, use the
>>>> + inner Source IPv6 address.
>>>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>>>> + associated inner extension header. If the inner extension header is not
>>>> + present, use the inner Destination IPv6 address.
>>>> + \end{itemize}
>>>> + \item Else skip inner IPv6 extension headers and calculate the hash as defined
>>>> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
>>>> + packet without extension headers
>>>> +\end{itemsize}
>>>> +
>>>> \paragraph{Hash reporting for incoming packets}
>>>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>>>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>>>> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
>>>> -and \field{hash_value} with the value of calculated hash.
>>>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
>>>> +hash for the packet, the device fills \field{hash_report} with the report type
>>>> +of calculated hash, and \field{hash_value} with the value of calculated hash.
>>>> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
>>>> +needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
>>>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>>>> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
>>>> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
>>>> +and \field{hash_report_tunnel} is no longer valid.
>>>> Possible values that the device can report in \field{hash_report} are defined below.
>>>> They correspond to supported hash types defined in
>>>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>>>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>>>> \end{lstlisting}
>>>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>>>> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
>>>> +packet to the driver over the inner header hash calculation.
>>>> +Possible values that the device can report in \field{hash_report_tunnel}
>>>> +are defined below:
>>>> +
>>>> +\begin{lstlisting}
>>>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>>>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>>>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>>>> +\end{lstlisting}
>>>> +
>>>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
>>>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
>>>> +of supported hash types defined in respectively
>>>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>>>> +
>>>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-14 9:16 ` Heng Qi
@ 2022-12-14 9:30 ` Cornelia Huck
2022-12-14 9:39 ` Heng Qi
2022-12-15 7:46 ` Michael S. Tsirkin
0 siblings, 2 replies; 15+ messages in thread
From: Cornelia Huck @ 2022-12-14 9:30 UTC (permalink / raw)
To: Heng Qi, Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Xuan Zhuo
On Wed, Dec 14 2022, Heng Qi <hengqi@linux.alibaba.com> wrote:
> 在 2022/12/14 下午5:12, Jason Wang 写道:
>> On Wed, Dec 14, 2022 at 5:01 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>>> On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
>>>> Do you have any comments on this new version?
>>>>
>>> If the reviewers do not have more comments on
>> I plan to review this no later than the end of this week.
>
> Okay, I see.
>
> @Cornelia Huck, Although I have submitted the voting issue, please wait
> for a while for @Jason Wang to review, thank you.
Ack, will do.
(I'm not sure if I'll start voting that close to the end of the year
anyway; many people in western countries will be offline until early
January soon.)
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-comment] Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-14 9:30 ` [virtio-comment] " Cornelia Huck
@ 2022-12-14 9:39 ` Heng Qi
2022-12-15 7:46 ` Michael S. Tsirkin
1 sibling, 0 replies; 15+ messages in thread
From: Heng Qi @ 2022-12-14 9:39 UTC (permalink / raw)
To: Cornelia Huck, Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Xuan Zhuo
在 2022/12/14 下午5:30, Cornelia Huck 写道:
> On Wed, Dec 14 2022, Heng Qi <hengqi@linux.alibaba.com> wrote:
>
>> 在 2022/12/14 下午5:12, Jason Wang 写道:
>>> On Wed, Dec 14, 2022 at 5:01 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>>>> On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
>>>>> Do you have any comments on this new version?
>>>>>
>>>> If the reviewers do not have more comments on
>>> I plan to review this no later than the end of this week.
>> Okay, I see.
>>
>> @Cornelia Huck, Although I have submitted the voting issue, please wait
>> for a while for @Jason Wang to review, thank you.
> Ack, will do.
>
> (I'm not sure if I'll start voting that close to the end of the year
> anyway; many people in western countries will be offline until early
> January soon.)
Ok, I got it.
Thanks.
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-dev] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-14 9:30 ` [virtio-comment] " Cornelia Huck
2022-12-14 9:39 ` Heng Qi
@ 2022-12-15 7:46 ` Michael S. Tsirkin
1 sibling, 0 replies; 15+ messages in thread
From: Michael S. Tsirkin @ 2022-12-15 7:46 UTC (permalink / raw)
To: Cornelia Huck
Cc: Heng Qi, Jason Wang, virtio-comment, virtio-dev,
Yuri Benditovich, Xuan Zhuo
On Wed, Dec 14, 2022 at 10:30:02AM +0100, Cornelia Huck wrote:
> On Wed, Dec 14 2022, Heng Qi <hengqi@linux.alibaba.com> wrote:
>
> > 在 2022/12/14 下午5:12, Jason Wang 写道:
> >> On Wed, Dec 14, 2022 at 5:01 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
> >>> On Fri, Dec 09, 2022 at 02:09:18PM +0800, Heng Qi wrote:
> >>>> Do you have any comments on this new version?
> >>>>
> >>> If the reviewers do not have more comments on
> >> I plan to review this no later than the end of this week.
> >
> > Okay, I see.
> >
> > @Cornelia Huck, Although I have submitted the voting issue, please wait
> > for a while for @Jason Wang to review, thank you.
>
> Ack, will do.
>
> (I'm not sure if I'll start voting that close to the end of the year
> anyway; many people in western countries will be offline until early
> January soon.)
Yes, let's start early January.
--
MST
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3] virtio_net: support inner header hash
2022-12-05 6:36 [PATCH v3] virtio_net: support inner header hash Heng Qi
2022-12-09 6:09 ` Heng Qi
@ 2022-12-16 6:33 ` Jason Wang
2022-12-16 10:14 ` Heng Qi
2022-12-16 12:49 ` Michael S. Tsirkin
2 siblings, 1 reply; 15+ messages in thread
From: Jason Wang @ 2022-12-16 6:33 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Mon, Dec 5, 2022 at 2:36 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
> encapsulate the packets, the hash calculated using the outer header
> of the receive packets is always fixed for the same flow packets,
> i.e. they will be steered to the same receive queue.
>
> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
I think we can simply use VIRTIO_NET_F_HASH_TUNNEL now. And introduce
VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extension?
> related bitmasks in \field{hash_types}, which instructs the device
> to calculate the hash using the inner headers of GRE, VXLAN or
> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
> added to report packet type when calculating hash over the inner header.
>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v2:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 181 insertions(+), 9 deletions(-)
>
> diff --git a/content.tex b/content.tex
> index e863709..675da87 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> to several segments when each of these smaller packets has UDP header.
>
> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> - value and a type of calculated hash.
> + value, a type of calculated hash and a tunnel packet type if
> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>
> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> value. Device benefits from knowing the exact header length.
> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
> le16 num_buffers;
> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
Only if both VIRTIO_NET_F_HASH_REPORT and
VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> };
> \end{lstlisting}
>
> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> A device attempts to calculate a per-packet hash in the following cases:
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
The tunnel type can only be reported when both HASH_REPORT and
HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> -\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
> +but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
> +\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
I think we can avoid this change by clarifying the feature dependency
where the tunnel hash types are defined.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> -\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
> +except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
Why exclude those tunnel hash types from _F_MQ? Or do we want to say
the automatic packet steering could be done via the inner packet when
negotiated?
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
> +Hash types applicable to inner payloads of GRE-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> +\end{lstlisting}
> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> +\end{lstlisting}
> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> +\end{lstlisting}
>
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
> +
> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> +payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv4 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv4 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IP address
> + \item inner Destination IP address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
> +payloads are IPv6 packets without extension headers according to 'Enabled hash types'
> +bitmasks as follows:
> +\begin{itemize}
> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemsize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item inner Source IPv6 address
> + \item inner Destination IPv6 address
> + \end{itemsize}
> + \item Else the device does not calculate the hash
> +\end{itemize}
> +
> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
> +payloads are IPv6 packets with extension headers according to 'Enabled hash types'
> +bitmasks as follows:
> +\begin{itemsize}
> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner TCPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source TCP port
> + \item inner Destination TCP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
> + packet has an inner UDPv6 header in its payload, the hash is calculated over
> + the following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \item inner Source UDP port
> + \item inner Destination UDP port
> + \end{itemize}
> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
> + following fields:
> + \begin{itemsize}
> + \item Home address from the home address option in the inner IPv6 destination
> + options header. If the inner extension header is not present, use the
> + inner Source IPv6 address.
> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
> + associated inner extension header. If the inner extension header is not
> + present, use the inner Destination IPv6 address.
> + \end{itemize}
> + \item Else skip inner IPv6 extension headers and calculate the hash as defined
> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
> + packet without extension headers
> +\end{itemsize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
> -and \field{hash_value} with the value of calculated hash.
> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
> +hash for the packet, the device fills \field{hash_report} with the report type
> +of calculated hash, and \field{hash_value} with the value of calculated hash.
> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
> +needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
>
> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
> +and \field{hash_report_tunnel} is no longer valid.
>
> Possible values that the device can report in \field{hash_report} are defined below.
> They correspond to supported hash types defined in
> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> +of supported hash types defined in respectively
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
I think we can simply say "They correspond to supported hash types
defined in \ref{xyz}" as what is done for non-tunnel hash types?
Thanks
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> --
> 2.19.1.6.gb485710b
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3] virtio_net: support inner header hash
2022-12-16 6:33 ` Jason Wang
@ 2022-12-16 10:14 ` Heng Qi
2022-12-20 14:27 ` [virtio-comment] " Heng Qi
0 siblings, 1 reply; 15+ messages in thread
From: Heng Qi @ 2022-12-16 10:14 UTC (permalink / raw)
To: Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2022/12/16 下午2:33, Jason Wang 写道:
> On Mon, Dec 5, 2022 at 2:36 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
>> encapsulate the packets, the hash calculated using the outer header
>> of the receive packets is always fixed for the same flow packets,
>> i.e. they will be steered to the same receive queue.
>>
>> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
> I think we can simply use VIRTIO_NET_F_HASH_TUNNEL now. And introduce
> VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extension?
Well, I very much agree.
>> related bitmasks in \field{hash_types}, which instructs the device
>> to calculate the hash using the inner headers of GRE, VXLAN or
>> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
>> added to report packet type when calculating hash over the inner header.
>>
>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> v2:
>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>>
>> v1:
>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>> 2. Clarify some paragraphs. @Jason Wang
>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>>
>> content.tex | 190 +++++++++++++++++++++++++++++++++++++++++++++++++---
>> 1 file changed, 181 insertions(+), 9 deletions(-)
>>
>> diff --git a/content.tex b/content.tex
>> index e863709..675da87 100644
>> --- a/content.tex
>> +++ b/content.tex
>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>> channel.
>>
>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports inner
>> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
>> +
>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>>
>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>> to several segments when each of these smaller packets has UDP header.
>>
>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>> - value and a type of calculated hash.
>> + value, a type of calculated hash and a tunnel packet type if
>> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>
>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>> value. Device benefits from knowing the exact header length.
>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
>> \end{description}
>>
>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device Types / Network Device / Device O
>> le16 num_buffers;
>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
> Only if both VIRTIO_NET_F_HASH_REPORT and
> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
Yes, I will make it clear.
>
>> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT negotiated)
>> };
>> \end{lstlisting}
>>
>> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> A device attempts to calculate a per-packet hash in the following cases:
>> \begin{itemize}
>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value, the hash type and the tunnel packet type.
> The tunnel type can only be reported when both HASH_REPORT and
> HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
Yes, I will make it clear.
>
>> \end{itemize}
>>
>> If the feature VIRTIO_NET_F_RSS was negotiated:
>> \begin{itemize}
>> -\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
>> +\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask,
>> +but VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
>> +\field{hash_types} can only be used when the feature VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> I think we can avoid this change by clarifying the feature dependency
> where the tunnel hash types are defined.
Ok, I'll do it in the next version.
>
>> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
>> \end{itemize}
>>
>> If the feature VIRTIO_NET_F_RSS was not negotiated:
>> \begin{itemize}
>> -\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
>> +\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask,
>> +except VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
> Why exclude those tunnel hash types from _F_MQ? Or do we want to say
> the automatic packet steering could be done via the inner packet when
> negotiated?
As far as I know about spec, there are two ways to select a queue to
place a packet.
The first one is _F_MQ. In this scenario, if a packet arrives for the
first time, it will be
randomly placed (not based on hash) in a queue , subsequent packets of
the same
data flow will be placed in the same queue. Here is what the spec says:
“For uni-directional protocols, or where no packets have been
transmitted yet, the device MAY steer a packet to a random queue out of
the specified receiveq1\ldots receiveqn.";
The second is _F_RSS. In this scenario, the device calculates the hash
based on the
outer or inner header, and uses the hash value to select the rx queue.
Therefore,
when _F_RSS is not negotiated, we seem to have no initiative to actively
select queues.
But there is no need to actively explain the exclusion of these types here.
Cc Xuan.
>
>> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
>> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
>> \end{itemize}
>> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>> \end{lstlisting}
>> +Hash types applicable to inner payloads of GRE-encapsulated packets
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>> +\end{lstlisting}
>> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>> +\end{lstlisting}
>> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>> +\end{lstlisting}
>>
>> \subparagraph{IPv4 packets}
>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
>> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
>> \end{itemize}
>>
>> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}
>> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
>> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
>> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as one of
>> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and VIRTIO_NET_HASH_TYPE_UDP_EX.
>> +
>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
>> +payloads are IPv4 packets according to 'Enabled hash types' bitmasks as follows:
>> +\begin{itemize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner TCPv4 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemsize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner UDPv4 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item inner Source IP address
>> + \item inner Destination IP address
>> + \end{itemsize}
>> + \item Else the device does not calculate the hash
>> +\end{itemize}
>> +
>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated packets whose inner
>> +payloads are IPv6 packets without extension headers according to 'Enabled hash types'
>> +bitmasks as follows:
>> +\begin{itemize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner TCPv6 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemsize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner UDPv6 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item inner Source IPv6 address
>> + \item inner Destination IPv6 address
>> + \end{itemsize}
>> + \item Else the device does not calculate the hash
>> +\end{itemize}
>> +
>> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated packets whose inner
>> +payloads are IPv6 packets with extension headers according to 'Enabled hash types'
>> +bitmasks as follows:
>> +\begin{itemsize}
>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner TCPv6 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \item inner Source TCP port
>> + \item inner Destination TCP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or GENEVE-encapsulated
>> + packet has an inner UDPv6 header in its payload, the hash is calculated over
>> + the following fields:
>> + \begin{itemsize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \item inner Source UDP port
>> + \item inner Destination UDP port
>> + \end{itemize}
>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
>> + VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated over the
>> + following fields:
>> + \begin{itemsize}
>> + \item Home address from the home address option in the inner IPv6 destination
>> + options header. If the inner extension header is not present, use the
>> + inner Source IPv6 address.
>> + \item IPv6 address that is contained in the Routing-Header-Type-2 from the
>> + associated inner extension header. If the inner extension header is not
>> + present, use the inner Destination IPv6 address.
>> + \end{itemize}
>> + \item Else skip inner IPv6 extension headers and calculate the hash as defined
>> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner payload is an IPv6
>> + packet without extension headers
>> +\end{itemsize}
>> +
>> \paragraph{Hash reporting for incoming packets}
>> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>>
>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>> - the device has calculated the hash for the packet, the device fills \field{hash_report} with the report type of calculated hash
>> -and \field{hash_value} with the value of calculated hash.
>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has calculated the
>> +hash for the packet, the device fills \field{hash_report} with the report type
>> +of calculated hash, and \field{hash_value} with the value of calculated hash.
>> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the device also
>> +needs to fills \field{hash_report_tunnel} with the type of the tunnel packet,
>>
>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>> -hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE.
>> +hash was not calculated, the device sets \field{hash_report} to VIRTIO_NET_HASH_REPORT_NONE,
>> +and \field{hash_report_tunnel} is no longer valid.
>>
>> Possible values that the device can report in \field{hash_report} are defined below.
>> They correspond to supported hash types defined in
>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>> \end{lstlisting}
>>
>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
>> +packet to the driver over the inner header hash calculation.
>> +Possible values that the device can report in \field{hash_report_tunnel}
>> +are defined below:
>> +
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>> +\end{lstlisting}
>> +
>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
>> +of supported hash types defined in respectively
>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>> +
> I think we can simply say "They correspond to supported hash types
> defined in \ref{xyz}" as what is done for non-tunnel hash types?
Ok, I agree with you.
Thanks.
>
> Thanks
>
>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>
>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>> --
>> 2.19.1.6.gb485710b
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3] virtio_net: support inner header hash
2022-12-05 6:36 [PATCH v3] virtio_net: support inner header hash Heng Qi
2022-12-09 6:09 ` Heng Qi
2022-12-16 6:33 ` Jason Wang
@ 2022-12-16 12:49 ` Michael S. Tsirkin
2022-12-19 2:52 ` [virtio-comment] " Heng Qi
2 siblings, 1 reply; 15+ messages in thread
From: Michael S. Tsirkin @ 2022-12-16 12:49 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
On Mon, Dec 05, 2022 at 02:36:39PM +0800, Heng Qi wrote:
> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> \end{lstlisting}
>
> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
> +packet to the driver over the inner header hash calculation.
> +Possible values that the device can report in \field{hash_report_tunnel}
> +are defined below:
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> +\end{lstlisting}
> +
> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> +of supported hash types defined in respectively
> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> +
> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>
> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
If the new feature flag is negotiated, we need to spell out more clearly what are the rules
for packets that are not encapsulated. Is hash_report_tunnel set to 0
then? Another comment is that we keep repeating GRE/VXLAN/GENEVE too
many times. Let's add a paragraph defining a concept e.g. a "tunnel" or
"tunneled packets", explaining how they are handled at a high level,
and then just refer to the tunnel everywhere.
Let's also add external references to specifications documenting the
relevant tunnel types.
> --
> 2.19.1.6.gb485710b
^ permalink raw reply [flat|nested] 15+ messages in thread
* [virtio-comment] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-16 12:49 ` Michael S. Tsirkin
@ 2022-12-19 2:52 ` Heng Qi
0 siblings, 0 replies; 15+ messages in thread
From: Heng Qi @ 2022-12-19 2:52 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, virtio-dev, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2022/12/16 下午8:49, Michael S. Tsirkin 写道:
> On Mon, Dec 05, 2022 at 02:36:39PM +0800, Heng Qi wrote:
>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>> \end{lstlisting}
>>
>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>> +\field{hash_report_tunnel} can report the type of the tunnel-encapsulated
>> +packet to the driver over the inner header hash calculation.
>> +Possible values that the device can report in \field{hash_report_tunnel}
>> +are defined below:
>> +
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>> +\end{lstlisting}
>> +
>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN and
>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to VIRTIO_NET_HASH_TYPE_GRE_INNER,
>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
>> +of supported hash types defined in respectively
>> +\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
>> +
>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>>
>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> If the new feature flag is negotiated, we need to spell out more clearly what are the rules
> for packets that are not encapsulated. Is hash_report_tunnel set to 0
Yes, we should. When the _TUNNEL feature is negotiated, for
non-encapsulated packets we set \field{hash_report_tunnel} to 0.
> then? Another comment is that we keep repeating GRE/VXLAN/GENEVE too
> many times. Let's add a paragraph defining a concept e.g. a "tunnel" or
> "tunneled packets", explaining how they are handled at a high level,
> and then just refer to the tunnel everywhere.
> Let's also add external references to specifications documenting the
> relevant tunnel types.
Ok, I'll try to make this clear.
Thanks.
>
>
>> --
>> 2.19.1.6.gb485710b
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-comment] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-16 10:14 ` Heng Qi
@ 2022-12-20 14:27 ` Heng Qi
2022-12-21 3:15 ` Jason Wang
0 siblings, 1 reply; 15+ messages in thread
From: Heng Qi @ 2022-12-20 14:27 UTC (permalink / raw)
To: Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2022/12/16 下午6:14, Heng Qi 写道:
>
>
> 在 2022/12/16 下午2:33, Jason Wang 写道:
>> On Mon, Dec 5, 2022 at 2:36 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>>> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
>>> encapsulate the packets, the hash calculated using the outer header
>>> of the receive packets is always fixed for the same flow packets,
>>> i.e. they will be steered to the same receive queue.
>>>
>>> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
>> I think we can simply use VIRTIO_NET_F_HASH_TUNNEL now. And introduce
>> VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extension?
>
> Well, I very much agree.
>
>>> related bitmasks in \field{hash_types}, which instructs the device
>>> to calculate the hash using the inner headers of GRE, VXLAN or
>>> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
>>> added to report packet type when calculating hash over the inner
>>> header.
>>>
>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> v2:
>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash.
>>> @Jason Wang
>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
>>> @Jason Wang, @Michael S. Tsirkin
>>>
>>> v1:
>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>> 2. Clarify some paragraphs. @Jason Wang
>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.
>>> @Yuri Benditovich
>>>
>>> content.tex | 190
>>> +++++++++++++++++++++++++++++++++++++++++++++++++---
>>> 1 file changed, 181 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/content.tex b/content.tex
>>> index e863709..675da87 100644
>>> --- a/content.tex
>>> +++ b/content.tex
>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device
>>> Types / Network Device / Feature bits
>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>>> channel.
>>>
>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports
>>> inner
>>> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
>>> +
>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
>>> coalescing.
>>>
>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device
>>> Types / Network Device / Feature bits
>>> to several segments when each of these smaller packets has UDP
>>> header.
>>>
>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>>> - value and a type of calculated hash.
>>> + value, a type of calculated hash and a tunnel packet type if
>>> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>>
>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
>>> \field{hdr_len}
>>> value. Device benefits from knowing the exact header length.
>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit
>>> requirements}\label{sec:Device Types / Network Device
>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
>>> VIRTIO_NET_F_HOST_TSO6.
>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires
>>> VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
>>> \end{description}
>>>
>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
>>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device
>>> Types / Network Device / Device O
>>> le16 num_buffers;
>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT
>>> negotiated)
>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT
>>> negotiated)
>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>>> negotiated)
>>> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
>>> negotiated)
>> Only if both VIRTIO_NET_F_HASH_REPORT and
>> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
>
> Yes, I will make it clear.
>
>>
>>> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>>> negotiated)
>>> };
>>> \end{lstlisting}
>>>
>>> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming
>>> Packets}\label{sec:Device Types / Network
>>> A device attempts to calculate a per-packet hash in the following
>>> cases:
>>> \begin{itemize}
>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
>>> the hash to determine the receive virtqueue to place incoming packets.
>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>>> device reports the hash value and the hash type with the packet.
>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>>> device reports the hash value, the hash type and the tunnel packet
>>> type.
>> The tunnel type can only be reported when both HASH_REPORT and
>> HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
>
> Yes, I will make it clear.
>
>>
>>> \end{itemize}
>>>
>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>> \begin{itemize}
>>> -\item The device uses \field{hash_types} of the
>>> virtio_net_rss_config structure as 'Enabled hash types' bitmask.
>>> +\item The device uses \field{hash_types} of the
>>> virtio_net_rss_config structure as 'Enabled hash types' bitmask,
>>> +but VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
>>> +\field{hash_types} can only be used when the feature
>>> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>> I think we can avoid this change by clarifying the feature dependency
>> where the tunnel hash types are defined.
>
> Ok, I'll do it in the next version.
>
>>
>>> \item The device uses a key as defined in \field{hash_key_data}
>>> and \field{hash_key_length} of the virtio_net_rss_config structure (see
>>> \ref{sec:Device Types / Network Device / Device Operation /
>>> Control Virtqueue / Receive-side scaling (RSS) / Setting RSS
>>> parameters}).
>>> \end{itemize}
>>>
>>> If the feature VIRTIO_NET_F_RSS was not negotiated:
>>> \begin{itemize}
>>> -\item The device uses \field{hash_types} of the
>>> virtio_net_hash_config structure as 'Enabled hash types' bitmask.
>>> +\item The device uses \field{hash_types} of the
>>> virtio_net_hash_config structure as 'Enabled hash types' bitmask,
>>> +except VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
>> Why exclude those tunnel hash types from _F_MQ? Or do we want to say
>> the automatic packet steering could be done via the inner packet when
>> negotiated?
>
> As far as I know about spec, there are two ways to select a queue to
> place a packet.
>
> The first one is _F_MQ. In this scenario, if a packet arrives for the
> first time, it will be
> randomly placed (not based on hash) in a queue , subsequent packets of
> the same
> data flow will be placed in the same queue. Here is what the spec says:
> “For uni-directional protocols, or where no packets have been
> transmitted yet, the device MAY steer a packet to a random queue out
> of the specified receiveq1\ldots receiveqn.";
>
> The second is _F_RSS. In this scenario, the device calculates the hash
> based on the
> outer or inner header, and uses the hash value to select the rx queue.
> Therefore,
> when _F_RSS is not negotiated, we seem to have no initiative to
> actively select queues.
> But there is no need to actively explain the exclusion of these types
> here.
>
> Cc Xuan.
>
Hi Jason, what do you think about this?
Thanks.
>>
>>> \item The device uses a key as defined in \field{hash_key_data}
>>> and \field{hash_key_length} of the virtio_net_hash_config structure
>>> (see
>>> \ref{sec:Device Types / Network Device / Device Operation /
>>> Control Virtqueue / Automatic receive steering in multiqueue mode /
>>> Hash calculation}).
>>> \end{itemize}
>>> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming
>>> Packets}\label{sec:Device Types / Network
>>> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
>>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>>> \end{lstlisting}
>>> +Hash types applicable to inner payloads of GRE-encapsulated packets
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>>> +\end{lstlisting}
>>> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>>> +\end{lstlisting}
>>> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>>> +\end{lstlisting}
>>>
>>> \subparagraph{IPv4 packets}
>>> \label{sec:Device Types / Network Device / Device Operation /
>>> Processing of Incoming Packets / Hash calculation for incoming
>>> packets / IPv4 packets}
>>> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming
>>> Packets}\label{sec:Device Types / Network
>>> (see \ref{sec:Device Types / Network Device / Device Operation /
>>> Processing of Incoming Packets / Hash calculation for incoming
>>> packets / IPv6 packets without extension header}).
>>> \end{itemize}
>>>
>>> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated
>>> packets}
>>> +\label{sec:Device Types / Network Device / Device Operation /
>>> Processing of Incoming Packets / Hash calculation for incoming
>>> packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
>>> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
>>> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as
>>> one of
>>> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and
>>> VIRTIO_NET_HASH_TYPE_UDP_EX.
>>> +
>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
>>> packets whose inner
>>> +payloads are IPv4 packets according to 'Enabled hash types'
>>> bitmasks as follows:
>>> +\begin{itemize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner TCPv4 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemsize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner UDPv4 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>> over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IP address
>>> + \item inner Destination IP address
>>> + \end{itemsize}
>>> + \item Else the device does not calculate the hash
>>> +\end{itemize}
>>> +
>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
>>> packets whose inner
>>> +payloads are IPv6 packets without extension headers according to
>>> 'Enabled hash types'
>>> +bitmasks as follows:
>>> +\begin{itemize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner TCPv6 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemsize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner UDPv6 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>> over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item inner Source IPv6 address
>>> + \item inner Destination IPv6 address
>>> + \end{itemsize}
>>> + \item Else the device does not calculate the hash
>>> +\end{itemize}
>>> +
>>> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated
>>> packets whose inner
>>> +payloads are IPv6 packets with extension headers according to
>>> 'Enabled hash types'
>>> +bitmasks as follows:
>>> +\begin{itemsize}
>>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner TCPv6 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemize}
>>> + \item Home address from the home address option in the
>>> inner IPv6 destination
>>> + options header. If the inner extension header is not
>>> present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the
>>> Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner
>>> extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \item inner Source TCP port
>>> + \item inner Destination TCP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>> GENEVE-encapsulated
>>> + packet has an inner UDPv6 header in its payload, the hash is
>>> calculated over
>>> + the following fields:
>>> + \begin{itemsize}
>>> + \item Home address from the home address option in the
>>> inner IPv6 destination
>>> + options header. If the inner extension header is not
>>> present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the
>>> Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner
>>> extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \item inner Source UDP port
>>> + \item inner Destination UDP port
>>> + \end{itemize}
>>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>> over the
>>> + following fields:
>>> + \begin{itemsize}
>>> + \item Home address from the home address option in the
>>> inner IPv6 destination
>>> + options header. If the inner extension header is not
>>> present, use the
>>> + inner Source IPv6 address.
>>> + \item IPv6 address that is contained in the
>>> Routing-Header-Type-2 from the
>>> + associated inner extension header. If the inner
>>> extension header is not
>>> + present, use the inner Destination IPv6 address.
>>> + \end{itemize}
>>> + \item Else skip inner IPv6 extension headers and calculate the
>>> hash as defined
>>> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner
>>> payload is an IPv6
>>> + packet without extension headers
>>> +\end{itemsize}
>>> +
>>> \paragraph{Hash reporting for incoming packets}
>>> \label{sec:Device Types / Network Device / Device Operation /
>>> Processing of Incoming Packets / Hash reporting for incoming packets}
>>>
>>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>>> - the device has calculated the hash for the packet, the device
>>> fills \field{hash_report} with the report type of calculated hash
>>> -and \field{hash_value} with the value of calculated hash.
>>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has
>>> calculated the
>>> +hash for the packet, the device fills \field{hash_report} with the
>>> report type
>>> +of calculated hash, and \field{hash_value} with the value of
>>> calculated hash.
>>> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the
>>> device also
>>> +needs to fills \field{hash_report_tunnel} with the type of the
>>> tunnel packet,
>>>
>>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>>> -hash was not calculated, the device sets \field{hash_report} to
>>> VIRTIO_NET_HASH_REPORT_NONE.
>>> +hash was not calculated, the device sets \field{hash_report} to
>>> VIRTIO_NET_HASH_REPORT_NONE,
>>> +and \field{hash_report_tunnel} is no longer valid.
>>>
>>> Possible values that the device can report in \field{hash_report}
>>> are defined below.
>>> They correspond to supported hash types defined in
>>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming
>>> Packets}\label{sec:Device Types / Network
>>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>>> \end{lstlisting}
>>>
>>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>>> +\field{hash_report_tunnel} can report the type of the
>>> tunnel-encapsulated
>>> +packet to the driver over the inner header hash calculation.
>>> +Possible values that the device can report in
>>> \field{hash_report_tunnel}
>>> +are defined below:
>>> +
>>> +\begin{lstlisting}
>>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>>> +\end{lstlisting}
>>> +
>>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN
>>> and
>>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to
>>> VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
>>> +of supported hash types defined in respectively
>>> +\ref{sec:Device Types / Network Device / Device Operation /
>>> Processing of Incoming Packets / Hash calculation for incoming
>>> packets / Supported/enabled hash types}.
>>> +
>> I think we can simply say "They correspond to supported hash types
>> defined in \ref{xyz}" as what is done for non-tunnel hash types?
>
> Ok, I agree with you.
>
> Thanks.
>
>>
>> Thanks
>>
>>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
>>> Device / Device Operation / Control Virtqueue}
>>>
>>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>>> --
>>> 2.19.1.6.gb485710b
>>>
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines:
> https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-comment] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-20 14:27 ` [virtio-comment] " Heng Qi
@ 2022-12-21 3:15 ` Jason Wang
2022-12-21 6:16 ` Heng Qi
0 siblings, 1 reply; 15+ messages in thread
From: Jason Wang @ 2022-12-21 3:15 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Tue, Dec 20, 2022 at 10:27 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
>
>
> 在 2022/12/16 下午6:14, Heng Qi 写道:
> >
> >
> > 在 2022/12/16 下午2:33, Jason Wang 写道:
> >> On Mon, Dec 5, 2022 at 2:36 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
> >>> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
> >>> encapsulate the packets, the hash calculated using the outer header
> >>> of the receive packets is always fixed for the same flow packets,
> >>> i.e. they will be steered to the same receive queue.
> >>>
> >>> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
> >> I think we can simply use VIRTIO_NET_F_HASH_TUNNEL now. And introduce
> >> VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extension?
> >
> > Well, I very much agree.
> >
> >>> related bitmasks in \field{hash_types}, which instructs the device
> >>> to calculate the hash using the inner headers of GRE, VXLAN or
> >>> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
> >>> added to report packet type when calculating hash over the inner
> >>> header.
> >>>
> >>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> >>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >>> ---
> >>> v2:
> >>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash.
> >>> @Jason Wang
> >>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
> >>> @Jason Wang, @Michael S. Tsirkin
> >>>
> >>> v1:
> >>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> >>> 2. Clarify some paragraphs. @Jason Wang
> >>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.
> >>> @Yuri Benditovich
> >>>
> >>> content.tex | 190
> >>> +++++++++++++++++++++++++++++++++++++++++++++++++---
> >>> 1 file changed, 181 insertions(+), 9 deletions(-)
> >>>
> >>> diff --git a/content.tex b/content.tex
> >>> index e863709..675da87 100644
> >>> --- a/content.tex
> >>> +++ b/content.tex
> >>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device
> >>> Types / Network Device / Feature bits
> >>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> >>> channel.
> >>>
> >>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports
> >>> inner
> >>> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
> >>> +
> >>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
> >>> coalescing.
> >>>
> >>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> >>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device
> >>> Types / Network Device / Feature bits
> >>> to several segments when each of these smaller packets has UDP
> >>> header.
> >>>
> >>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
> >>> - value and a type of calculated hash.
> >>> + value, a type of calculated hash and a tunnel packet type if
> >>> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> >>>
> >>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
> >>> \field{hdr_len}
> >>> value. Device benefits from knowing the exact header length.
> >>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit
> >>> requirements}\label{sec:Device Types / Network Device
> >>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
> >>> VIRTIO_NET_F_HOST_TSO6.
> >>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires
> >>> VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> >>> \end{description}
> >>>
> >>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
> >>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> >>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device
> >>> Types / Network Device / Device O
> >>> le16 num_buffers;
> >>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT
> >>> negotiated)
> >>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT
> >>> negotiated)
> >>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
> >>> negotiated)
> >>> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
> >>> negotiated)
> >> Only if both VIRTIO_NET_F_HASH_REPORT and
> >> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
> >
> > Yes, I will make it clear.
> >
> >>
> >>> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
> >>> negotiated)
> >>> };
> >>> \end{lstlisting}
> >>>
> >>> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming
> >>> Packets}\label{sec:Device Types / Network
> >>> A device attempts to calculate a per-packet hash in the following
> >>> cases:
> >>> \begin{itemize}
> >>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
> >>> the hash to determine the receive virtqueue to place incoming packets.
> >>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
> >>> device reports the hash value and the hash type with the packet.
> >>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
> >>> device reports the hash value, the hash type and the tunnel packet
> >>> type.
> >> The tunnel type can only be reported when both HASH_REPORT and
> >> HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
> >
> > Yes, I will make it clear.
> >
> >>
> >>> \end{itemize}
> >>>
> >>> If the feature VIRTIO_NET_F_RSS was negotiated:
> >>> \begin{itemize}
> >>> -\item The device uses \field{hash_types} of the
> >>> virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> >>> +\item The device uses \field{hash_types} of the
> >>> virtio_net_rss_config structure as 'Enabled hash types' bitmask,
> >>> +but VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
> >>> +\field{hash_types} can only be used when the feature
> >>> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
> >> I think we can avoid this change by clarifying the feature dependency
> >> where the tunnel hash types are defined.
> >
> > Ok, I'll do it in the next version.
> >
> >>
> >>> \item The device uses a key as defined in \field{hash_key_data}
> >>> and \field{hash_key_length} of the virtio_net_rss_config structure (see
> >>> \ref{sec:Device Types / Network Device / Device Operation /
> >>> Control Virtqueue / Receive-side scaling (RSS) / Setting RSS
> >>> parameters}).
> >>> \end{itemize}
> >>>
> >>> If the feature VIRTIO_NET_F_RSS was not negotiated:
> >>> \begin{itemize}
> >>> -\item The device uses \field{hash_types} of the
> >>> virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> >>> +\item The device uses \field{hash_types} of the
> >>> virtio_net_hash_config structure as 'Enabled hash types' bitmask,
> >>> +except VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
> >> Why exclude those tunnel hash types from _F_MQ? Or do we want to say
> >> the automatic packet steering could be done via the inner packet when
> >> negotiated?
> >
> > As far as I know about spec, there are two ways to select a queue to
> > place a packet.
> >
> > The first one is _F_MQ. In this scenario, if a packet arrives for the
> > first time, it will be
> > randomly placed (not based on hash) in a queue , subsequent packets of
> > the same
> > data flow will be placed in the same queue.
See the part of automatic steering:
"""
When multiqueue is enabled by VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command,
the device MUST use automatic receive steering based on packet flow.
Programming of the receive steering classificator is implicit. After
the driver transmitted a packet of a flow on transmitqX, the device
SHOULD cause incoming packets for that flow to be steered to
receiveqX.
"""
So it doesn't exclude the possibility of using hash for steering. In
this case should we steer based on inner or outer?
Thanks
Here is what the spec says:
> > “For uni-directional protocols, or where no packets have been
> > transmitted yet, the device MAY steer a packet to a random queue out
> > of the specified receiveq1\ldots receiveqn.";
> >
> > The second is _F_RSS. In this scenario, the device calculates the hash
> > based on the
> > outer or inner header, and uses the hash value to select the rx queue.
> > Therefore,
> > when _F_RSS is not negotiated, we seem to have no initiative to
> > actively select queues.
> > But there is no need to actively explain the exclusion of these types
> > here.
> >
> > Cc Xuan.
> >
>
> Hi Jason, what do you think about this?
>
> Thanks.
>
> >>
> >>> \item The device uses a key as defined in \field{hash_key_data}
> >>> and \field{hash_key_length} of the virtio_net_hash_config structure
> >>> (see
> >>> \ref{sec:Device Types / Network Device / Device Operation /
> >>> Control Virtqueue / Automatic receive steering in multiqueue mode /
> >>> Hash calculation}).
> >>> \end{itemize}
> >>> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming
> >>> Packets}\label{sec:Device Types / Network
> >>> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
> >>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> >>> \end{lstlisting}
> >>> +Hash types applicable to inner payloads of GRE-encapsulated packets
> >>> +\begin{lstlisting}
> >>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
> >>> +\end{lstlisting}
> >>> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
> >>> +\begin{lstlisting}
> >>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
> >>> +\end{lstlisting}
> >>> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
> >>> +\begin{lstlisting}
> >>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
> >>> +\end{lstlisting}
> >>>
> >>> \subparagraph{IPv4 packets}
> >>> \label{sec:Device Types / Network Device / Device Operation /
> >>> Processing of Incoming Packets / Hash calculation for incoming
> >>> packets / IPv4 packets}
> >>> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming
> >>> Packets}\label{sec:Device Types / Network
> >>> (see \ref{sec:Device Types / Network Device / Device Operation /
> >>> Processing of Incoming Packets / Hash calculation for incoming
> >>> packets / IPv6 packets without extension header}).
> >>> \end{itemize}
> >>>
> >>> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated
> >>> packets}
> >>> +\label{sec:Device Types / Network Device / Device Operation /
> >>> Processing of Incoming Packets / Hash calculation for incoming
> >>> packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
> >>> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
> >>> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as
> >>> one of
> >>> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and
> >>> VIRTIO_NET_HASH_TYPE_UDP_EX.
> >>> +
> >>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
> >>> packets whose inner
> >>> +payloads are IPv4 packets according to 'Enabled hash types'
> >>> bitmasks as follows:
> >>> +\begin{itemize}
> >>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner TCPv4 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IP address
> >>> + \item inner Destination IP address
> >>> + \item inner Source TCP port
> >>> + \item inner Destination TCP port
> >>> + \end{itemsize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner UDPv4 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IP address
> >>> + \item inner Destination IP address
> >>> + \item inner Source UDP port
> >>> + \item inner Destination UDP port
> >>> + \end{itemize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
> >>> over the
> >>> + following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IP address
> >>> + \item inner Destination IP address
> >>> + \end{itemsize}
> >>> + \item Else the device does not calculate the hash
> >>> +\end{itemize}
> >>> +
> >>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
> >>> packets whose inner
> >>> +payloads are IPv6 packets without extension headers according to
> >>> 'Enabled hash types'
> >>> +bitmasks as follows:
> >>> +\begin{itemize}
> >>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner TCPv6 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IPv6 address
> >>> + \item inner Destination IPv6 address
> >>> + \item inner Source TCP port
> >>> + \item inner Destination TCP port
> >>> + \end{itemsize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner UDPv6 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IPv6 address
> >>> + \item inner Destination IPv6 address
> >>> + \item inner Source UDP port
> >>> + \item inner Destination UDP port
> >>> + \end{itemize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
> >>> over the
> >>> + following fields:
> >>> + \begin{itemsize}
> >>> + \item inner Source IPv6 address
> >>> + \item inner Destination IPv6 address
> >>> + \end{itemsize}
> >>> + \item Else the device does not calculate the hash
> >>> +\end{itemize}
> >>> +
> >>> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated
> >>> packets whose inner
> >>> +payloads are IPv6 packets with extension headers according to
> >>> 'Enabled hash types'
> >>> +bitmasks as follows:
> >>> +\begin{itemsize}
> >>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner TCPv6 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemize}
> >>> + \item Home address from the home address option in the
> >>> inner IPv6 destination
> >>> + options header. If the inner extension header is not
> >>> present, use the
> >>> + inner Source IPv6 address.
> >>> + \item IPv6 address that is contained in the
> >>> Routing-Header-Type-2 from the
> >>> + associated inner extension header. If the inner
> >>> extension header is not
> >>> + present, use the inner Destination IPv6 address.
> >>> + \item inner Source TCP port
> >>> + \item inner Destination TCP port
> >>> + \end{itemize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
> >>> GENEVE-encapsulated
> >>> + packet has an inner UDPv6 header in its payload, the hash is
> >>> calculated over
> >>> + the following fields:
> >>> + \begin{itemsize}
> >>> + \item Home address from the home address option in the
> >>> inner IPv6 destination
> >>> + options header. If the inner extension header is not
> >>> present, use the
> >>> + inner Source IPv6 address.
> >>> + \item IPv6 address that is contained in the
> >>> Routing-Header-Type-2 from the
> >>> + associated inner extension header. If the inner
> >>> extension header is not
> >>> + present, use the inner Destination IPv6 address.
> >>> + \item inner Source UDP port
> >>> + \item inner Destination UDP port
> >>> + \end{itemize}
> >>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
> >>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
> >>> over the
> >>> + following fields:
> >>> + \begin{itemsize}
> >>> + \item Home address from the home address option in the
> >>> inner IPv6 destination
> >>> + options header. If the inner extension header is not
> >>> present, use the
> >>> + inner Source IPv6 address.
> >>> + \item IPv6 address that is contained in the
> >>> Routing-Header-Type-2 from the
> >>> + associated inner extension header. If the inner
> >>> extension header is not
> >>> + present, use the inner Destination IPv6 address.
> >>> + \end{itemize}
> >>> + \item Else skip inner IPv6 extension headers and calculate the
> >>> hash as defined
> >>> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner
> >>> payload is an IPv6
> >>> + packet without extension headers
> >>> +\end{itemsize}
> >>> +
> >>> \paragraph{Hash reporting for incoming packets}
> >>> \label{sec:Device Types / Network Device / Device Operation /
> >>> Processing of Incoming Packets / Hash reporting for incoming packets}
> >>>
> >>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
> >>> - the device has calculated the hash for the packet, the device
> >>> fills \field{hash_report} with the report type of calculated hash
> >>> -and \field{hash_value} with the value of calculated hash.
> >>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has
> >>> calculated the
> >>> +hash for the packet, the device fills \field{hash_report} with the
> >>> report type
> >>> +of calculated hash, and \field{hash_value} with the value of
> >>> calculated hash.
> >>> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the
> >>> device also
> >>> +needs to fills \field{hash_report_tunnel} with the type of the
> >>> tunnel packet,
> >>>
> >>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
> >>> -hash was not calculated, the device sets \field{hash_report} to
> >>> VIRTIO_NET_HASH_REPORT_NONE.
> >>> +hash was not calculated, the device sets \field{hash_report} to
> >>> VIRTIO_NET_HASH_REPORT_NONE,
> >>> +and \field{hash_report_tunnel} is no longer valid.
> >>>
> >>> Possible values that the device can report in \field{hash_report}
> >>> are defined below.
> >>> They correspond to supported hash types defined in
> >>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming
> >>> Packets}\label{sec:Device Types / Network
> >>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
> >>> \end{lstlisting}
> >>>
> >>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
> >>> +\field{hash_report_tunnel} can report the type of the
> >>> tunnel-encapsulated
> >>> +packet to the driver over the inner header hash calculation.
> >>> +Possible values that the device can report in
> >>> \field{hash_report_tunnel}
> >>> +are defined below:
> >>> +
> >>> +\begin{lstlisting}
> >>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
> >>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
> >>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
> >>> +\end{lstlisting}
> >>> +
> >>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN
> >>> and
> >>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to
> >>> VIRTIO_NET_HASH_TYPE_GRE_INNER,
> >>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
> >>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
> >>> +of supported hash types defined in respectively
> >>> +\ref{sec:Device Types / Network Device / Device Operation /
> >>> Processing of Incoming Packets / Hash calculation for incoming
> >>> packets / Supported/enabled hash types}.
> >>> +
> >> I think we can simply say "They correspond to supported hash types
> >> defined in \ref{xyz}" as what is done for non-tunnel hash types?
> >
> > Ok, I agree with you.
> >
> > Thanks.
> >
> >>
> >> Thanks
> >>
> >>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
> >>> Device / Device Operation / Control Virtqueue}
> >>>
> >>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> >>> --
> >>> 2.19.1.6.gb485710b
> >>>
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines:
> > https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [virtio-comment] Re: [PATCH v3] virtio_net: support inner header hash
2022-12-21 3:15 ` Jason Wang
@ 2022-12-21 6:16 ` Heng Qi
0 siblings, 0 replies; 15+ messages in thread
From: Heng Qi @ 2022-12-21 6:16 UTC (permalink / raw)
To: Jason Wang
Cc: virtio-comment, virtio-dev, Michael S . Tsirkin,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2022/12/21 上午11:15, Jason Wang 写道:
> On Tue, Dec 20, 2022 at 10:27 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>>
>>
>> 在 2022/12/16 下午6:14, Heng Qi 写道:
>>>
>>> 在 2022/12/16 下午2:33, Jason Wang 写道:
>>>> On Mon, Dec 5, 2022 at 2:36 PM Heng Qi <hengqi@linux.alibaba.com> wrote:
>>>>> When VIRTIO_NET_F_RSS is negotiated and the tunnel is used to
>>>>> encapsulate the packets, the hash calculated using the outer header
>>>>> of the receive packets is always fixed for the same flow packets,
>>>>> i.e. they will be steered to the same receive queue.
>>>>>
>>>>> We add a VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER feature bit and
>>>> I think we can simply use VIRTIO_NET_F_HASH_TUNNEL now. And introduce
>>>> VIRTIO_NET_F_HASH_TUNNEL_XYZ for future extension?
>>> Well, I very much agree.
>>>
>>>>> related bitmasks in \field{hash_types}, which instructs the device
>>>>> to calculate the hash using the inner headers of GRE, VXLAN or
>>>>> GENEVE-encapsulated packets. Besides, \field{hash_report_tunnel} are
>>>>> added to report packet type when calculating hash over the inner
>>>>> header.
>>>>>
>>>>> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>> ---
>>>>> v2:
>>>>> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash.
>>>>> @Jason Wang
>>>>> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}.
>>>>> @Jason Wang, @Michael S. Tsirkin
>>>>>
>>>>> v1:
>>>>> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
>>>>> 2. Clarify some paragraphs. @Jason Wang
>>>>> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE.
>>>>> @Yuri Benditovich
>>>>>
>>>>> content.tex | 190
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++---
>>>>> 1 file changed, 181 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/content.tex b/content.tex
>>>>> index e863709..675da87 100644
>>>>> --- a/content.tex
>>>>> +++ b/content.tex
>>>>> @@ -3084,6 +3084,9 @@ \subsection{Feature bits}\label{sec:Device
>>>>> Types / Network Device / Feature bits
>>>>> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>>>>> channel.
>>>>>
>>>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER(52)] Device supports
>>>>> inner
>>>>> + header hash for GRE, VXLAN and GENEVE-encapsulated packets.
>>>>> +
>>>>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
>>>>> coalescing.
>>>>>
>>>>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
>>>>> @@ -3095,7 +3098,8 @@ \subsection{Feature bits}\label{sec:Device
>>>>> Types / Network Device / Feature bits
>>>>> to several segments when each of these smaller packets has UDP
>>>>> header.
>>>>>
>>>>> \item[VIRTIO_NET_F_HASH_REPORT(57)] Device can report per-packet hash
>>>>> - value and a type of calculated hash.
>>>>> + value, a type of calculated hash and a tunnel packet type if
>>>>> + VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>>>>
>>>>> \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact
>>>>> \field{hdr_len}
>>>>> value. Device benefits from knowing the exact header length.
>>>>> @@ -3140,6 +3144,7 @@ \subsubsection{Feature bit
>>>>> requirements}\label{sec:Device Types / Network Device
>>>>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
>>>>> VIRTIO_NET_F_HOST_TSO6.
>>>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>>>> +\item[VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER] Requires
>>>>> VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
>>>>> \end{description}
>>>>>
>>>>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
>>>>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>>>>> @@ -3386,7 +3391,8 @@ \subsection{Device Operation}\label{sec:Device
>>>>> Types / Network Device / Device O
>>>>> le16 num_buffers;
>>>>> le32 hash_value; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>> negotiated)
>>>>> le16 hash_report; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>> negotiated)
>>>>> - le16 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>> negotiated)
>>>>> + le8 hash_report_tunnel; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>> negotiated)
>>>> Only if both VIRTIO_NET_F_HASH_REPORT and
>>>> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
>>> Yes, I will make it clear.
>>>
>>>>> + le8 padding_reserved; (Only if VIRTIO_NET_F_HASH_REPORT
>>>>> negotiated)
>>>>> };
>>>>> \end{lstlisting}
>>>>>
>>>>> @@ -3837,19 +3843,22 @@ \subsubsection{Processing of Incoming
>>>>> Packets}\label{sec:Device Types / Network
>>>>> A device attempts to calculate a per-packet hash in the following
>>>>> cases:
>>>>> \begin{itemize}
>>>>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
>>>>> the hash to determine the receive virtqueue to place incoming packets.
>>>>> -\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>>>>> device reports the hash value and the hash type with the packet.
>>>>> +\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>>>>> device reports the hash value, the hash type and the tunnel packet
>>>>> type.
>>>> The tunnel type can only be reported when both HASH_REPORT and
>>>> HASH_GRE_VXLAN_GENEVE_INNER are negotiated?
>>> Yes, I will make it clear.
>>>
>>>>> \end{itemize}
>>>>>
>>>>> If the feature VIRTIO_NET_F_RSS was negotiated:
>>>>> \begin{itemize}
>>>>> -\item The device uses \field{hash_types} of the
>>>>> virtio_net_rss_config structure as 'Enabled hash types' bitmask.
>>>>> +\item The device uses \field{hash_types} of the
>>>>> virtio_net_rss_config structure as 'Enabled hash types' bitmask,
>>>>> +but VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER in
>>>>> +\field{hash_types} can only be used when the feature
>>>>> VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER is negotiated.
>>>> I think we can avoid this change by clarifying the feature dependency
>>>> where the tunnel hash types are defined.
>>> Ok, I'll do it in the next version.
>>>
>>>>> \item The device uses a key as defined in \field{hash_key_data}
>>>>> and \field{hash_key_length} of the virtio_net_rss_config structure (see
>>>>> \ref{sec:Device Types / Network Device / Device Operation /
>>>>> Control Virtqueue / Receive-side scaling (RSS) / Setting RSS
>>>>> parameters}).
>>>>> \end{itemize}
>>>>>
>>>>> If the feature VIRTIO_NET_F_RSS was not negotiated:
>>>>> \begin{itemize}
>>>>> -\item The device uses \field{hash_types} of the
>>>>> virtio_net_hash_config structure as 'Enabled hash types' bitmask.
>>>>> +\item The device uses \field{hash_types} of the
>>>>> virtio_net_hash_config structure as 'Enabled hash types' bitmask,
>>>>> +except VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and VIRTIO_NET_HASH_TYPE_GENEVE_INNER.
>>>> Why exclude those tunnel hash types from _F_MQ? Or do we want to say
>>>> the automatic packet steering could be done via the inner packet when
>>>> negotiated?
>>> As far as I know about spec, there are two ways to select a queue to
>>> place a packet.
>>>
>>> The first one is _F_MQ. In this scenario, if a packet arrives for the
>>> first time, it will be
>>> randomly placed (not based on hash) in a queue , subsequent packets of
>>> the same
>>> data flow will be placed in the same queue.
> See the part of automatic steering:
>
> """
> When multiqueue is enabled by VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command,
> the device MUST use automatic receive steering based on packet flow.
> Programming of the receive steering classificator is implicit. After
> the driver transmitted a packet of a flow on transmitqX, the device
> SHOULD cause incoming packets for that flow to be steered to
> receiveqX.
> """
>
> So it doesn't exclude the possibility of using hash for steering. In
> this case should we steer based on inner or outer?
If so, the packet flow of automatic receive steering has the possibility
of hash-based operations,
and I will keep the tunnel hash types in F_MQ in the next version.
Thanks.
>
> Thanks
>
> Here is what the spec says:
>>> “For uni-directional protocols, or where no packets have been
>>> transmitted yet, the device MAY steer a packet to a random queue out
>>> of the specified receiveq1\ldots receiveqn.";
>>>
>>> The second is _F_RSS. In this scenario, the device calculates the hash
>>> based on the
>>> outer or inner header, and uses the hash value to select the rx queue.
>>> Therefore,
>>> when _F_RSS is not negotiated, we seem to have no initiative to
>>> actively select queues.
>>> But there is no need to actively explain the exclusion of these types
>>> here.
>>>
>>> Cc Xuan.
>>>
>> Hi Jason, what do you think about this?
>>
>> Thanks.
>>
>>>>> \item The device uses a key as defined in \field{hash_key_data}
>>>>> and \field{hash_key_length} of the virtio_net_hash_config structure
>>>>> (see
>>>>> \ref{sec:Device Types / Network Device / Device Operation /
>>>>> Control Virtqueue / Automatic receive steering in multiqueue mode /
>>>>> Hash calculation}).
>>>>> \end{itemize}
>>>>> @@ -3883,6 +3892,18 @@ \subsubsection{Processing of Incoming
>>>>> Packets}\label{sec:Device Types / Network
>>>>> #define VIRTIO_NET_HASH_TYPE_TCP_EX (1 << 7)
>>>>> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
>>>>> \end{lstlisting}
>>>>> +Hash types applicable to inner payloads of GRE-encapsulated packets
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_GRE_INNER (1 << 9)
>>>>> +\end{lstlisting}
>>>>> +Hash types applicable to inner payloads of VXLAN-encapsulated packets
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_VXLAN_INNER (1 << 10)
>>>>> +\end{lstlisting}
>>>>> +Hash types applicable to inner payloads of GENEVE-encapsulated packets
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_TYPE_GENEVE_INNER (1 << 11)
>>>>> +\end{lstlisting}
>>>>>
>>>>> \subparagraph{IPv4 packets}
>>>>> \label{sec:Device Types / Network Device / Device Operation /
>>>>> Processing of Incoming Packets / Hash calculation for incoming
>>>>> packets / IPv4 packets}
>>>>> @@ -3975,15 +3996,148 @@ \subsubsection{Processing of Incoming
>>>>> Packets}\label{sec:Device Types / Network
>>>>> (see \ref{sec:Device Types / Network Device / Device Operation /
>>>>> Processing of Incoming Packets / Hash calculation for incoming
>>>>> packets / IPv6 packets without extension header}).
>>>>> \end{itemize}
>>>>>
>>>>> +\subparagraph{Inner payloads of GRE, VXLAN or GENEVE-encapsulated
>>>>> packets}
>>>>> +\label{sec:Device Types / Network Device / Device Operation /
>>>>> Processing of Incoming Packets / Hash calculation for incoming
>>>>> packets / Inner payloads of GRE, VXLAN or GENEVE-encapsulated packets}}
>>>>> +VIRTIO_NET_HASH_TYPE_GRE_INNER, VIRTIO_NET_HASH_TYPE_VXLAN_INNER or
>>>>> +VIRTIO_NET_HASH_TYPE_GENEVE_INNER bit is set at the same time as
>>>>> one of
>>>>> +the bits between VIRTIO_NET_HASH_TYPE_IPv4 and
>>>>> VIRTIO_NET_HASH_TYPE_UDP_EX.
>>>>> +
>>>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
>>>>> packets whose inner
>>>>> +payloads are IPv4 packets according to 'Enabled hash types'
>>>>> bitmasks as follows:
>>>>> +\begin{itemize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv4 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner TCPv4 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemsize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv4 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner UDPv4 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv4 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>>>> over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IP address
>>>>> + \item inner Destination IP address
>>>>> + \end{itemsize}
>>>>> + \item Else the device does not calculate the hash
>>>>> +\end{itemize}
>>>>> +
>>>>> +The device calculates the hash on GRE, VXLAN or GENEVE-encapsulated
>>>>> packets whose inner
>>>>> +payloads are IPv6 packets without extension headers according to
>>>>> 'Enabled hash types'
>>>>> +bitmasks as follows:
>>>>> +\begin{itemize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCPv6 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner TCPv6 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemsize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDPv6 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner UDPv6 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IPv6 is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>>>> over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item inner Source IPv6 address
>>>>> + \item inner Destination IPv6 address
>>>>> + \end{itemsize}
>>>>> + \item Else the device does not calculate the hash
>>>>> +\end{itemize}
>>>>> +
>>>>> +The device calculates the hash on GRE, VXLAN or GENEV-encapsulated
>>>>> packets whose inner
>>>>> +payloads are IPv6 packets with extension headers according to
>>>>> 'Enabled hash types'
>>>>> +bitmasks as follows:
>>>>> +\begin{itemsize}
>>>>> + \item If VIRTIO_NET_HASH_TYPE_TCP_EX is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner TCPv6 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemize}
>>>>> + \item Home address from the home address option in the
>>>>> inner IPv6 destination
>>>>> + options header. If the inner extension header is not
>>>>> present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the
>>>>> Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner
>>>>> extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \item inner Source TCP port
>>>>> + \item inner Destination TCP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_UDP_EX is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, and the GRE, VXLAN or
>>>>> GENEVE-encapsulated
>>>>> + packet has an inner UDPv6 header in its payload, the hash is
>>>>> calculated over
>>>>> + the following fields:
>>>>> + \begin{itemsize}
>>>>> + \item Home address from the home address option in the
>>>>> inner IPv6 destination
>>>>> + options header. If the inner extension header is not
>>>>> present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the
>>>>> Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner
>>>>> extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \item inner Source UDP port
>>>>> + \item inner Destination UDP port
>>>>> + \end{itemize}
>>>>> + \item Else if VIRTIO_NET_HASH_TYPE_IP_EX is set along with one of
>>>>> + VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> + VIRTIO_NET_HASH_TYPE_GENEVE_INNER, the hash is calculated
>>>>> over the
>>>>> + following fields:
>>>>> + \begin{itemsize}
>>>>> + \item Home address from the home address option in the
>>>>> inner IPv6 destination
>>>>> + options header. If the inner extension header is not
>>>>> present, use the
>>>>> + inner Source IPv6 address.
>>>>> + \item IPv6 address that is contained in the
>>>>> Routing-Header-Type-2 from the
>>>>> + associated inner extension header. If the inner
>>>>> extension header is not
>>>>> + present, use the inner Destination IPv6 address.
>>>>> + \end{itemize}
>>>>> + \item Else skip inner IPv6 extension headers and calculate the
>>>>> hash as defined
>>>>> + for a GRE, VXLAN or GENEVE-encapsulated packet whose inner
>>>>> payload is an IPv6
>>>>> + packet without extension headers
>>>>> +\end{itemsize}
>>>>> +
>>>>> \paragraph{Hash reporting for incoming packets}
>>>>> \label{sec:Device Types / Network Device / Device Operation /
>>>>> Processing of Incoming Packets / Hash reporting for incoming packets}
>>>>>
>>>>> -If VIRTIO_NET_F_HASH_REPORT was negotiated and
>>>>> - the device has calculated the hash for the packet, the device
>>>>> fills \field{hash_report} with the report type of calculated hash
>>>>> -and \field{hash_value} with the value of calculated hash.
>>>>> +If VIRTIO_NET_F_HASH_REPORT was negotiated and the device has
>>>>> calculated the
>>>>> +hash for the packet, the device fills \field{hash_report} with the
>>>>> report type
>>>>> +of calculated hash, and \field{hash_value} with the value of
>>>>> calculated hash.
>>>>> +And if VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER was negotiated, the
>>>>> device also
>>>>> +needs to fills \field{hash_report_tunnel} with the type of the
>>>>> tunnel packet,
>>>>>
>>>>> If VIRTIO_NET_F_HASH_REPORT was negotiated but due to any reason the
>>>>> -hash was not calculated, the device sets \field{hash_report} to
>>>>> VIRTIO_NET_HASH_REPORT_NONE.
>>>>> +hash was not calculated, the device sets \field{hash_report} to
>>>>> VIRTIO_NET_HASH_REPORT_NONE,
>>>>> +and \field{hash_report_tunnel} is no longer valid.
>>>>>
>>>>> Possible values that the device can report in \field{hash_report}
>>>>> are defined below.
>>>>> They correspond to supported hash types defined in
>>>>> @@ -4005,6 +4159,24 @@ \subsubsection{Processing of Incoming
>>>>> Packets}\label{sec:Device Types / Network
>>>>> #define VIRTIO_NET_HASH_REPORT_UDPv6_EX 9
>>>>> \end{lstlisting}
>>>>>
>>>>> +If \field{hash_report} differs from VIRTIO_NET_HASH_REPORT_NONE,
>>>>> +\field{hash_report_tunnel} can report the type of the
>>>>> tunnel-encapsulated
>>>>> +packet to the driver over the inner header hash calculation.
>>>>> +Possible values that the device can report in
>>>>> \field{hash_report_tunnel}
>>>>> +are defined below:
>>>>> +
>>>>> +\begin{lstlisting}
>>>>> +#define VIRTIO_NET_HASH_REPORT_GRE 1
>>>>> +#define VIRTIO_NET_HASH_REPORT_VXLAN 2
>>>>> +#define VIRTIO_NET_HASH_REPORT_GENEVE 3
>>>>> +\end{lstlisting}
>>>>> +
>>>>> +The values VIRTIO_NET_HASH_REPORT_GRE, VIRTIO_NET_HASH_REPORT_VXLAN
>>>>> and
>>>>> +VIRTIO_NET_HASH_REPORT_GENEVE correspond to
>>>>> VIRTIO_NET_HASH_TYPE_GRE_INNER,
>>>>> +VIRTIO_NET_HASH_TYPE_VXLAN_INNER and
>>>>> VIRTIO_NET_HASH_TYPE_GENEVE_INNER bits
>>>>> +of supported hash types defined in respectively
>>>>> +\ref{sec:Device Types / Network Device / Device Operation /
>>>>> Processing of Incoming Packets / Hash calculation for incoming
>>>>> packets / Supported/enabled hash types}.
>>>>> +
>>>> I think we can simply say "They correspond to supported hash types
>>>> defined in \ref{xyz}" as what is done for non-tunnel hash types?
>>> Ok, I agree with you.
>>>
>>> Thanks.
>>>
>>>> Thanks
>>>>
>>>>> \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
>>>>> Device / Device Operation / Control Virtqueue}
>>>>>
>>>>> The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
>>>>> --
>>>>> 2.19.1.6.gb485710b
>>>>>
>>>
>>> This publicly archived list offers a means to provide input to the
>>> OASIS Virtual I/O Device (VIRTIO) TC.
>>>
>>> In order to verify user consent to the Feedback License terms and
>>> to minimize spam in the list archive, subscription is required
>>> before posting.
>>>
>>> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
>>> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
>>> List help: virtio-comment-help@lists.oasis-open.org
>>> List archive: https://lists.oasis-open.org/archives/virtio-comment/
>>> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
>>> List Guidelines:
>>> https://www.oasis-open.org/policies-guidelines/mailing-lists
>>> Committee: https://www.oasis-open.org/committees/virtio/
>>> Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-12-21 6:16 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-05 6:36 [PATCH v3] virtio_net: support inner header hash Heng Qi
2022-12-09 6:09 ` Heng Qi
2022-12-14 9:01 ` [virtio-dev] " Heng Qi
2022-12-14 9:12 ` Jason Wang
2022-12-14 9:16 ` Heng Qi
2022-12-14 9:30 ` [virtio-comment] " Cornelia Huck
2022-12-14 9:39 ` Heng Qi
2022-12-15 7:46 ` Michael S. Tsirkin
2022-12-16 6:33 ` Jason Wang
2022-12-16 10:14 ` Heng Qi
2022-12-20 14:27 ` [virtio-comment] " Heng Qi
2022-12-21 3:15 ` Jason Wang
2022-12-21 6:16 ` Heng Qi
2022-12-16 12:49 ` Michael S. Tsirkin
2022-12-19 2:52 ` [virtio-comment] " Heng Qi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.