* [virtio-comment] [PATCH v10] virtio-net: support inner header hash
@ 2023-03-06 15:48 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-06 15:48 UTC (permalink / raw)
To: virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Parav Pandit, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
Currently, a received encapsulated packet has an outer and an inner header, but
the virtio device is unable to calculate the hash for the inner header. Multiple
flows with the same outer header but different inner headers are steered to the
same receive queue. This results in poor receive performance.
To address this limitation, a new feature VIRTIO_NET_F_HASH_TUNNEL has been
introduced, which enables the device to advertise the capability to calculate the
hash for the inner packet header. Compared with the out header hash, it regains
better receive performance.
Reviewed-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
v9->v10:
1. Removed hash_report_tunnel related information. @Parav Pandit
2. Re-describe the limitations of QoS for tunneling.
3. Some clarification.
v8->v9:
1. Merge hash_report_tunnel_types into hash_report. @Parav Pandit
2. Add tunnel security section. @Michael S . Tsirkin
3. Add VIRTIO_NET_F_HASH_REPORT_TUNNEL.
4. Fix some typos.
5. Add more tunnel types. @Michael S . Tsirkin
v7->v8:
1. Add supported_hash_tunnel_types. @Jason Wang, @Parav Pandit
2. Change hash_report_tunnel to hash_report_tunnel_types. @Parav Pandit
3. Removed re-definition for inner packet hashing. @Parav Pandit
4. Fix some typos. @Michael S . Tsirkin
5. Clarify some sentences. @Michael S . Tsirkin
v6->v7:
1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
2. Fix some syntax issues. @Michael S. Tsirkin
v5->v6:
1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
3. Move the links to introduction section. @Michael S. Tsirkin
4. Clarify some sentences. @Michael S. Tsirkin
v4->v5:
1. Clarify some paragraphs. @Cornelia Huck
2. Fix the u8 type. @Cornelia Huck
v3->v4:
1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
2. Make things clearer. @Jason Wang @Michael S. Tsirkin
3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
v2->v3:
1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
v1->v2:
1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
2. Clarify some paragraphs. @Jason Wang
3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
device-types/net/description.tex | 120 +++++++++++++++++++++++++++++--
introduction.tex | 24 +++++++
2 files changed, 140 insertions(+), 4 deletions(-)
diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index 0500bb6..e53d625 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -83,6 +83,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
\item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
channel.
+\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
+ for tunnel-encapsulated packets.
+
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
\item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
@@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
\item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
\item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
\end{description}
\subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
u8 rss_max_key_size;
le16 rss_max_indirection_table_length;
le32 supported_hash_types;
+ le32 supported_tunnel_hash_types;
};
\end{lstlisting}
-The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
+The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
It specifies the maximum supported length of RSS key in bytes.
The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
It specifies the maximum number of 16-bit entries in RSS indirection table.
The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
-i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
+i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
Field \field{supported_hash_types} contains the bitmask of supported hash types.
See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
+The next field, \field{supported_tunnel_hash_types} only exists if the device
+supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
+
+Field \field{supported_tunnel_hash_types} contains the bitmask of supported tunnel hash types.
+See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types} for details of supported tunnel hash types.
+
\devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
@@ -235,7 +246,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
negotiated.
The device MUST set \field{rss_max_key_size} to at least 40, if it offers
-VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
+VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
VIRTIO_NET_F_RSS.
@@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
\begin{itemize}
\item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
\item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
+\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
\end{itemize}
If the feature VIRTIO_NET_F_RSS was negotiated:
\begin{itemize}
\item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
+\item The device uses \field{hash_tunnel_types} of the virtio_net_rss_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
\item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
\end{itemize}
@@ -855,6 +868,7 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
If the feature VIRTIO_NET_F_RSS was not negotiated:
\begin{itemize}
\item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
+\item The device uses \field{hash_tunnel_types} of the virtio_net_hash_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
\item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
\end{itemize}
@@ -868,8 +882,26 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
\end{itemize}
+\subparagraph{Tunnel/Encapsulated packet}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
+A tunnel packet is encapsulated from the original packet based on the tunneling
+protocol (only a single level of encapsulation is currently supported). The
+encapsulated packet contains an outer header and an inner header, and the device
+calculates the hash over either the inner header or the outer header.
+
+When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and a received encapsulated
+packet's outer header matches one of the supported \field{hash_tunnel_types},
+the hash of the inner header is calculated. Supported encapsulation types are listed
+in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming
+Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
+
+Some encapsulated packet types: \hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]},
+\hyperref[intro:GENEVE]{[GENEVE]}, \hyperref[intro:IPIP]{[IPIP]} and \hyperref[intro:NVGRE]{[NVGRE]}.
+
\subparagraph{Supported/enabled hash types}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
+This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
+\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
Hash types applicable for IPv4 packets:
\begin{lstlisting}
#define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
@@ -889,6 +921,39 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
#define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
\end{lstlisting}
+\subparagraph{Supported/enabled tunnel hash types}
+\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types}
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the value of
+\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
+the device calculates the hash using the outer header of the encapsulated packet.
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_NONE (1 << 0)
+\end{lstlisting}
+
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
+hash type below indicates that the hash is calculated over the inner header of
+the encapsulated packet:
+Hash type applicable for inner payload of the gre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
+\end{lstlisting}
+Hash type applicable for inner payload of the vxlan-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
+\end{lstlisting}
+Hash type applicable for inner payload of the geneve-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
+\end{lstlisting}
+Hash type applicable for inner payload of the ip-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
+\end{lstlisting}
+Hash type applicable for inner payload of the nvgre-encapsulated packet
+\begin{lstlisting}
+#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
+\end{lstlisting}
+
\subparagraph{IPv4 packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
@@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
(see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
\end{itemize}
+\subparagraph{Inner hash calculation of an encapsulated packet}
+If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
+hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
+sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
+feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
+configure the hash parameters. If multiple commands are sent, the device configuration
+will be defined by the last command received.
+
+If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
+encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
+hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
+/ Network Device / Device Operation / Processing of Incoming Packets /
+Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
+type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
+of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
+the hash on the outer header.
+
+\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
+unencapsulated packets.
+
+\subparagraph{Tunnel QoS limitation}
+Note that the limitation mentioned below is not only introduced by inner hash calculation,
+and the limitation of the tunnel itself, and even the driver may have only one receive queue.
+
+When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
+there is no quality of service (QoS) for these packets of multiple tunnels. For example, when the
+flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
+queue to be unbalanced, resulting in potential packet loss and data delay.
+
+Possible mitigations:
+\begin{itemize}
+\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
+\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
+ VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
+\item Choose a hash key that can avoid queue collisions.
+\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
+\end{itemize}
+
\paragraph{Hash reporting for incoming packets}
\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
@@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
le16 reserved[4];
u8 hash_key_length;
u8 hash_key_data[hash_key_length];
+ le32 hash_tunnel_types;
};
\end{lstlisting}
Field \field{hash_types} contains a bitmask of allowed hash types as
defined in
\ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
-Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
+
+Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
+defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
+
+Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
@@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
le16 max_tx_vq;
u8 hash_key_length;
u8 hash_key_data[hash_key_length];
+ le32 hash_tunnel_types;
};
\end{lstlisting}
Field \field{hash_types} contains a bitmask of allowed hash types as
@@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
+Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
+defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
+
\drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
diff --git a/introduction.tex b/introduction.tex
index 287c5fc..25c9d48 100644
--- a/introduction.tex
+++ b/introduction.tex
@@ -99,6 +99,30 @@ \section{Normative References}\label{sec:Normative References}
Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
\newline\url{https://www.secg.org/sec1-v2.pdf}\\
+ \phantomsection\label{intro:GRE}\textbf{[GRE]} &
+ Generic Routing Encapsulation
+ \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
+ \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
+ Virtual eXtensible Local Area Network
+ \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
+ \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
+ Generic Network Virtualization Encapsulation
+ \phantomsection\label{intro:IPIP}\textbf{[IPIP]} &
+ IP Encapsulation within IP
+ \newline\url{https://www.rfc-editor.org/rfc/rfc2003}\\
+ \phantomsection\label{intro:IPIP}\textbf{[NVGRE]} &
+ NVGRE: Network Virtualization Using Generic Routing Encapsulation
+ \newline\url{https://www.rfc-editor.org/rfc/rfc7637.html}\\
+ \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
+ \phantomsection\label{intro:IP}\textbf{[IP]} &
+ INTERNET PROTOCOL
+ \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
+ \phantomsection\label{intro:UDP}\textbf{[UDP]} &
+ User Datagram Protocol
+ \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
+ \phantomsection\label{intro:TCP}\textbf{[TCP]} &
+ TRANSMISSION CONTROL PROTOCOL
+ \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
\end{longtable}
\section{Non-Normative References}
--
2.19.1.6.gb485710b
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 36+ messages in thread* Re: [virtio-dev] [PATCH v10] virtio-net: support inner header hash
2023-03-06 15:48 ` [virtio-comment] " Heng Qi
@ 2023-03-08 14:27 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-08 14:27 UTC (permalink / raw)
To: virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Parav Pandit, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
Do you have any comments, please?
Thanks. :)
在 2023/3/6 下午11:48, Heng Qi 写道:
> Currently, a received encapsulated packet has an outer and an inner header, but
> the virtio device is unable to calculate the hash for the inner header. Multiple
> flows with the same outer header but different inner headers are steered to the
> same receive queue. This results in poor receive performance.
>
> To address this limitation, a new feature VIRTIO_NET_F_HASH_TUNNEL has been
> introduced, which enables the device to advertise the capability to calculate the
> hash for the inner packet header. Compared with the out header hash, it regains
> better receive performance.
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v9->v10:
> 1. Removed hash_report_tunnel related information. @Parav Pandit
> 2. Re-describe the limitations of QoS for tunneling.
> 3. Some clarification.
>
> v8->v9:
> 1. Merge hash_report_tunnel_types into hash_report. @Parav Pandit
> 2. Add tunnel security section. @Michael S . Tsirkin
> 3. Add VIRTIO_NET_F_HASH_REPORT_TUNNEL.
> 4. Fix some typos.
> 5. Add more tunnel types. @Michael S . Tsirkin
>
> v7->v8:
> 1. Add supported_hash_tunnel_types. @Jason Wang, @Parav Pandit
> 2. Change hash_report_tunnel to hash_report_tunnel_types. @Parav Pandit
> 3. Removed re-definition for inner packet hashing. @Parav Pandit
> 4. Fix some typos. @Michael S . Tsirkin
> 5. Clarify some sentences. @Michael S . Tsirkin
>
> v6->v7:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5->v6:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4->v5:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3->v4:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2->v3:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1->v2:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> device-types/net/description.tex | 120 +++++++++++++++++++++++++++++--
> introduction.tex | 24 +++++++
> 2 files changed, 140 insertions(+), 4 deletions(-)
>
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 0500bb6..e53d625 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -83,6 +83,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
> +Field \field{supported_tunnel_hash_types} contains the bitmask of supported tunnel hash types.
> +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types} for details of supported tunnel hash types.
> +
> \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>
> The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> @@ -235,7 +246,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> negotiated.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_rss_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
> @@ -855,6 +868,7 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_hash_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -868,8 +882,26 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and a received encapsulated
> +packet's outer header matches one of the supported \field{hash_tunnel_types},
> +the hash of the inner header is calculated. Supported encapsulation types are listed
> +in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming
> +Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Some encapsulated packet types: \hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]},
> +\hyperref[intro:GENEVE]{[GENEVE]}, \hyperref[intro:IPIP]{[IPIP]} and \hyperref[intro:NVGRE]{[NVGRE]}.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -889,6 +921,39 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +\subparagraph{Supported/enabled tunnel hash types}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the value of
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
> +the device calculates the hash using the outer header of the encapsulated packet.
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NONE (1 << 0)
> +\end{lstlisting}
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels. For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
> A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..25c9d48 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -99,6 +99,30 @@ \section{Normative References}\label{sec:Normative References}
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \phantomsection\label{intro:IPIP}\textbf{[IPIP]} &
> + IP Encapsulation within IP
> + \newline\url{https://www.rfc-editor.org/rfc/rfc2003}\\
> + \phantomsection\label{intro:IPIP}\textbf{[NVGRE]} &
> + NVGRE: Network Virtualization Using Generic Routing Encapsulation
> + \newline\url{https://www.rfc-editor.org/rfc/rfc7637.html}\\
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> \end{longtable}
>
> \section{Non-Normative References}
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-comment] Re: [virtio-dev] [PATCH v10] virtio-net: support inner header hash
@ 2023-03-08 14:27 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-08 14:27 UTC (permalink / raw)
To: virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Parav Pandit, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
Do you have any comments, please?
Thanks. :)
在 2023/3/6 下午11:48, Heng Qi 写道:
> Currently, a received encapsulated packet has an outer and an inner header, but
> the virtio device is unable to calculate the hash for the inner header. Multiple
> flows with the same outer header but different inner headers are steered to the
> same receive queue. This results in poor receive performance.
>
> To address this limitation, a new feature VIRTIO_NET_F_HASH_TUNNEL has been
> introduced, which enables the device to advertise the capability to calculate the
> hash for the inner packet header. Compared with the out header hash, it regains
> better receive performance.
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v9->v10:
> 1. Removed hash_report_tunnel related information. @Parav Pandit
> 2. Re-describe the limitations of QoS for tunneling.
> 3. Some clarification.
>
> v8->v9:
> 1. Merge hash_report_tunnel_types into hash_report. @Parav Pandit
> 2. Add tunnel security section. @Michael S . Tsirkin
> 3. Add VIRTIO_NET_F_HASH_REPORT_TUNNEL.
> 4. Fix some typos.
> 5. Add more tunnel types. @Michael S . Tsirkin
>
> v7->v8:
> 1. Add supported_hash_tunnel_types. @Jason Wang, @Parav Pandit
> 2. Change hash_report_tunnel to hash_report_tunnel_types. @Parav Pandit
> 3. Removed re-definition for inner packet hashing. @Parav Pandit
> 4. Fix some typos. @Michael S . Tsirkin
> 5. Clarify some sentences. @Michael S . Tsirkin
>
> v6->v7:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5->v6:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4->v5:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3->v4:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2->v3:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1->v2:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> device-types/net/description.tex | 120 +++++++++++++++++++++++++++++--
> introduction.tex | 24 +++++++
> 2 files changed, 140 insertions(+), 4 deletions(-)
>
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 0500bb6..e53d625 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -83,6 +83,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
> +Field \field{supported_tunnel_hash_types} contains the bitmask of supported tunnel hash types.
> +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types} for details of supported tunnel hash types.
> +
> \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>
> The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> @@ -235,7 +246,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> negotiated.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_rss_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
> @@ -855,6 +868,7 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_hash_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -868,8 +882,26 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and a received encapsulated
> +packet's outer header matches one of the supported \field{hash_tunnel_types},
> +the hash of the inner header is calculated. Supported encapsulation types are listed
> +in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming
> +Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Some encapsulated packet types: \hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]},
> +\hyperref[intro:GENEVE]{[GENEVE]}, \hyperref[intro:IPIP]{[IPIP]} and \hyperref[intro:NVGRE]{[NVGRE]}.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -889,6 +921,39 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +\subparagraph{Supported/enabled tunnel hash types}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the value of
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
> +the device calculates the hash using the outer header of the encapsulated packet.
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NONE (1 << 0)
> +\end{lstlisting}
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels. For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
> A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..25c9d48 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -99,6 +99,30 @@ \section{Normative References}\label{sec:Normative References}
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \phantomsection\label{intro:IPIP}\textbf{[IPIP]} &
> + IP Encapsulation within IP
> + \newline\url{https://www.rfc-editor.org/rfc/rfc2003}\\
> + \phantomsection\label{intro:IPIP}\textbf{[NVGRE]} &
> + NVGRE: Network Virtualization Using Generic Routing Encapsulation
> + \newline\url{https://www.rfc-editor.org/rfc/rfc7637.html}\\
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> \end{longtable}
>
> \section{Non-Normative References}
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [virtio-dev] [PATCH v10] virtio-net: support inner header hash
2023-03-08 14:27 ` [virtio-comment] " Heng Qi
@ 2023-03-08 14:31 ` Michael S. Tsirkin
-1 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-08 14:31 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-dev, virtio-comment, Jason Wang, Parav Pandit,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 08, 2023 at 10:27:13PM +0800, Heng Qi wrote:
> Do you have any comments, please?
>
> Thanks. :)
>
> 在 2023/3/6 下午11:48, Heng Qi 写道:
It's been just 2 partial days. Sit tight pls.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-comment] Re: [virtio-dev] [PATCH v10] virtio-net: support inner header hash
@ 2023-03-08 14:31 ` Michael S. Tsirkin
0 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-08 14:31 UTC (permalink / raw)
To: Heng Qi
Cc: virtio-dev, virtio-comment, Jason Wang, Parav Pandit,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 08, 2023 at 10:27:13PM +0800, Heng Qi wrote:
> Do you have any comments, please?
>
> Thanks. :)
>
> 在 2023/3/6 下午11:48, Heng Qi 写道:
It's been just 2 partial days. Sit tight pls.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-06 15:48 ` [virtio-comment] " Heng Qi
@ 2023-03-14 13:59 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-14 13:59 UTC (permalink / raw)
To: virtio-dev, virtio-comment, Parav Pandit
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
Hi, Parav, do you have any comments on this? :)
Thanks!
在 2023/3/6 下午11:48, Heng Qi 写道:
> Currently, a received encapsulated packet has an outer and an inner header, but
> the virtio device is unable to calculate the hash for the inner header. Multiple
> flows with the same outer header but different inner headers are steered to the
> same receive queue. This results in poor receive performance.
>
> To address this limitation, a new feature VIRTIO_NET_F_HASH_TUNNEL has been
> introduced, which enables the device to advertise the capability to calculate the
> hash for the inner packet header. Compared with the out header hash, it regains
> better receive performance.
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v9->v10:
> 1. Removed hash_report_tunnel related information. @Parav Pandit
> 2. Re-describe the limitations of QoS for tunneling.
> 3. Some clarification.
>
> v8->v9:
> 1. Merge hash_report_tunnel_types into hash_report. @Parav Pandit
> 2. Add tunnel security section. @Michael S . Tsirkin
> 3. Add VIRTIO_NET_F_HASH_REPORT_TUNNEL.
> 4. Fix some typos.
> 5. Add more tunnel types. @Michael S . Tsirkin
>
> v7->v8:
> 1. Add supported_hash_tunnel_types. @Jason Wang, @Parav Pandit
> 2. Change hash_report_tunnel to hash_report_tunnel_types. @Parav Pandit
> 3. Removed re-definition for inner packet hashing. @Parav Pandit
> 4. Fix some typos. @Michael S . Tsirkin
> 5. Clarify some sentences. @Michael S . Tsirkin
>
> v6->v7:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5->v6:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4->v5:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3->v4:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2->v3:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1->v2:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> device-types/net/description.tex | 120 +++++++++++++++++++++++++++++--
> introduction.tex | 24 +++++++
> 2 files changed, 140 insertions(+), 4 deletions(-)
>
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 0500bb6..e53d625 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -83,6 +83,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
> +Field \field{supported_tunnel_hash_types} contains the bitmask of supported tunnel hash types.
> +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types} for details of supported tunnel hash types.
> +
> \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>
> The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> @@ -235,7 +246,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> negotiated.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_rss_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
> @@ -855,6 +868,7 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_hash_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -868,8 +882,26 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and a received encapsulated
> +packet's outer header matches one of the supported \field{hash_tunnel_types},
> +the hash of the inner header is calculated. Supported encapsulation types are listed
> +in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming
> +Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Some encapsulated packet types: \hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]},
> +\hyperref[intro:GENEVE]{[GENEVE]}, \hyperref[intro:IPIP]{[IPIP]} and \hyperref[intro:NVGRE]{[NVGRE]}.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -889,6 +921,39 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +\subparagraph{Supported/enabled tunnel hash types}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the value of
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
> +the device calculates the hash using the outer header of the encapsulated packet.
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NONE (1 << 0)
> +\end{lstlisting}
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels. For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
> A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..25c9d48 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -99,6 +99,30 @@ \section{Normative References}\label{sec:Normative References}
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \phantomsection\label{intro:IPIP}\textbf{[IPIP]} &
> + IP Encapsulation within IP
> + \newline\url{https://www.rfc-editor.org/rfc/rfc2003}\\
> + \phantomsection\label{intro:IPIP}\textbf{[NVGRE]} &
> + NVGRE: Network Virtualization Using Generic Routing Encapsulation
> + \newline\url{https://www.rfc-editor.org/rfc/rfc7637.html}\\
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> \end{longtable}
>
> \section{Non-Normative References}
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-14 13:59 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-14 13:59 UTC (permalink / raw)
To: virtio-dev, virtio-comment, Parav Pandit
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
Hi, Parav, do you have any comments on this? :)
Thanks!
在 2023/3/6 下午11:48, Heng Qi 写道:
> Currently, a received encapsulated packet has an outer and an inner header, but
> the virtio device is unable to calculate the hash for the inner header. Multiple
> flows with the same outer header but different inner headers are steered to the
> same receive queue. This results in poor receive performance.
>
> To address this limitation, a new feature VIRTIO_NET_F_HASH_TUNNEL has been
> introduced, which enables the device to advertise the capability to calculate the
> hash for the inner packet header. Compared with the out header hash, it regains
> better receive performance.
>
> Reviewed-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> v9->v10:
> 1. Removed hash_report_tunnel related information. @Parav Pandit
> 2. Re-describe the limitations of QoS for tunneling.
> 3. Some clarification.
>
> v8->v9:
> 1. Merge hash_report_tunnel_types into hash_report. @Parav Pandit
> 2. Add tunnel security section. @Michael S . Tsirkin
> 3. Add VIRTIO_NET_F_HASH_REPORT_TUNNEL.
> 4. Fix some typos.
> 5. Add more tunnel types. @Michael S . Tsirkin
>
> v7->v8:
> 1. Add supported_hash_tunnel_types. @Jason Wang, @Parav Pandit
> 2. Change hash_report_tunnel to hash_report_tunnel_types. @Parav Pandit
> 3. Removed re-definition for inner packet hashing. @Parav Pandit
> 4. Fix some typos. @Michael S . Tsirkin
> 5. Clarify some sentences. @Michael S . Tsirkin
>
> v6->v7:
> 1. Modify the wording of some sentences for clarity. @Michael S. Tsirkin
> 2. Fix some syntax issues. @Michael S. Tsirkin
>
> v5->v6:
> 1. Fix some syntax and capitalization issues. @Michael S. Tsirkin
> 2. Use encapsulated/encaptulation uniformly. @Michael S. Tsirkin
> 3. Move the links to introduction section. @Michael S. Tsirkin
> 4. Clarify some sentences. @Michael S. Tsirkin
>
> v4->v5:
> 1. Clarify some paragraphs. @Cornelia Huck
> 2. Fix the u8 type. @Cornelia Huck
>
> v3->v4:
> 1. Rename VIRTIO_NET_F_HASH_GRE_VXLAN_GENEVE_INNER to VIRTIO_NET_F_HASH_TUNNEL. @Jason Wang
> 2. Make things clearer. @Jason Wang @Michael S. Tsirkin
> 3. Keep the possibility to use inner hash for automatic receive steering. @Jason Wang
> 4. Add the "Tunnel packet" paragraph to avoid repeating the GRE etc. many times. @Michael S. Tsirkin
>
> v2->v3:
> 1. Add a feature bit for GRE/VXLAN/GENEVE inner hash. @Jason Wang
> 2. Chang \field{hash_tunnel} to \field{hash_report_tunnel}. @Jason Wang, @Michael S. Tsirkin
>
> v1->v2:
> 1. Remove the patch for the bitmask fix. @Michael S. Tsirkin
> 2. Clarify some paragraphs. @Jason Wang
> 3. Add \field{hash_tunnel} and VIRTIO_NET_HASH_REPORT_GRE. @Yuri Benditovich
>
> device-types/net/description.tex | 120 +++++++++++++++++++++++++++++--
> introduction.tex | 24 +++++++
> 2 files changed, 140 insertions(+), 4 deletions(-)
>
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 0500bb6..e53d625 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -83,6 +83,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> channel.
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
> +Field \field{supported_tunnel_hash_types} contains the bitmask of supported tunnel hash types.
> +See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types} for details of supported tunnel hash types.
> +
> \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>
> The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> @@ -235,7 +246,7 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> negotiated.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
> If the feature VIRTIO_NET_F_RSS was negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_rss_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_rss_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_rss_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}).
> \end{itemize}
> @@ -855,6 +868,7 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> If the feature VIRTIO_NET_F_RSS was not negotiated:
> \begin{itemize}
> \item The device uses \field{hash_types} of the virtio_net_hash_config structure as 'Enabled hash types' bitmask.
> +\item The device uses \field{hash_tunnel_types} of the virtio_net_hash_config structure as 'Enabled hash tunnel types' bitmask if VIRTIO_NET_F_HASH_TUNNEL was negotiated.
> \item The device uses a key as defined in \field{hash_key_data} and \field{hash_key_length} of the virtio_net_hash_config structure (see
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}).
> \end{itemize}
> @@ -868,8 +882,26 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode / Hash calculation}.
> \end{itemize}
>
> +\subparagraph{Tunnel/Encapsulated packet}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Tunnel/Encapsulated packet}
> +A tunnel packet is encapsulated from the original packet based on the tunneling
> +protocol (only a single level of encapsulation is currently supported). The
> +encapsulated packet contains an outer header and an inner header, and the device
> +calculates the hash over either the inner header or the outer header.
> +
> +When the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and a received encapsulated
> +packet's outer header matches one of the supported \field{hash_tunnel_types},
> +the hash of the inner header is calculated. Supported encapsulation types are listed
> +in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming
> +Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Some encapsulated packet types: \hyperref[intro:GRE]{[GRE]}, \hyperref[intro:VXLAN]{[VXLAN]},
> +\hyperref[intro:GENEVE]{[GENEVE]}, \hyperref[intro:IPIP]{[IPIP]} and \hyperref[intro:NVGRE]{[NVGRE]}.
> +
> \subparagraph{Supported/enabled hash types}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}
> +This paragraph relies on definitions from \hyperref[intro:IP]{[IP]},
> +\hyperref[intro:UDP]{[UDP]} and \hyperref[intro:TCP]{[TCP]}.
> Hash types applicable for IPv4 packets:
> \begin{lstlisting}
> #define VIRTIO_NET_HASH_TYPE_IPv4 (1 << 0)
> @@ -889,6 +921,39 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> #define VIRTIO_NET_HASH_TYPE_UDP_EX (1 << 8)
> \end{lstlisting}
>
> +\subparagraph{Supported/enabled tunnel hash types}
> +\label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled tunnel hash types}
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the value of
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
> +the device calculates the hash using the outer header of the encapsulated packet.
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NONE (1 << 0)
> +\end{lstlisting}
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels. For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
> A driver MUST NOT send the VIRTIO_NET_CTRL_MQ_RSS_CONFIG command if the feature VIRTIO_NET_F_RSS has not been negotiated.
> diff --git a/introduction.tex b/introduction.tex
> index 287c5fc..25c9d48 100644
> --- a/introduction.tex
> +++ b/introduction.tex
> @@ -99,6 +99,30 @@ \section{Normative References}\label{sec:Normative References}
> Standards for Efficient Cryptography Group(SECG), ``SEC1: Elliptic Cureve Cryptography'', Version 1.0, September 2000.
> \newline\url{https://www.secg.org/sec1-v2.pdf}\\
>
> + \phantomsection\label{intro:GRE}\textbf{[GRE]} &
> + Generic Routing Encapsulation
> + \newline\url{https://datatracker.ietf.org/doc/rfc2784/}\\
> + \phantomsection\label{intro:VXLAN}\textbf{[VXLAN]} &
> + Virtual eXtensible Local Area Network
> + \newline\url{https://datatracker.ietf.org/doc/rfc7348/}\\
> + \phantomsection\label{intro:GENEVE}\textbf{[GENEVE]} &
> + Generic Network Virtualization Encapsulation
> + \phantomsection\label{intro:IPIP}\textbf{[IPIP]} &
> + IP Encapsulation within IP
> + \newline\url{https://www.rfc-editor.org/rfc/rfc2003}\\
> + \phantomsection\label{intro:IPIP}\textbf{[NVGRE]} &
> + NVGRE: Network Virtualization Using Generic Routing Encapsulation
> + \newline\url{https://www.rfc-editor.org/rfc/rfc7637.html}\\
> + \newline\url{https://datatracker.ietf.org/doc/rfc8926/}\\
> + \phantomsection\label{intro:IP}\textbf{[IP]} &
> + INTERNET PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc791}\\
> + \phantomsection\label{intro:UDP}\textbf{[UDP]} &
> + User Datagram Protocol
> + \newline\url{https://www.rfc-editor.org/rfc/rfc768}\\
> + \phantomsection\label{intro:TCP}\textbf{[TCP]} &
> + TRANSMISSION CONTROL PROTOCOL
> + \newline\url{https://www.rfc-editor.org/rfc/rfc793}\\
> \end{longtable}
>
> \section{Non-Normative References}
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-06 15:48 ` [virtio-comment] " Heng Qi
@ 2023-03-15 3:23 ` Parav Pandit
-1 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 3:23 UTC (permalink / raw)
To: Heng Qi, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
On 3/6/2023 10:48 AM, Heng Qi wrote:
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
May be to say inner packet header hash..
This make its little more clear about "which header" that you explained
in the commit log.
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
I think this should also say that HASH_TUNNEL requires either of the
F_RSS or F_HASH_REPORT.
Because without it HASH_TUNNEL is not useful.
Right?
if no, than my below comments are meaningless.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
I think rss_max_key_size field dependency should be only of the existing
feature bits F_RSS and F_HASH_REPORT.
This is because those are the bits really deciding to consider
rss_max_key_size.
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
Same as above.
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
Above line "the next field .." can be just same as "Device supports
inner packet header hash calculation, i.e..."
This is because here, the term "header" is missed, which is present in
the definition of feature bit 52.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
This needs change if the above first comment about rss_max_key_size is
right.
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
inner packet header hash ..
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
Any encapsulation technology that includes UDP/L4 header likely do not
prefer based on the inner header. This is because the outer header src
port entropy is added based on the inner header.
I was not able to follow the discussion in v9 that you had with Michael.
Did you conclude if this is needed for vxlan too?
If not, for now it may be better to skip vxlan and nvegre as they
inherently have unique outer header UDP src port based on the inner header.
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
When there is a single receive queue, there is no tunnel QoS issue.
Because the same queue is used one or more tunnels.
So please remove mentioning single queue text here.
Also it is better to first describe the limitation and after the reader
understand, it make sense to say that the limitation already exists
with/without inner header hash calculation.
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels.
"of multiple tunnels at the tail of sentence is not needed because you
already have it at "shared by multiple tunnels".
> For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
Your example is only talking about single queue..
You probably wanted to say,
When the packets of certain tunnels results in spreading these packets
to multiple receive queues, these receive queues may have unbalanced
amount of packets. This can result in a specific receive queue being
full resulting in packet loss.
Or something similar..
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
I dont think it is necessary to talk about specific tools like DPDK in
specification.
Even if you omit "such as DPDK", the line reads fine.
so I suggest to drop "such as DPDK".
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
It can still enter the switch but you want to avoid entering the virtio
device.
Cluster is very broad term and not defined in the spec, better to avoid it.
you can rewrite it something like, Perform appropriate QoS before
packets consume the receive buffers..
This is generic solution that can be done in device or egress of switch
etc, which keep the spec scope upto the device.
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
You likely need to add device and driver normative statements derived
from above text and add their references into
device-types/net/*-conformance.tex files.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 3:23 ` Parav Pandit
0 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 3:23 UTC (permalink / raw)
To: Heng Qi, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
On 3/6/2023 10:48 AM, Heng Qi wrote:
>
> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
May be to say inner packet header hash..
This make its little more clear about "which header" that you explained
in the commit log.
> + for tunnel-encapsulated packets.
> +
> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing.
>
> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets.
> @@ -139,6 +142,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
I think this should also say that HASH_TUNNEL requires either of the
F_RSS or F_HASH_REPORT.
Because without it HASH_TUNNEL is not useful.
Right?
if no, than my below comments are meaningless.
> \end{description}
>
> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -198,20 +202,27 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> u8 rss_max_key_size;
> le16 rss_max_indirection_table_length;
> le32 supported_hash_types;
> + le32 supported_tunnel_hash_types;
> };
> \end{lstlisting}
> -The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +The following field, \field{rss_max_key_size} only exists if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
> It specifies the maximum supported length of RSS key in bytes.
>
I think rss_max_key_size field dependency should be only of the existing
feature bits F_RSS and F_HASH_REPORT.
This is because those are the bits really deciding to consider
rss_max_key_size.
> The following field, \field{rss_max_indirection_table_length} only exists if VIRTIO_NET_F_RSS is set.
> It specifies the maximum number of 16-bit entries in RSS indirection table.
>
> The next field, \field{supported_hash_types} only exists if the device supports hash calculation,
> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL is set.
>
Same as above.
> Field \field{supported_hash_types} contains the bitmask of supported hash types.
> See \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types} for details of supported hash types.
>
> +The next field, \field{supported_tunnel_hash_types} only exists if the device
> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is set.
> +
Above line "the next field .." can be just same as "Device supports
inner packet header hash calculation, i.e..."
This is because here, the term "header" is missed, which is present in
the definition of feature bit 52.
>
> The device MUST set \field{rss_max_key_size} to at least 40, if it offers
> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
>
This needs change if the above first comment about rss_max_key_size is
right.
> The device MUST set \field{rss_max_indirection_table_length} to at least 128, if it offers
> VIRTIO_NET_F_RSS.
> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> \begin{itemize}
> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses the hash to determine the receive virtqueue to place incoming packets.
> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The device reports the hash value and the hash type with the packet.
> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The device supports inner hash calculation.
> \end{itemize}
>
inner packet header hash ..
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the encapsulation
> +hash type below indicates that the hash is calculated over the inner header of
> +the encapsulated packet:
> +Hash type applicable for inner payload of the gre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the vxlan-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the geneve-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the ip-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
> +\end{lstlisting}
> +Hash type applicable for inner payload of the nvgre-encapsulated packet
> +\begin{lstlisting}
> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
> +\end{lstlisting}
> +
Any encapsulation technology that includes UDP/L4 header likely do not
prefer based on the inner header. This is because the outer header src
port entropy is added based on the inner header.
I was not able to follow the discussion in v9 that you had with Michael.
Did you conclude if this is needed for vxlan too?
If not, for now it may be better to skip vxlan and nvegre as they
inherently have unique outer header UDP src port based on the inner header.
> \subparagraph{IPv4 packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv4 packets}
> The device calculates the hash on IPv4 packets according to 'Enabled hash types' bitmask as follows:
> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
> (see \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / IPv6 packets without extension header}).
> \end{itemize}
>
> +\subparagraph{Inner hash calculation of an encapsulated packet}
> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it can configure the
> +hash parameters (including \field{hash_tunnel_types}) for inner hash calculation by
> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if the VIRTIO_NET_F_RSS
> +feature is also negotiated, the driver can use the VIRTIO_NET_CTRL_RSS_CONFIG command to
> +configure the hash parameters. If multiple commands are sent, the device configuration
> +will be defined by the last command received.
> +
> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the corresponding
> +encapsulation hash type is set in \field{hash_tunnel_types}, the device calculates the
> +hash on the inner header of an encapsulated packet (See \ref{sec:Device Types
> +/ Network Device / Device Operation / Processing of Incoming Packets /
> +Hash calculation for incoming packets / Tunnel/Encapsulated packet}). If the encapsulation
> +type of an encapsulated packet is not included in \field{hash_tunnel_types} or the value
> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE, the device calculates
> +the hash on the outer header.
> +
> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> +unencapsulated packets.
> +
> +\subparagraph{Tunnel QoS limitation}
> +Note that the limitation mentioned below is not only introduced by inner hash calculation,
> +and the limitation of the tunnel itself, and even the driver may have only one receive queue.
> +
When there is a single receive queue, there is no tunnel QoS issue.
Because the same queue is used one or more tunnels.
So please remove mentioning single queue text here.
Also it is better to first describe the limitation and after the reader
understand, it make sense to say that the limitation already exists
with/without inner header hash calculation.
> +When a specific receive queue is shared by multiple tunnels to receive encapsulating packets,
> +there is no quality of service (QoS) for these packets of multiple tunnels.
"of multiple tunnels at the tail of sentence is not needed because you
already have it at "shared by multiple tunnels".
> For example, when the
> +flooded packets of a certain tunnel are hashed to the queue, it may cause the traffic of this
> +queue to be unbalanced, resulting in potential packet loss and data delay.
> +
Your example is only talking about single queue..
You probably wanted to say,
When the packets of certain tunnels results in spreading these packets
to multiple receive queues, these receive queues may have unbalanced
amount of packets. This can result in a specific receive queue being
full resulting in packet loss.
Or something similar..
> +Possible mitigations:
> +\begin{itemize}
> +\item Use a tool with good forwarding performance such as DPDK to keep the queue from filling up.
I dont think it is necessary to talk about specific tools like DPDK in
specification.
Even if you omit "such as DPDK", the line reads fine.
so I suggest to drop "such as DPDK".
> +\item If the quality of service is unavailable, the driver can set \field{hash_tunnel_types} to
> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash calculation for encapsulated packets.
> +\item Choose a hash key that can avoid queue collisions.
> +\item Outside the device, prevent abnormal traffic from entering or switch the traffic to attack clusters.
> +\end{itemize}
It can still enter the switch but you want to avoid entering the virtio
device.
Cluster is very broad term and not defined in the spec, better to avoid it.
you can rewrite it something like, Perform appropriate QoS before
packets consume the receive buffers..
This is generic solution that can be done in device or egress of switch
etc, which keep the spec scope upto the device.
> +
> \paragraph{Hash reporting for incoming packets}
> \label{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash reporting for incoming packets}
>
> @@ -1392,12 +1495,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 reserved[4];
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> defined in
> \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash types}.
> -Initially the device has all hash types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
> +
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> +Initially the device has all hash types and hash tunnel types disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>
> Field \field{reserved} MUST contain zeroes. It is defined to make the structure to match the layout of virtio_net_rss_config structure,
> defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}.
> @@ -1421,6 +1529,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> le16 max_tx_vq;
> u8 hash_key_length;
> u8 hash_key_data[hash_key_length];
> + le32 hash_tunnel_types;
> };
> \end{lstlisting}
> Field \field{hash_types} contains a bitmask of allowed hash types as
> @@ -1441,6 +1550,9 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>
> Fields \field{hash_key_length} and \field{hash_key_data} define the key to be used in hash calculation.
>
> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash tunnel types as
> +defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets / Hash calculation for incoming packets / Supported/enabled hash tunnel types}.
> +
> \drivernormative{\subparagraph}{Setting RSS parameters}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) }
>
You likely need to add device and driver normative statements derived
from above text and add their references into
device-types/net/*-conformance.tex files.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 3:23 ` [virtio-comment] " Parav Pandit
@ 2023-03-15 12:10 ` Michael S. Tsirkin
-1 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-15 12:10 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
> If not, for now it may be better to skip vxlan and nvegre as they inherently
> have unique outer header UDP src port based on the inner header.
So what's left, GRE? GRE is actually different, in that it's not IP at
all.
So if we are talking about GRE, hash is indeed not calculated at all at
the moment, right? And I would say a natural first step for GRE is
actually adding a hash type that will support this protocol.
How about doing that? It seems like this should be a small step
and completely uncontroversial.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 12:10 ` Michael S. Tsirkin
0 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-15 12:10 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
> If not, for now it may be better to skip vxlan and nvegre as they inherently
> have unique outer header UDP src port based on the inner header.
So what's left, GRE? GRE is actually different, in that it's not IP at
all.
So if we are talking about GRE, hash is indeed not calculated at all at
the moment, right? And I would say a natural first step for GRE is
actually adding a hash type that will support this protocol.
How about doing that? It seems like this should be a small step
and completely uncontroversial.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 12:10 ` Michael S. Tsirkin
@ 2023-03-15 13:27 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-15 13:27 UTC (permalink / raw)
To: Michael S. Tsirkin, Parav Pandit
Cc: virtio-dev, virtio-comment, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/3/15 下午8:10, Michael S. Tsirkin 写道:
> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>> If not, for now it may be better to skip vxlan and nvegre as they inherently
>> have unique outer header UDP src port based on the inner header.
> So what's left, GRE? GRE is actually different, in that it's not IP at
> all.
I do not think so, I mentioned that VXLAN and GENEVE need inner
symmetric hashing, and we need this.
And we know inner hashing doesn't conflict with other ways of adding
entropy.
Thanks.
> So if we are talking about GRE, hash is indeed not calculated at all at
> the moment, right? And I would say a natural first step for GRE is
> actually adding a hash type that will support this protocol.
>
> How about doing that? It seems like this should be a small step
> and completely uncontroversial.
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 13:27 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-15 13:27 UTC (permalink / raw)
To: Michael S. Tsirkin, Parav Pandit
Cc: virtio-dev, virtio-comment, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/3/15 下午8:10, Michael S. Tsirkin 写道:
> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>> If not, for now it may be better to skip vxlan and nvegre as they inherently
>> have unique outer header UDP src port based on the inner header.
> So what's left, GRE? GRE is actually different, in that it's not IP at
> all.
I do not think so, I mentioned that VXLAN and GENEVE need inner
symmetric hashing, and we need this.
And we know inner hashing doesn't conflict with other ways of adding
entropy.
Thanks.
> So if we are talking about GRE, hash is indeed not calculated at all at
> the moment, right? And I would say a natural first step for GRE is
> actually adding a hash type that will support this protocol.
>
> How about doing that? It seems like this should be a small step
> and completely uncontroversial.
>
>
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 12:10 ` Michael S. Tsirkin
@ 2023-03-15 23:24 ` Parav Pandit
-1 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 23:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On 3/15/2023 8:10 AM, Michael S. Tsirkin wrote:
> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>> If not, for now it may be better to skip vxlan and nvegre as they inherently
>> have unique outer header UDP src port based on the inner header.
>
> So what's left, GRE? GRE is actually different, in that it's not IP at
> all.
Sorry, I wrongly wrote nvegre above.
IPoIP, GRE and NVGRE are left.
vxlan and geneve has the udp src entropy.
>
Not sure I understand "its not IP at all".
GRE has outer IP header + GRE header with the key to identify the flow.
The key is effectively the hash for the flow.
> So if we are talking about GRE, hash is indeed not calculated at all at
> the moment, right?
Hash of the outer IP header of the src and dst IP can be still
calculated currently for GRE when the optional key is not present.
> And I would say a natural first step for GRE is
> actually adding a hash type that will support this protocol.
>
For GRE and NVGRE GRE_header.key as the flow/hash identifier should work
without inner header hash.
Older version of the GRE doesn't have key, so inner header hash is useful.
> How about doing that? It seems like this should be a small step
> and completely uncontroversial.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 23:24 ` Parav Pandit
0 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 23:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On 3/15/2023 8:10 AM, Michael S. Tsirkin wrote:
> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>> If not, for now it may be better to skip vxlan and nvegre as they inherently
>> have unique outer header UDP src port based on the inner header.
>
> So what's left, GRE? GRE is actually different, in that it's not IP at
> all.
Sorry, I wrongly wrote nvegre above.
IPoIP, GRE and NVGRE are left.
vxlan and geneve has the udp src entropy.
>
Not sure I understand "its not IP at all".
GRE has outer IP header + GRE header with the key to identify the flow.
The key is effectively the hash for the flow.
> So if we are talking about GRE, hash is indeed not calculated at all at
> the moment, right?
Hash of the outer IP header of the src and dst IP can be still
calculated currently for GRE when the optional key is not present.
> And I would say a natural first step for GRE is
> actually adding a hash type that will support this protocol.
>
For GRE and NVGRE GRE_header.key as the flow/hash identifier should work
without inner header hash.
Older version of the GRE doesn't have key, so inner header hash is useful.
> How about doing that? It seems like this should be a small step
> and completely uncontroversial.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 23:24 ` Parav Pandit
@ 2023-03-16 7:36 ` Michael S. Tsirkin
-1 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-16 7:36 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 07:24:57PM -0400, Parav Pandit wrote:
> > So what's left, GRE? GRE is actually different, in that it's not IP at
> > all.
> Sorry, I wrongly wrote nvegre above.
>
> IPoIP, GRE and NVGRE are left.
>
> vxlan and geneve has the udp src entropy.
>
> >
> Not sure I understand "its not IP at all".
>
GRE header is distinct from IP header and has its own flow ID.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 7:36 ` Michael S. Tsirkin
0 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-16 7:36 UTC (permalink / raw)
To: Parav Pandit
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 07:24:57PM -0400, Parav Pandit wrote:
> > So what's left, GRE? GRE is actually different, in that it's not IP at
> > all.
> Sorry, I wrongly wrote nvegre above.
>
> IPoIP, GRE and NVGRE are left.
>
> vxlan and geneve has the udp src entropy.
>
> >
> Not sure I understand "its not IP at all".
>
GRE header is distinct from IP header and has its own flow ID.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-16 7:36 ` Michael S. Tsirkin
@ 2023-03-16 18:45 ` Parav Pandit
-1 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-16 18:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Thursday, March 16, 2023 3:36 AM
>
> On Wed, Mar 15, 2023 at 07:24:57PM -0400, Parav Pandit wrote:
> > > So what's left, GRE? GRE is actually different, in that it's not IP
> > > at all.
> > Sorry, I wrongly wrote nvegre above.
> >
> > IPoIP, GRE and NVGRE are left.
> >
> > vxlan and geneve has the udp src entropy.
> >
> > >
> > Not sure I understand "its not IP at all".
> >
>
> GRE header is distinct from IP header and has its own flow ID.
Yes. GRE is IP header + GRE header.
The latest one has the key in GRE header, the older one has only IP.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 18:45 ` Parav Pandit
0 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-16 18:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Heng Qi, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Thursday, March 16, 2023 3:36 AM
>
> On Wed, Mar 15, 2023 at 07:24:57PM -0400, Parav Pandit wrote:
> > > So what's left, GRE? GRE is actually different, in that it's not IP
> > > at all.
> > Sorry, I wrongly wrote nvegre above.
> >
> > IPoIP, GRE and NVGRE are left.
> >
> > vxlan and geneve has the udp src entropy.
> >
> > >
> > Not sure I understand "its not IP at all".
> >
>
> GRE header is distinct from IP header and has its own flow ID.
Yes. GRE is IP header + GRE header.
The latest one has the key in GRE header, the older one has only IP.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 23:24 ` Parav Pandit
@ 2023-03-16 13:45 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:45 UTC (permalink / raw)
To: Parav Pandit, Michael S. Tsirkin
Cc: virtio-dev, virtio-comment, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/3/16 上午7:24, Parav Pandit 写道:
>
> On 3/15/2023 8:10 AM, Michael S. Tsirkin wrote:
>> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>>> If not, for now it may be better to skip vxlan and nvegre as they
>>> inherently
>>> have unique outer header UDP src port based on the inner header.
>>
>> So what's left, GRE? GRE is actually different, in that it's not IP at
>> all.
> Sorry, I wrongly wrote nvegre above.
>
> IPoIP, GRE and NVGRE are left.
>
> vxlan and geneve has the udp src entropy.
>
>>
> Not sure I understand "its not IP at all".
>
> GRE has outer IP header + GRE header with the key to identify the flow.
> The key is effectively the hash for the flow.
>
>> So if we are talking about GRE, hash is indeed not calculated at all at
>> the moment, right?
> Hash of the outer IP header of the src and dst IP can be still
> calculated currently for GRE when the optional key is not present.
>
>> And I would say a natural first step for GRE is
>> actually adding a hash type that will support this protocol.
>>
> For GRE and NVGRE GRE_header.key as the flow/hash identifier should
> work without inner header hash.
> Older version of the GRE doesn't have key, so inner header hash is
> useful.
Yes, indeed.
The old GRE does not have keys such as flow id. Even with the new GRE,
we have no chance to use optional fields,
because we have connected with various operators, and the most basic
fields can get the best compatibility.
Thanks.
>
>> How about doing that? It seems like this should be a small step
>> and completely uncontroversial.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 13:45 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:45 UTC (permalink / raw)
To: Parav Pandit, Michael S. Tsirkin
Cc: virtio-dev, virtio-comment, Jason Wang, Yuri Benditovich,
Cornelia Huck, Xuan Zhuo
在 2023/3/16 上午7:24, Parav Pandit 写道:
>
> On 3/15/2023 8:10 AM, Michael S. Tsirkin wrote:
>> On Tue, Mar 14, 2023 at 11:23:55PM -0400, Parav Pandit wrote:
>>> If not, for now it may be better to skip vxlan and nvegre as they
>>> inherently
>>> have unique outer header UDP src port based on the inner header.
>>
>> So what's left, GRE? GRE is actually different, in that it's not IP at
>> all.
> Sorry, I wrongly wrote nvegre above.
>
> IPoIP, GRE and NVGRE are left.
>
> vxlan and geneve has the udp src entropy.
>
>>
> Not sure I understand "its not IP at all".
>
> GRE has outer IP header + GRE header with the key to identify the flow.
> The key is effectively the hash for the flow.
>
>> So if we are talking about GRE, hash is indeed not calculated at all at
>> the moment, right?
> Hash of the outer IP header of the src and dst IP can be still
> calculated currently for GRE when the optional key is not present.
>
>> And I would say a natural first step for GRE is
>> actually adding a hash type that will support this protocol.
>>
> For GRE and NVGRE GRE_header.key as the flow/hash identifier should
> work without inner header hash.
> Older version of the GRE doesn't have key, so inner header hash is
> useful.
Yes, indeed.
The old GRE does not have keys such as flow id. Even with the new GRE,
we have no chance to use optional fields,
because we have connected with various operators, and the most basic
fields can get the best compatibility.
Thanks.
>
>> How about doing that? It seems like this should be a small step
>> and completely uncontroversial.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 3:23 ` [virtio-comment] " Parav Pandit
@ 2023-03-15 13:19 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-15 13:19 UTC (permalink / raw)
To: Parav Pandit, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
在 2023/3/15 上午11:23, Parav Pandit 写道:
>
>
> On 3/6/2023 10:48 AM, Heng Qi wrote:
>
>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> May be to say inner packet header hash..
> This make its little more clear about "which header" that you
> explained in the commit log.
>
Sure, I'll add this.
>> + for tunnel-encapsulated packets.
>> +
>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
>> coalescing.
>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4
>> packets.
>> @@ -139,6 +142,7 @@ \subsubsection{Feature bit
>> requirements}\label{sec:Device Types / Network Device
>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
>> VIRTIO_NET_F_HOST_TSO6.
>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> I think this should also say that HASH_TUNNEL requires either of the
> F_RSS or F_HASH_REPORT.
> Because without it HASH_TUNNEL is not useful.
F_HASH_TUNNEL indicates that the hash should be calculated using the
inner packet header, even without F_RSS or F_HASH_REPORT,
we can continue to use the hash value in scenarios such as RPS or ebpf
programs.
> Right?
> if no, than my below comments are meaningless.
I think it's fine to let F_HASH_TUNNEL rely on F_RSS or _F_HASH_REPORT
as those are probably important scenarios where inner packet header hash
is used.
>
>
>> \end{description}
>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>> @@ -198,20 +202,27 @@ \subsection{Device configuration
>> layout}\label{sec:Device Types / Network Device
>> u8 rss_max_key_size;
>> le16 rss_max_indirection_table_length;
>> le32 supported_hash_types;
>> + le32 supported_tunnel_hash_types;
>> };
>> \end{lstlisting}
>> -The following field, \field{rss_max_key_size} only exists if
>> VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
>> +The following field, \field{rss_max_key_size} only exists if
>> VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or
>> VIRTIO_NET_F_HASH_TUNNEL is set.
>> It specifies the maximum supported length of RSS key in bytes.
>>
> I think rss_max_key_size field dependency should be only of the
> existing feature bits F_RSS and F_HASH_REPORT.
> This is because those are the bits really deciding to consider
> rss_max_key_size.
>
>> The following field, \field{rss_max_indirection_table_length} only
>> exists if VIRTIO_NET_F_RSS is set.
>> It specifies the maximum number of 16-bit entries in RSS
>> indirection table.
>> The next field, \field{supported_hash_types} only exists if the
>> device supports hash calculation,
>> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
>> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or
>> VIRTIO_NET_F_HASH_TUNNEL is set.
>>
> Same as above.
>
>> Field \field{supported_hash_types} contains the bitmask of
>> supported hash types.
>> See \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash types} for details of supported hash
>> types.
>> +The next field, \field{supported_tunnel_hash_types} only exists if
>> the device
>> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is
>> set.
>> +
> Above line "the next field .." can be just same as "Device supports
> inner packet header hash calculation, i.e..."
> This is because here, the term "header" is missed, which is present in
> the definition of feature bit 52.
I'll rephrase the description to make the whole text more consistent.
>
>> The device MUST set \field{rss_max_key_size} to at least 40, if
>> it offers
>> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
>> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
> This needs change if the above first comment about rss_max_key_size is
> right.
>
>> The device MUST set \field{rss_max_indirection_table_length} to at
>> least 128, if it offers
>> VIRTIO_NET_F_RSS.
>> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>> \begin{itemize}
>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
>> the hash to determine the receive virtqueue to place incoming packets.
>> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>> device reports the hash value and the hash type with the packet.
>> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The
>> device supports inner hash calculation.
>> \end{itemize}
>>
> inner packet header hash ..
Ok.
>
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the
>> encapsulation
>> +hash type below indicates that the hash is calculated over the inner
>> header of
>> +the encapsulated packet:
>> +Hash type applicable for inner payload of the gre-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the vxlan-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the geneve-encapsulated
>> packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the ip-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the nvgre-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
>> +\end{lstlisting}
>> +
> Any encapsulation technology that includes UDP/L4 header likely do not
> prefer based on the inner header. This is because the outer header src
> port entropy is added based on the inner header.
>
> I was not able to follow the discussion in v9 that you had with Michael.
> Did you conclude if this is needed for vxlan too?
>
> If not, for now it may be better to skip vxlan and nvegre as they
> inherently have unique outer header UDP src port based on the inner
> header.
Symmetric hashing ignores the order of the five-tuples when calculating
the hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively
can calculate the same hash.
There is a scenario that the two directions client1->client2 and
client2->client1 of the same flow may pass through different tunnels.
In order to allow the data in two directions to be processed by the same
CPU, we need to calculate a symmetric hash based on the inner packet header.
Sorry I didn't mention this earlier just to avoid introducing the
concept of symmetric hashing.
>
>> \subparagraph{IPv4 packets}
>> \label{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / IPv4 packets}
>> The device calculates the hash on IPv4 packets according to
>> 'Enabled hash types' bitmask as follows:
>> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>> (see \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / IPv6 packets without extension header}).
>> \end{itemize}
>> +\subparagraph{Inner hash calculation of an encapsulated packet}
>> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it
>> can configure the
>> +hash parameters (including \field{hash_tunnel_types}) for inner hash
>> calculation by
>> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if
>> the VIRTIO_NET_F_RSS
>> +feature is also negotiated, the driver can use the
>> VIRTIO_NET_CTRL_RSS_CONFIG command to
>> +configure the hash parameters. If multiple commands are sent, the
>> device configuration
>> +will be defined by the last command received.
>> +
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
>> corresponding
>> +encapsulation hash type is set in \field{hash_tunnel_types}, the
>> device calculates the
>> +hash on the inner header of an encapsulated packet (See
>> \ref{sec:Device Types
>> +/ Network Device / Device Operation / Processing of Incoming Packets /
>> +Hash calculation for incoming packets / Tunnel/Encapsulated
>> packet}). If the encapsulation
>> +type of an encapsulated packet is not included in
>> \field{hash_tunnel_types} or the value
>> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
>> the device calculates
>> +the hash on the outer header.
>> +
>> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE
>> by the device for the
>> +unencapsulated packets.
>> +
>> +\subparagraph{Tunnel QoS limitation}
>> +Note that the limitation mentioned below is not only introduced by
>> inner hash calculation,
>> +and the limitation of the tunnel itself, and even the driver may
>> have only one receive queue.
>> +
> When there is a single receive queue, there is no tunnel QoS issue.
> Because the same queue is used one or more tunnels.
> So please remove mentioning single queue text here.
>
Ok.
> Also it is better to first describe the limitation and after the
> reader understand, it make sense to say that the limitation already
> exists with/without inner header hash calculation.
I'll update this.
>
>> +When a specific receive queue is shared by multiple tunnels to
>> receive encapsulating packets,
>> +there is no quality of service (QoS) for these packets of multiple
>> tunnels.
> "of multiple tunnels at the tail of sentence is not needed because you
> already have it at "shared by multiple tunnels".
Ok.
>
>> For example, when the
>> +flooded packets of a certain tunnel are hashed to the queue, it may
>> cause the traffic of this
>> +queue to be unbalanced, resulting in potential packet loss and data
>> delay.
>> +
> Your example is only talking about single queue..
>
> You probably wanted to say,
>
> When the packets of certain tunnels results in spreading these packets
> to multiple receive queues, these receive queues may have unbalanced
> amount of packets. This can result in a specific receive queue being
> full resulting in packet loss.
Yes, I didn't describe clearly. Thanks for the example!
>
> Or something similar..
>
>> +Possible mitigations:
>> +\begin{itemize}
>> +\item Use a tool with good forwarding performance such as DPDK to
>> keep the queue from filling up.
> I dont think it is necessary to talk about specific tools like DPDK in
> specification.
> Even if you omit "such as DPDK", the line reads fine.
> so I suggest to drop "such as DPDK".
I agree.
>
>> +\item If the quality of service is unavailable, the driver can set
>> \field{hash_tunnel_types} to
>> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash
>> calculation for encapsulated packets.
>> +\item Choose a hash key that can avoid queue collisions.
>> +\item Outside the device, prevent abnormal traffic from entering or
>> switch the traffic to attack clusters.
>> +\end{itemize}
> It can still enter the switch but you want to avoid entering the
> virtio device.
> Cluster is very broad term and not defined in the spec, better to
> avoid it.
>
> you can rewrite it something like, Perform appropriate QoS before
> packets consume the receive buffers..
>
> This is generic solution that can be done in device or egress of
> switch etc, which keep the spec scope upto the device.
I see. This does add a new concept "cluster" beyond the device, and I'll
remove it.
>
>> +
>> \paragraph{Hash reporting for incoming packets}
>> \label{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash reporting for incoming packets}
>> @@ -1392,12 +1495,17 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> le16 reserved[4];
>> u8 hash_key_length;
>> u8 hash_key_data[hash_key_length];
>> + le32 hash_tunnel_types;
>> };
>> \end{lstlisting}
>> Field \field{hash_types} contains a bitmask of allowed hash types as
>> defined in
>> \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash types}.
>> -Initially the device has all hash types disabled and reports only
>> VIRTIO_NET_HASH_REPORT_NONE.
>> +
>> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash
>> tunnel types as
>> +defined in \ref{sec:Device Types / Network Device / Device Operation
>> / Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash tunnel types}.
>> +
>> +Initially the device has all hash types and hash tunnel types
>> disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>> Field \field{reserved} MUST contain zeroes. It is defined to make
>> the structure to match the layout of virtio_net_rss_config structure,
>> defined in \ref{sec:Device Types / Network Device / Device
>> Operation / Control Virtqueue / Receive-side scaling (RSS)}.
>> @@ -1421,6 +1529,7 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> le16 max_tx_vq;
>> u8 hash_key_length;
>> u8 hash_key_data[hash_key_length];
>> + le32 hash_tunnel_types;
>> };
>> \end{lstlisting}
>> Field \field{hash_types} contains a bitmask of allowed hash types as
>> @@ -1441,6 +1550,9 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> Fields \field{hash_key_length} and \field{hash_key_data} define
>> the key to be used in hash calculation.
>> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash
>> tunnel types as
>> +defined in \ref{sec:Device Types / Network Device / Device Operation
>> / Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash tunnel types}.
>> +
>> \drivernormative{\subparagraph}{Setting RSS parameters}{Device
>> Types / Network Device / Device Operation / Control Virtqueue /
>> Receive-side scaling (RSS) }
> You likely need to add device and driver normative statements derived
> from above text and add their references into
> device-types/net/*-conformance.tex files.
Sure, thanks for the suggestion! :)
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 13:19 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-15 13:19 UTC (permalink / raw)
To: Parav Pandit, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
在 2023/3/15 上午11:23, Parav Pandit 写道:
>
>
> On 3/6/2023 10:48 AM, Heng Qi wrote:
>
>> +\item[VIRTIO_NET_F_HASH_TUNNEL(52)] Device supports inner header hash
> May be to say inner packet header hash..
> This make its little more clear about "which header" that you
> explained in the commit log.
>
Sure, I'll add this.
>> + for tunnel-encapsulated packets.
>> +
>> \item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications
>> coalescing.
>> \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4
>> packets.
>> @@ -139,6 +142,7 @@ \subsubsection{Feature bit
>> requirements}\label{sec:Device Types / Network Device
>> \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>> \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or
>> VIRTIO_NET_F_HOST_TSO6.
>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> I think this should also say that HASH_TUNNEL requires either of the
> F_RSS or F_HASH_REPORT.
> Because without it HASH_TUNNEL is not useful.
F_HASH_TUNNEL indicates that the hash should be calculated using the
inner packet header, even without F_RSS or F_HASH_REPORT,
we can continue to use the hash value in scenarios such as RPS or ebpf
programs.
> Right?
> if no, than my below comments are meaningless.
I think it's fine to let F_HASH_TUNNEL rely on F_RSS or _F_HASH_REPORT
as those are probably important scenarios where inner packet header hash
is used.
>
>
>> \end{description}
>> \subsubsection{Legacy Interface: Feature bits}\label{sec:Device
>> Types / Network Device / Feature bits / Legacy Interface: Feature bits}
>> @@ -198,20 +202,27 @@ \subsection{Device configuration
>> layout}\label{sec:Device Types / Network Device
>> u8 rss_max_key_size;
>> le16 rss_max_indirection_table_length;
>> le32 supported_hash_types;
>> + le32 supported_tunnel_hash_types;
>> };
>> \end{lstlisting}
>> -The following field, \field{rss_max_key_size} only exists if
>> VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
>> +The following field, \field{rss_max_key_size} only exists if
>> VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or
>> VIRTIO_NET_F_HASH_TUNNEL is set.
>> It specifies the maximum supported length of RSS key in bytes.
>>
> I think rss_max_key_size field dependency should be only of the
> existing feature bits F_RSS and F_HASH_REPORT.
> This is because those are the bits really deciding to consider
> rss_max_key_size.
>
>> The following field, \field{rss_max_indirection_table_length} only
>> exists if VIRTIO_NET_F_RSS is set.
>> It specifies the maximum number of 16-bit entries in RSS
>> indirection table.
>> The next field, \field{supported_hash_types} only exists if the
>> device supports hash calculation,
>> -i.e. if VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT is set.
>> +i.e. if VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or
>> VIRTIO_NET_F_HASH_TUNNEL is set.
>>
> Same as above.
>
>> Field \field{supported_hash_types} contains the bitmask of
>> supported hash types.
>> See \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash types} for details of supported hash
>> types.
>> +The next field, \field{supported_tunnel_hash_types} only exists if
>> the device
>> +supports inner hash calculation, i.e. if VIRTIO_NET_F_HASH_TUNNEL is
>> set.
>> +
> Above line "the next field .." can be just same as "Device supports
> inner packet header hash calculation, i.e..."
> This is because here, the term "header" is missed, which is present in
> the definition of feature bit 52.
I'll rephrase the description to make the whole text more consistent.
>
>> The device MUST set \field{rss_max_key_size} to at least 40, if
>> it offers
>> -VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
>> +VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT or VIRTIO_NET_F_HASH_TUNNEL.
> This needs change if the above first comment about rss_max_key_size is
> right.
>
>> The device MUST set \field{rss_max_indirection_table_length} to at
>> least 128, if it offers
>> VIRTIO_NET_F_RSS.
>> @@ -843,11 +854,13 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>> \begin{itemize}
>> \item The feature VIRTIO_NET_F_RSS was negotiated. The device uses
>> the hash to determine the receive virtqueue to place incoming packets.
>> \item The feature VIRTIO_NET_F_HASH_REPORT was negotiated. The
>> device reports the hash value and the hash type with the packet.
>> +\item The feature VIRTIO_NET_F_HASH_TUNNEL was negotiated. The
>> device supports inner hash calculation.
>> \end{itemize}
>>
> inner packet header hash ..
Ok.
>
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated, the
>> encapsulation
>> +hash type below indicates that the hash is calculated over the inner
>> header of
>> +the encapsulated packet:
>> +Hash type applicable for inner payload of the gre-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GRE (1 << 1)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the vxlan-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_VXLAN (1 << 2)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the geneve-encapsulated
>> packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_GENEVE (1 << 3)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the ip-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_IPIP (1 << 4)
>> +\end{lstlisting}
>> +Hash type applicable for inner payload of the nvgre-encapsulated packet
>> +\begin{lstlisting}
>> +#define VIRTIO_NET_HASH_TUNNEL_TYPE_NVGRE (1 << 5)
>> +\end{lstlisting}
>> +
> Any encapsulation technology that includes UDP/L4 header likely do not
> prefer based on the inner header. This is because the outer header src
> port entropy is added based on the inner header.
>
> I was not able to follow the discussion in v9 that you had with Michael.
> Did you conclude if this is needed for vxlan too?
>
> If not, for now it may be better to skip vxlan and nvegre as they
> inherently have unique outer header UDP src port based on the inner
> header.
Symmetric hashing ignores the order of the five-tuples when calculating
the hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively
can calculate the same hash.
There is a scenario that the two directions client1->client2 and
client2->client1 of the same flow may pass through different tunnels.
In order to allow the data in two directions to be processed by the same
CPU, we need to calculate a symmetric hash based on the inner packet header.
Sorry I didn't mention this earlier just to avoid introducing the
concept of symmetric hashing.
>
>> \subparagraph{IPv4 packets}
>> \label{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / IPv4 packets}
>> The device calculates the hash on IPv4 packets according to
>> 'Enabled hash types' bitmask as follows:
>> @@ -980,6 +1045,44 @@ \subsubsection{Processing of Incoming
>> Packets}\label{sec:Device Types / Network
>> (see \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / IPv6 packets without extension header}).
>> \end{itemize}
>> +\subparagraph{Inner hash calculation of an encapsulated packet}
>> +If the driver negotiates the VIRTIO_NET_F_HASH_TUNNEL feature, it
>> can configure the
>> +hash parameters (including \field{hash_tunnel_types}) for inner hash
>> calculation by
>> +sending the VIRTIO_NET_CTRL_MQ_HASH_CONFIG command. Additionally, if
>> the VIRTIO_NET_F_RSS
>> +feature is also negotiated, the driver can use the
>> VIRTIO_NET_CTRL_RSS_CONFIG command to
>> +configure the hash parameters. If multiple commands are sent, the
>> device configuration
>> +will be defined by the last command received.
>> +
>> +If the feature VIRTIO_NET_F_HASH_TUNNEL is negotiated and the
>> corresponding
>> +encapsulation hash type is set in \field{hash_tunnel_types}, the
>> device calculates the
>> +hash on the inner header of an encapsulated packet (See
>> \ref{sec:Device Types
>> +/ Network Device / Device Operation / Processing of Incoming Packets /
>> +Hash calculation for incoming packets / Tunnel/Encapsulated
>> packet}). If the encapsulation
>> +type of an encapsulated packet is not included in
>> \field{hash_tunnel_types} or the value
>> +of \field{hash_tunnel_types} is VIRTIO_NET_HASH_TUNNEL_TYPE_NONE,
>> the device calculates
>> +the hash on the outer header.
>> +
>> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE
>> by the device for the
>> +unencapsulated packets.
>> +
>> +\subparagraph{Tunnel QoS limitation}
>> +Note that the limitation mentioned below is not only introduced by
>> inner hash calculation,
>> +and the limitation of the tunnel itself, and even the driver may
>> have only one receive queue.
>> +
> When there is a single receive queue, there is no tunnel QoS issue.
> Because the same queue is used one or more tunnels.
> So please remove mentioning single queue text here.
>
Ok.
> Also it is better to first describe the limitation and after the
> reader understand, it make sense to say that the limitation already
> exists with/without inner header hash calculation.
I'll update this.
>
>> +When a specific receive queue is shared by multiple tunnels to
>> receive encapsulating packets,
>> +there is no quality of service (QoS) for these packets of multiple
>> tunnels.
> "of multiple tunnels at the tail of sentence is not needed because you
> already have it at "shared by multiple tunnels".
Ok.
>
>> For example, when the
>> +flooded packets of a certain tunnel are hashed to the queue, it may
>> cause the traffic of this
>> +queue to be unbalanced, resulting in potential packet loss and data
>> delay.
>> +
> Your example is only talking about single queue..
>
> You probably wanted to say,
>
> When the packets of certain tunnels results in spreading these packets
> to multiple receive queues, these receive queues may have unbalanced
> amount of packets. This can result in a specific receive queue being
> full resulting in packet loss.
Yes, I didn't describe clearly. Thanks for the example!
>
> Or something similar..
>
>> +Possible mitigations:
>> +\begin{itemize}
>> +\item Use a tool with good forwarding performance such as DPDK to
>> keep the queue from filling up.
> I dont think it is necessary to talk about specific tools like DPDK in
> specification.
> Even if you omit "such as DPDK", the line reads fine.
> so I suggest to drop "such as DPDK".
I agree.
>
>> +\item If the quality of service is unavailable, the driver can set
>> \field{hash_tunnel_types} to
>> + VIRTIO_NET_HASH_TUNNEL_TYPE_NONE to disable inner hash
>> calculation for encapsulated packets.
>> +\item Choose a hash key that can avoid queue collisions.
>> +\item Outside the device, prevent abnormal traffic from entering or
>> switch the traffic to attack clusters.
>> +\end{itemize}
> It can still enter the switch but you want to avoid entering the
> virtio device.
> Cluster is very broad term and not defined in the spec, better to
> avoid it.
>
> you can rewrite it something like, Perform appropriate QoS before
> packets consume the receive buffers..
>
> This is generic solution that can be done in device or egress of
> switch etc, which keep the spec scope upto the device.
I see. This does add a new concept "cluster" beyond the device, and I'll
remove it.
>
>> +
>> \paragraph{Hash reporting for incoming packets}
>> \label{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash reporting for incoming packets}
>> @@ -1392,12 +1495,17 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> le16 reserved[4];
>> u8 hash_key_length;
>> u8 hash_key_data[hash_key_length];
>> + le32 hash_tunnel_types;
>> };
>> \end{lstlisting}
>> Field \field{hash_types} contains a bitmask of allowed hash types as
>> defined in
>> \ref{sec:Device Types / Network Device / Device Operation /
>> Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash types}.
>> -Initially the device has all hash types disabled and reports only
>> VIRTIO_NET_HASH_REPORT_NONE.
>> +
>> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash
>> tunnel types as
>> +defined in \ref{sec:Device Types / Network Device / Device Operation
>> / Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash tunnel types}.
>> +
>> +Initially the device has all hash types and hash tunnel types
>> disabled and reports only VIRTIO_NET_HASH_REPORT_NONE.
>> Field \field{reserved} MUST contain zeroes. It is defined to make
>> the structure to match the layout of virtio_net_rss_config structure,
>> defined in \ref{sec:Device Types / Network Device / Device
>> Operation / Control Virtqueue / Receive-side scaling (RSS)}.
>> @@ -1421,6 +1529,7 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> le16 max_tx_vq;
>> u8 hash_key_length;
>> u8 hash_key_data[hash_key_length];
>> + le32 hash_tunnel_types;
>> };
>> \end{lstlisting}
>> Field \field{hash_types} contains a bitmask of allowed hash types as
>> @@ -1441,6 +1550,9 @@ \subsubsection{Control
>> Virtqueue}\label{sec:Device Types / Network Device / Devi
>> Fields \field{hash_key_length} and \field{hash_key_data} define
>> the key to be used in hash calculation.
>> +Field \field{hash_tunnel_types} contains a bitmask of allowed hash
>> tunnel types as
>> +defined in \ref{sec:Device Types / Network Device / Device Operation
>> / Processing of Incoming Packets / Hash calculation for incoming
>> packets / Supported/enabled hash tunnel types}.
>> +
>> \drivernormative{\subparagraph}{Setting RSS parameters}{Device
>> Types / Network Device / Device Operation / Control Virtqueue /
>> Receive-side scaling (RSS) }
> You likely need to add device and driver normative statements derived
> from above text and add their references into
> device-types/net/*-conformance.tex files.
Sure, thanks for the suggestion! :)
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 13:19 ` [virtio-comment] " Heng Qi
@ 2023-03-15 15:09 ` Michael S. Tsirkin
-1 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-15 15:09 UTC (permalink / raw)
To: Heng Qi
Cc: Parav Pandit, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 09:19:43PM +0800, Heng Qi wrote:
> > Any encapsulation technology that includes UDP/L4 header likely do not
> > prefer based on the inner header. This is because the outer header src
> > port entropy is added based on the inner header.
> >
> > I was not able to follow the discussion in v9 that you had with Michael.
> > Did you conclude if this is needed for vxlan too?
> >
> > If not, for now it may be better to skip vxlan and nvegre as they
> > inherently have unique outer header UDP src port based on the inner
> > header.
>
> Symmetric hashing ignores the order of the five-tuples when calculating the
> hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively can
> calculate the same hash.
> There is a scenario that the two directions client1->client2 and
> client2->client1 of the same flow may pass through different tunnels.
> In order to allow the data in two directions to be processed by the same
> CPU, we need to calculate a symmetric hash based on the inner packet header.
> Sorry I didn't mention this earlier just to avoid introducing the concept of
> symmetric hashing.
But the hash is already there in the port. Is it then maybe just
the question of ignoring the IP addresses when hashing?
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 15:09 ` Michael S. Tsirkin
0 siblings, 0 replies; 36+ messages in thread
From: Michael S. Tsirkin @ 2023-03-15 15:09 UTC (permalink / raw)
To: Heng Qi
Cc: Parav Pandit, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 09:19:43PM +0800, Heng Qi wrote:
> > Any encapsulation technology that includes UDP/L4 header likely do not
> > prefer based on the inner header. This is because the outer header src
> > port entropy is added based on the inner header.
> >
> > I was not able to follow the discussion in v9 that you had with Michael.
> > Did you conclude if this is needed for vxlan too?
> >
> > If not, for now it may be better to skip vxlan and nvegre as they
> > inherently have unique outer header UDP src port based on the inner
> > header.
>
> Symmetric hashing ignores the order of the five-tuples when calculating the
> hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively can
> calculate the same hash.
> There is a scenario that the two directions client1->client2 and
> client2->client1 of the same flow may pass through different tunnels.
> In order to allow the data in two directions to be processed by the same
> CPU, we need to calculate a symmetric hash based on the inner packet header.
> Sorry I didn't mention this earlier just to avoid introducing the concept of
> symmetric hashing.
But the hash is already there in the port. Is it then maybe just
the question of ignoring the IP addresses when hashing?
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 15:09 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-03-16 13:26 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Parav Pandit, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/3/15 下午11:09, Michael S. Tsirkin 写道:
> On Wed, Mar 15, 2023 at 09:19:43PM +0800, Heng Qi wrote:
>>> Any encapsulation technology that includes UDP/L4 header likely do not
>>> prefer based on the inner header. This is because the outer header src
>>> port entropy is added based on the inner header.
>>>
>>> I was not able to follow the discussion in v9 that you had with Michael.
>>> Did you conclude if this is needed for vxlan too?
>>>
>>> If not, for now it may be better to skip vxlan and nvegre as they
>>> inherently have unique outer header UDP src port based on the inner
>>> header.
>> Symmetric hashing ignores the order of the five-tuples when calculating the
>> hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively can
>> calculate the same hash.
>> There is a scenario that the two directions client1->client2 and
>> client2->client1 of the same flow may pass through different tunnels.
>> In order to allow the data in two directions to be processed by the same
>> CPU, we need to calculate a symmetric hash based on the inner packet header.
>> Sorry I didn't mention this earlier just to avoid introducing the concept of
>> symmetric hashing.
> But the hash is already there in the port. Is it then maybe just
> the question of ignoring the IP addresses when hashing?
We do not ignore the IP address, because after the tunnel sends the
packets to the processing host,
the processing host will parse the outer headers, and then use the inner
symmetric hash to hand over the
packets of the same flow to the same cpu for processing (for the network
topology, please check my latest reply thread).
Thanks.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 13:26 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Parav Pandit, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/3/15 下午11:09, Michael S. Tsirkin 写道:
> On Wed, Mar 15, 2023 at 09:19:43PM +0800, Heng Qi wrote:
>>> Any encapsulation technology that includes UDP/L4 header likely do not
>>> prefer based on the inner header. This is because the outer header src
>>> port entropy is added based on the inner header.
>>>
>>> I was not able to follow the discussion in v9 that you had with Michael.
>>> Did you conclude if this is needed for vxlan too?
>>>
>>> If not, for now it may be better to skip vxlan and nvegre as they
>>> inherently have unique outer header UDP src port based on the inner
>>> header.
>> Symmetric hashing ignores the order of the five-tuples when calculating the
>> hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively can
>> calculate the same hash.
>> There is a scenario that the two directions client1->client2 and
>> client2->client1 of the same flow may pass through different tunnels.
>> In order to allow the data in two directions to be processed by the same
>> CPU, we need to calculate a symmetric hash based on the inner packet header.
>> Sorry I didn't mention this earlier just to avoid introducing the concept of
>> symmetric hashing.
> But the hash is already there in the port. Is it then maybe just
> the question of ignoring the IP addresses when hashing?
We do not ignore the IP address, because after the tunnel sends the
packets to the processing host,
the processing host will parse the outer headers, and then use the inner
symmetric hash to hand over the
packets of the same flow to the same cpu for processing (for the network
topology, please check my latest reply thread).
Thanks.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-dev] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 13:19 ` [virtio-comment] " Heng Qi
@ 2023-03-15 23:06 ` Parav Pandit
-1 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 23:06 UTC (permalink / raw)
To: Heng Qi, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
On 3/15/2023 9:19 AM, Heng Qi wrote:
>
>
> 在 2023/3/15 上午11:23, Parav Pandit 写道:
>>
>>
>> On 3/6/2023 10:48 AM, Heng Qi wrote:
>>
[..]
>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
>> I think this should also say that HASH_TUNNEL requires either of the
>> F_RSS or F_HASH_REPORT.
>> Because without it HASH_TUNNEL is not useful.
>
> F_HASH_TUNNEL indicates that the hash should be calculated using the
> inner packet header, even without F_RSS or F_HASH_REPORT,
> we can continue to use the hash value in scenarios such as RPS or ebpf
> programs.
Yes.
Even for rps or ebpf programs, F_HASH_TUNNEL is fine.
When such feature arrives in future, it above line will have OR
condition for RPS feature bit.
>
>
> I think it's fine to let F_HASH_TUNNEL rely on F_RSS or _F_HASH_REPORT
> as those are probably important scenarios where inner packet header hash
> is used.
Yes.
>> If not, for now it may be better to skip vxlan and nvegre as they
>> inherently have unique outer header UDP src port based on the inner
>> header.
>
> Symmetric hashing ignores the order of the five-tuples when calculating
> the hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively
> can calculate the same hash.
> There is a scenario that the two directions client1->client2 and
> client2->client1 of the same flow may pass through different tunnels.
> In order to allow the data in two directions to be processed by the same
> CPU, we need to calculate a symmetric hash based on the inner packet
> header.
I am lost in two directions and two clients above.
When you say two directions, do you mean tx and rx?
and do symmetric hashing between tx and rx between two end points within
single tunnel?
>>> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE
>>> by the device for the
>>> +unencapsulated packets.
>>> +
I missed this before. unencapsulated is not a term.
s/unencapsulated packets/Non encapsulated packets or non tunneled packets
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-15 23:06 ` Parav Pandit
0 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-15 23:06 UTC (permalink / raw)
To: Heng Qi, virtio-dev, virtio-comment
Cc: Michael S . Tsirkin, Jason Wang, Yuri Benditovich, Cornelia Huck,
Xuan Zhuo
On 3/15/2023 9:19 AM, Heng Qi wrote:
>
>
> 在 2023/3/15 上午11:23, Parav Pandit 写道:
>>
>>
>> On 3/6/2023 10:48 AM, Heng Qi wrote:
>>
[..]
>>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
>>> +\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
>> I think this should also say that HASH_TUNNEL requires either of the
>> F_RSS or F_HASH_REPORT.
>> Because without it HASH_TUNNEL is not useful.
>
> F_HASH_TUNNEL indicates that the hash should be calculated using the
> inner packet header, even without F_RSS or F_HASH_REPORT,
> we can continue to use the hash value in scenarios such as RPS or ebpf
> programs.
Yes.
Even for rps or ebpf programs, F_HASH_TUNNEL is fine.
When such feature arrives in future, it above line will have OR
condition for RPS feature bit.
>
>
> I think it's fine to let F_HASH_TUNNEL rely on F_RSS or _F_HASH_REPORT
> as those are probably important scenarios where inner packet header hash
> is used.
Yes.
>> If not, for now it may be better to skip vxlan and nvegre as they
>> inherently have unique outer header UDP src port based on the inner
>> header.
>
> Symmetric hashing ignores the order of the five-tuples when calculating
> the hash, that is, using (a1,a2,p1,p2) and (a2,a1,p2,p1) respectively
> can calculate the same hash.
> There is a scenario that the two directions client1->client2 and
> client2->client1 of the same flow may pass through different tunnels.
> In order to allow the data in two directions to be processed by the same
> CPU, we need to calculate a symmetric hash based on the inner packet
> header.
I am lost in two directions and two clients above.
When you say two directions, do you mean tx and rx?
and do symmetric hashing between tx and rx between two end points within
single tunnel?
>>> +\field{hash_tunnel_types} is set to VIRTIO_NET_HASH_TUNNEL_TYPE_NONE
>>> by the device for the
>>> +unencapsulated packets.
>>> +
I missed this before. unencapsulated is not a term.
s/unencapsulated packets/Non encapsulated packets or non tunneled packets
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-dev] Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-15 23:06 ` [virtio-comment] " Parav Pandit
@ 2023-03-16 13:38 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:38 UTC (permalink / raw)
To: Parav Pandit
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 07:06:53PM -0400, Parav Pandit wrote:
>
>
> On 3/15/2023 9:19 AM, Heng Qi wrote:
> >
> >
> >在 2023/3/15 上午11:23, Parav Pandit 写道:
> >>
> >>
> >>On 3/6/2023 10:48 AM, Heng Qi wrote:
> >>
> [..]
> >>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >>>+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>I think this should also say that HASH_TUNNEL requires either of
> >>the F_RSS or F_HASH_REPORT.
> >>Because without it HASH_TUNNEL is not useful.
> >
> >F_HASH_TUNNEL indicates that the hash should be calculated using
> >the inner packet header, even without F_RSS or F_HASH_REPORT,
> >we can continue to use the hash value in scenarios such as RPS or
> >ebpf programs.
> Yes.
> Even for rps or ebpf programs, F_HASH_TUNNEL is fine.
> When such feature arrives in future, it above line will have OR
> condition for RPS feature bit.
>
> >
> >
> >I think it's fine to let F_HASH_TUNNEL rely on F_RSS or
> >_F_HASH_REPORT as those are probably important scenarios where
> >inner packet header hash is used.
> Yes.
>
> >>If not, for now it may be better to skip vxlan and nvegre as
> >>they inherently have unique outer header UDP src port based on
> >>the inner header.
> >
> >Symmetric hashing ignores the order of the five-tuples when
> >calculating the hash, that is, using (a1,a2,p1,p2) and
> >(a2,a1,p2,p1) respectively can calculate the same hash.
> >There is a scenario that the two directions client1->client2 and
> >client2->client1 of the same flow may pass through different
> >tunnels.
> >In order to allow the data in two directions to be processed by
> >the same CPU, we need to calculate a symmetric hash based on the
> >inner packet header.
> I am lost in two directions and two clients above.
> When you say two directions, do you mean tx and rx?
> and do symmetric hashing between tx and rx between two end points
> within single tunnel?
>
A rough description of this scene is as follows:
Client1 sends packets to client2, and client2 sends packets to client1 respectively.
This is called two directions, and they are in the same flow.
The packets in the two directions may reach the processing host through
different tunnels. In this scenario, we need to hash these packets to the
same queue for the same cpu to process, so inner symmetric hashing is required.
client1 client2
| |
| __________ |
+----->| tunnels |<-------+
|---------|
| |
| |
| |
v v
+-----------------+
| processing host |
+-----------------+
> >>>+\field{hash_tunnel_types} is set to
> >>>VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> >>>+unencapsulated packets.
> >>>+
> I missed this before. unencapsulated is not a term.
> s/unencapsulated packets/Non encapsulated packets or non tunneled packets
>
Yes, you are right!
Thanks.
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 13:38 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-16 13:38 UTC (permalink / raw)
To: Parav Pandit
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
On Wed, Mar 15, 2023 at 07:06:53PM -0400, Parav Pandit wrote:
>
>
> On 3/15/2023 9:19 AM, Heng Qi wrote:
> >
> >
> >在 2023/3/15 上午11:23, Parav Pandit 写道:
> >>
> >>
> >>On 3/6/2023 10:48 AM, Heng Qi wrote:
> >>
> [..]
> >>> \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> >>>+\item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ.
> >>I think this should also say that HASH_TUNNEL requires either of
> >>the F_RSS or F_HASH_REPORT.
> >>Because without it HASH_TUNNEL is not useful.
> >
> >F_HASH_TUNNEL indicates that the hash should be calculated using
> >the inner packet header, even without F_RSS or F_HASH_REPORT,
> >we can continue to use the hash value in scenarios such as RPS or
> >ebpf programs.
> Yes.
> Even for rps or ebpf programs, F_HASH_TUNNEL is fine.
> When such feature arrives in future, it above line will have OR
> condition for RPS feature bit.
>
> >
> >
> >I think it's fine to let F_HASH_TUNNEL rely on F_RSS or
> >_F_HASH_REPORT as those are probably important scenarios where
> >inner packet header hash is used.
> Yes.
>
> >>If not, for now it may be better to skip vxlan and nvegre as
> >>they inherently have unique outer header UDP src port based on
> >>the inner header.
> >
> >Symmetric hashing ignores the order of the five-tuples when
> >calculating the hash, that is, using (a1,a2,p1,p2) and
> >(a2,a1,p2,p1) respectively can calculate the same hash.
> >There is a scenario that the two directions client1->client2 and
> >client2->client1 of the same flow may pass through different
> >tunnels.
> >In order to allow the data in two directions to be processed by
> >the same CPU, we need to calculate a symmetric hash based on the
> >inner packet header.
> I am lost in two directions and two clients above.
> When you say two directions, do you mean tx and rx?
> and do symmetric hashing between tx and rx between two end points
> within single tunnel?
>
A rough description of this scene is as follows:
Client1 sends packets to client2, and client2 sends packets to client1 respectively.
This is called two directions, and they are in the same flow.
The packets in the two directions may reach the processing host through
different tunnels. In this scenario, we need to hash these packets to the
same queue for the same cpu to process, so inner symmetric hashing is required.
client1 client2
| |
| __________ |
+----->| tunnels |<-------+
|---------|
| |
| |
| |
v v
+-----------------+
| processing host |
+-----------------+
> >>>+\field{hash_tunnel_types} is set to
> >>>VIRTIO_NET_HASH_TUNNEL_TYPE_NONE by the device for the
> >>>+unencapsulated packets.
> >>>+
> I missed this before. unencapsulated is not a term.
> s/unencapsulated packets/Non encapsulated packets or non tunneled packets
>
Yes, you are right!
Thanks.
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread* [virtio-dev] RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-16 13:38 ` Heng Qi
@ 2023-03-16 19:55 ` Parav Pandit
-1 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-16 19:55 UTC (permalink / raw)
To: Heng Qi
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Thursday, March 16, 2023 9:39 AM
> A rough description of this scene is as follows:
> Client1 sends packets to client2, and client2 sends packets to client1
> respectively.
> This is called two directions, and they are in the same flow.
> The packets in the two directions may reach the processing host through
> different tunnels. In this scenario, we need to hash these packets to the same
> queue for the same cpu to process, so inner symmetric hashing is required.
>
>
> client1 client2
> | |
> | __________ |
> +----->| tunnels |<-------+
> |---------|
> | |
> | |
> | |
> v v
> +-----------------+
> | processing host |
> +-----------------+
>
I understood now. The key piece is the processing host is different from clients 1 and 2, I was missing that.
The key is there are two tunnels and processing data on the inner header for the different tunnels is your primary requirement.
Hence inner hashing helps to serialize them.
^ permalink raw reply [flat|nested] 36+ messages in thread
* RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-16 19:55 ` Parav Pandit
0 siblings, 0 replies; 36+ messages in thread
From: Parav Pandit @ 2023-03-16 19:55 UTC (permalink / raw)
To: Heng Qi
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Thursday, March 16, 2023 9:39 AM
> A rough description of this scene is as follows:
> Client1 sends packets to client2, and client2 sends packets to client1
> respectively.
> This is called two directions, and they are in the same flow.
> The packets in the two directions may reach the processing host through
> different tunnels. In this scenario, we need to hash these packets to the same
> queue for the same cpu to process, so inner symmetric hashing is required.
>
>
> client1 client2
> | |
> | __________ |
> +----->| tunnels |<-------+
> |---------|
> | |
> | |
> | |
> v v
> +-----------------+
> | processing host |
> +-----------------+
>
I understood now. The key piece is the processing host is different from clients 1 and 2, I was missing that.
The key is there are two tunnels and processing data on the inner header for the different tunnels is your primary requirement.
Hence inner hashing helps to serialize them.
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [virtio-dev] RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
2023-03-16 19:55 ` Parav Pandit
@ 2023-03-17 8:38 ` Heng Qi
-1 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-17 8:38 UTC (permalink / raw)
To: Parav Pandit
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/3/17 上午3:55, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Thursday, March 16, 2023 9:39 AM
>> A rough description of this scene is as follows:
>> Client1 sends packets to client2, and client2 sends packets to client1
>> respectively.
>> This is called two directions, and they are in the same flow.
>> The packets in the two directions may reach the processing host through
>> different tunnels. In this scenario, we need to hash these packets to the same
>> queue for the same cpu to process, so inner symmetric hashing is required.
>>
>>
>> client1 client2
>> | |
>> | __________ |
>> +----->| tunnels |<-------+
>> |---------|
>> | |
>> | |
>> | |
>> v v
>> +-----------------+
>> | processing host |
>> +-----------------+
>>
> I understood now. The key piece is the processing host is different from clients 1 and 2, I was missing that.
> The key is there are two tunnels and processing data on the inner header for the different tunnels is your primary requirement.
> Hence inner hashing helps to serialize them.
Yes, you are right.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 36+ messages in thread
* [virtio-comment] Re: [virtio-dev] RE: [virtio-comment] Re: [PATCH v10] virtio-net: support inner header hash
@ 2023-03-17 8:38 ` Heng Qi
0 siblings, 0 replies; 36+ messages in thread
From: Heng Qi @ 2023-03-17 8:38 UTC (permalink / raw)
To: Parav Pandit
Cc: Michael S . Tsirkin, virtio-dev, virtio-comment, Jason Wang,
Yuri Benditovich, Cornelia Huck, Xuan Zhuo
在 2023/3/17 上午3:55, Parav Pandit 写道:
>> From: Heng Qi <hengqi@linux.alibaba.com>
>> Sent: Thursday, March 16, 2023 9:39 AM
>> A rough description of this scene is as follows:
>> Client1 sends packets to client2, and client2 sends packets to client1
>> respectively.
>> This is called two directions, and they are in the same flow.
>> The packets in the two directions may reach the processing host through
>> different tunnels. In this scenario, we need to hash these packets to the same
>> queue for the same cpu to process, so inner symmetric hashing is required.
>>
>>
>> client1 client2
>> | |
>> | __________ |
>> +----->| tunnels |<-------+
>> |---------|
>> | |
>> | |
>> | |
>> v v
>> +-----------------+
>> | processing host |
>> +-----------------+
>>
> I understood now. The key piece is the processing host is different from clients 1 and 2, I was missing that.
> The key is there are two tunnels and processing data on the inner header for the different tunnels is your primary requirement.
> Hence inner hashing helps to serialize them.
Yes, you are right.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 36+ messages in thread