[PATCH 1/1] doc: add steps to configure VF interface as trusted

[PATCH 1/1] doc: add steps to configure VF interface as trusted

[Asaf Penso]

Trusted VF is needed to offload rules with rte_flow to a group
that is bigger than 0.
The configuration is done in two parts: driver and FW.

This patch adds the needed steps to configure a VF to be trusted.

Signed-off-by: Asaf Penso <asafp@nvidia.com>
Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
---
 doc/guides/nics/mlx5.rst | 50 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 0a92145796..6d08ae53cb 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -1614,3 +1614,53 @@ both the meters in hierarchy on that flow.
    add port meter policy 0 2 g_actions meter mtr_id M / end y_actions end r_actions drop / end
    create port meter 0 N 2 2 yes 0xffff 1 0
    flow create 0 ingress group 1 pattern eth / end actions meter mtr_id N / end
+
+How to configure a VF as trusted
+--------------------------------
+
+This section demonstrates how to configure a virtual function (VF) interface as trusted.
+Trusted VF is needed to offload rules with rte_flow to a group that is bigger than 0.
+The configuration is done in two parts: driver and FW.
+
+The procedure below is an example of using a ConnectX-5 adapter card (pf0) with 2 VFs:
+
+#. Create 2 VFs on the PF pf0 when in Legacy SR-IOV mode::
+
+   $ echo 2 > /sys/class/net/pf0/device/mlx5_num_vfs
+
+#. Verify the VFs are created:
+
+   .. code-block:: console
+
+      $ lspci | grep Mellanox
+      82:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+      82:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+
+#. Unbind all VFs. For each VF PCIe, using the following command to unbind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/unbind
+
+#. Set the VFs to be trusted for the kernel by using one of the methods below:
+      - Using sysfs file::
+
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/0/trust
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/1/trust
+
+      - Using “ip link” command::
+
+        $ ip link set p0 vf 0 trust on
+        $ ip link set p0 vf 1 trust on
+
+#. Configure all VFs using mlxreg::
+
+   $ mlxreg -d /dev/mst/mt4121_pciconf0 --reg_name VHCA_TRUST_LEVEL --yes --set "all_vhca=0x1,trust_level=0x1"
+
+   .. note::
+
+      Firmware version used must be >= xx.29.1016 and MFT >= 4.18
+
+#. For each VF PCIe, using the following command to bind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
\ No newline at end of file
-- 
2.18.2

Re: [PATCH 1/1] doc: add steps to configure VF interface as trusted

[Stephen Hemminger]

On Sun, 27 Feb 2022 17:46:17 +0200
Asaf Penso <asafp@nvidia.com> wrote:

> +#. For each VF PCIe, using the following command to bind the driver::
> +
> +   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
> \ No newline at end of file

Please change your editor and/or git settings so there is a new line
at the end of all text files.

Re: [PATCH 1/1] doc: add steps to configure VF interface as trusted

[Thomas Monjalon]

27/02/2022 17:44, Stephen Hemminger:
> On Sun, 27 Feb 2022 17:46:17 +0200
> Asaf Penso <asafp@nvidia.com> wrote:
> 
> > +#. For each VF PCIe, using the following command to bind the driver::
> > +
> > +   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
> > \ No newline at end of file
> 
> Please change your editor and/or git settings so there is a new line
> at the end of all text files.

Isn't it a problem with Eclipse editor?

Re: [PATCH 1/1] doc: add steps to configure VF interface as trusted

[Stephen Hemminger]

On Sun, 27 Feb 2022 18:24:42 +0100
Thomas Monjalon <thomas@monjalon.net> wrote:

> 27/02/2022 17:44, Stephen Hemminger:
> > On Sun, 27 Feb 2022 17:46:17 +0200
> > Asaf Penso <asafp@nvidia.com> wrote:
> >   
> > > +#. For each VF PCIe, using the following command to bind the driver::
> > > +
> > > +   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
> > > \ No newline at end of file  
> > 
> > Please change your editor and/or git settings so there is a new line
> > at the end of all text files.  
> 
> Isn't it a problem with Eclipse editor?

It is a configuration option in almost all editors:

Eclipse:
https://www.newt.com/java/eclipse/

Customize code formatting
Go to Windows > Preferences > Java > Code Style > Code Formatter and click on the Edit button.

Check New Lines > Insert new line at end of file

https://thoughtbot.com/blog/no-newline-at-end-of-file

Following the rules in your editor
You can make sure you follow this rule easily:

For Vim users, you’re all set out of the box! Just don’t change your eol setting.
For Emacs users, add (setq require-final-newline t) to your .emacs or .emacs.d/init.el file.
For Android Studio, RubyMine, PyCharm, and other IntelliJ, set “Ensure line feed at file end on Save” under “Editor.”
For Atom, you’re also all set out of the box. Keep that via the Whitespace plugin.
For VS Code, set "files.insertFinalNewline": true.
For Sublime, set the ensure_newline_at_eof_on_save option to true.
For TextMate, you can install the Avian Missing Bundle and add TM_STRIP_WHITESPACE_ON_SAVE = true to your .tm_properties file.

[PATCH v2] doc: add steps to configure VF interface as trusted

[Asaf Penso]

Trusted VF is needed to offload rules with rte_flow to a group
that is bigger than 0.
The configuration is done in two parts: driver and FW.

This patch adds the needed steps to configure a VF to be trusted.

Signed-off-by: Asaf Penso <asafp@nvidia.com>
Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
---
 doc/guides/nics/mlx5.rst | 75 +++++++++++++++++++++++++++++++++-------
 1 file changed, 63 insertions(+), 12 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index c31a154181..e853d6f6a2 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -498,9 +498,9 @@ Limitations
 
   - c_rsvd0_v: C bit, K bit, S bit
   - protocol type
-  - checksum
-  - key
-  - sequence
+  - Checksum
+  - Key
+  - Sequence
 
   Matching on checksum and sequence needs OFED 5.6+.
 
@@ -847,10 +847,10 @@ for an additional list of options shared with other mlx5 drivers.
   By default (if the ``tx_pp`` is not specified) send scheduling on timestamps
   feature is disabled.
 
-  Starting with ConnectX-7 the capability to schedule traffic directly
-  on timestamp specified in descriptor is provided,
-  no extra objects are needed anymore and scheduling capability
-  is advertised and handled regardless ``tx_pp`` parameter presence.
+  Starting since ConnectX-7 the capability to schedule traffic directly
+  on timestamp specified in descriptor is provided, no extra objects are
+  needed anymore and scheduling capability is advertised and handled
+  regardless tx_pp parameter presence.
 
 - ``tx_skew`` parameter [int]
 
@@ -963,13 +963,14 @@ for an additional list of options shared with other mlx5 drivers.
   Value 0 means legacy Verbs flow offloading.
 
   Value 1 enables the DV flow steering assuming it is supported by the
-  driver (requires rdma-core 24 or higher).
+  driver (RDMA Core library version is rdma-core-24.0 or higher).
 
-  Value 2 enables the WQE based hardware steering.
-  In this mode, only queue-based flow management is supported.
+  Value 2 enables the WQE based hardware steering. In this mode only
+  the queue-based rte_flow_q flow management is supported.
 
-  It is configured by default to 1 (DV flow steering) if supported.
-  Otherwise, the value is 0 which indicates legacy Verbs flow offloading.
+  Configured by default to 1 DV flow steering if the driver(RDMA CORE library)
+  supported. Otherwise, the value will be 0 which indicates legacy Verbs flow
+  offloading.
 
 - ``dv_esw_en`` parameter [int]
 
@@ -1613,3 +1614,53 @@ both the meters in hierarchy on that flow.
    add port meter policy 0 2 g_actions meter mtr_id M / end y_actions end r_actions drop / end
    create port meter 0 N 2 2 yes 0xffff 1 0
    flow create 0 ingress group 1 pattern eth / end actions meter mtr_id N / end
+
+How to configure a VF as trusted
+--------------------------------
+
+This section demonstrates how to configure a virtual function (VF) interface as trusted.
+Trusted VF is needed to offload rules with rte_flow to a group that is bigger than 0.
+The configuration is done in two parts: driver and FW.
+
+The procedure below is an example of using a ConnectX-5 adapter card (pf0) with 2 VFs:
+
+#. Create 2 VFs on the PF pf0 when in Legacy SR-IOV mode::
+
+   $ echo 2 > /sys/class/net/pf0/device/mlx5_num_vfs
+
+#. Verify the VFs are created:
+
+   .. code-block:: console
+
+      $ lspci | grep Mellanox
+      82:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+      82:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+
+#. Unbind all VFs. For each VF PCIe, using the following command to unbind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/unbind
+
+#. Set the VFs to be trusted for the kernel by using one of the methods below:
+      - Using sysfs file::
+
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/0/trust
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/1/trust
+
+      - Using “ip link” command::
+
+        $ ip link set p0 vf 0 trust on
+        $ ip link set p0 vf 1 trust on
+
+#. Configure all VFs using mlxreg::
+
+   $ mlxreg -d /dev/mst/mt4121_pciconf0 --reg_name VHCA_TRUST_LEVEL --yes --set "all_vhca=0x1,trust_level=0x1"
+
+   .. note::
+
+      Firmware version used must be >= xx.29.1016 and MFT >= 4.18
+
+#. For each VF PCIe, using the following command to bind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
-- 
2.18.2

[PATCH v2] doc: add steps to configure VF interface as trusted

[Asaf Penso]

Trusted VF is needed to offload rules with rte_flow to a group
that is bigger than 0.
The configuration is done in two parts: driver and FW.

This patch adds the needed steps to configure a VF to be trusted.

Signed-off-by: Asaf Penso <asafp@nvidia.com>
Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
---
v3: cleanup the patches
v2: added new line at the end of the file - comment from Stephen Hemminger
---
doc/guides/nics/mlx5.rst | 50 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index c31a154181..a28b31d751 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -1613,3 +1613,53 @@ both the meters in hierarchy on that flow.
    add port meter policy 0 2 g_actions meter mtr_id M / end y_actions end r_actions drop / end
    create port meter 0 N 2 2 yes 0xffff 1 0
    flow create 0 ingress group 1 pattern eth / end actions meter mtr_id N / end
+
+How to configure a VF as trusted
+--------------------------------
+
+This section demonstrates how to configure a virtual function (VF) interface as trusted.
+Trusted VF is needed to offload rules with rte_flow to a group that is bigger than 0.
+The configuration is done in two parts: driver and FW.
+
+The procedure below is an example of using a ConnectX-5 adapter card (pf0) with 2 VFs:
+
+#. Create 2 VFs on the PF pf0 when in Legacy SR-IOV mode::
+
+   $ echo 2 > /sys/class/net/pf0/device/mlx5_num_vfs
+
+#. Verify the VFs are created:
+
+   .. code-block:: console
+
+      $ lspci | grep Mellanox
+      82:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+      82:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+
+#. Unbind all VFs. For each VF PCIe, using the following command to unbind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/unbind
+
+#. Set the VFs to be trusted for the kernel by using one of the methods below:
+      - Using sysfs file::
+
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/0/trust
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/1/trust
+
+      - Using “ip link” command::
+
+        $ ip link set p0 vf 0 trust on
+        $ ip link set p0 vf 1 trust on
+
+#. Configure all VFs using mlxreg::
+
+   $ mlxreg -d /dev/mst/mt4121_pciconf0 --reg_name VHCA_TRUST_LEVEL --yes --set "all_vhca=0x1,trust_level=0x1"
+
+   .. note::
+
+      Firmware version used must be >= xx.29.1016 and MFT >= 4.18
+
+#. For each VF PCIe, using the following command to bind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
-- 
2.18.2

[PATCH v3] doc: add steps to configure VF interface as trusted

[Asaf Penso]

Trusted VF is needed to offload rules with rte_flow to a group
that is bigger than 0.
The configuration is done in two parts: driver and FW.

This patch adds the needed steps to configure a VF to be trusted.

Signed-off-by: Asaf Penso <asafp@nvidia.com>
Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
---
v3: cleanup the patches
v2: added new line at the end of the file - comment from Stephen Hemminger
---
 doc/guides/nics/mlx5.rst | 50 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index c31a154181..a28b31d751 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -1613,3 +1613,53 @@ both the meters in hierarchy on that flow.
    add port meter policy 0 2 g_actions meter mtr_id M / end y_actions end r_actions drop / end
    create port meter 0 N 2 2 yes 0xffff 1 0
    flow create 0 ingress group 1 pattern eth / end actions meter mtr_id N / end
+
+How to configure a VF as trusted
+--------------------------------
+
+This section demonstrates how to configure a virtual function (VF) interface as trusted.
+Trusted VF is needed to offload rules with rte_flow to a group that is bigger than 0.
+The configuration is done in two parts: driver and FW.
+
+The procedure below is an example of using a ConnectX-5 adapter card (pf0) with 2 VFs:
+
+#. Create 2 VFs on the PF pf0 when in Legacy SR-IOV mode::
+
+   $ echo 2 > /sys/class/net/pf0/device/mlx5_num_vfs
+
+#. Verify the VFs are created:
+
+   .. code-block:: console
+
+      $ lspci | grep Mellanox
+      82:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
+      82:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+      82:00.3 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function]
+
+#. Unbind all VFs. For each VF PCIe, using the following command to unbind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/unbind
+
+#. Set the VFs to be trusted for the kernel by using one of the methods below:
+      - Using sysfs file::
+
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/0/trust
+        $ echo ON | tee /sys/class/net/pf0/device/sriov/1/trust
+
+      - Using “ip link” command::
+
+        $ ip link set p0 vf 0 trust on
+        $ ip link set p0 vf 1 trust on
+
+#. Configure all VFs using mlxreg::
+
+   $ mlxreg -d /dev/mst/mt4121_pciconf0 --reg_name VHCA_TRUST_LEVEL --yes --set "all_vhca=0x1,trust_level=0x1"
+
+   .. note::
+
+      Firmware version used must be >= xx.29.1016 and MFT >= 4.18
+
+#. For each VF PCIe, using the following command to bind the driver::
+
+   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
-- 
2.18.2

RE: [PATCH 1/1] doc: add steps to configure VF interface as trusted

[Asaf Penso]

Thank you for the comment, Stephen.
I also saw you update the tool, which is why I didn't see the missing line.
I uploaded v3: http://patches.dpdk.org/project/dpdk/patch/20220228081117.3013452-1-asafp@nvidia.com/

Regards,
Asaf Penso

>-----Original Message-----
>From: Stephen Hemminger <stephen@networkplumber.org>
>Sent: Sunday, February 27, 2022 8:25 PM
>To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
>Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Slava Ovsiienko
><viacheslavo@nvidia.com>; Matan Azrad <matan@nvidia.com>; Raslan
>Darawsheh <rasland@nvidia.com>; Michael Baum <michaelba@nvidia.com>
>Subject: Re: [PATCH 1/1] doc: add steps to configure VF interface as trusted
>
>On Sun, 27 Feb 2022 18:24:42 +0100
>Thomas Monjalon <thomas@monjalon.net> wrote:
>
>> 27/02/2022 17:44, Stephen Hemminger:
>> > On Sun, 27 Feb 2022 17:46:17 +0200
>> > Asaf Penso <asafp@nvidia.com> wrote:
>> >
>> > > +#. For each VF PCIe, using the following command to bind the driver::
>> > > +
>> > > +   $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind
>> > > \ No newline at end of file
>> >
>> > Please change your editor and/or git settings so there is a new line
>> > at the end of all text files.
>>
>> Isn't it a problem with Eclipse editor?
>
>It is a configuration option in almost all editors:
>
>Eclipse:
>https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
>.newt.com%2Fjava%2Feclipse%2F&amp;data=04%7C01%7Casafp%40nvidia.co
>m%7Cbd5c204df1c9488605ae08d9fa1e6adf%7C43083d15727340c1b7db39efd9
>ccc17a%7C0%7C0%7C637815831547239436%7CUnknown%7CTWFpbGZsb3d8e
>yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>%7C2000&amp;sdata=sutVpcYtoUmUX2TJmK8bWqTss4kefKYcu%2BLYTt%2B%
>2FrDk%3D&amp;reserved=0
>
>Customize code formatting
>Go to Windows > Preferences > Java > Code Style > Code Formatter and click
>on the Edit button.
>
>Check New Lines > Insert new line at end of file
>
>https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fthou
>ghtbot.com%2Fblog%2Fno-newline-at-end-of-
>file&amp;data=04%7C01%7Casafp%40nvidia.com%7Cbd5c204df1c9488605ae0
>8d9fa1e6adf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C63781583
>1547239436%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIj
>oiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=dJuXju04
>tJauNDNRmj%2BXwcXykJQq2MgRbLNNtSaMgM4%3D&amp;reserved=0
>
>Following the rules in your editor
>You can make sure you follow this rule easily:
>
>For Vim users, you're all set out of the box! Just don't change your eol setting.
>For Emacs users, add (setq require-final-newline t) to your .emacs or
>.emacs.d/init.el file.
>For Android Studio, RubyMine, PyCharm, and other IntelliJ, set "Ensure line
>feed at file end on Save" under "Editor."
>For Atom, you're also all set out of the box. Keep that via the Whitespace
>plugin.
>For VS Code, set "files.insertFinalNewline": true.
>For Sublime, set the ensure_newline_at_eof_on_save option to true.
>For TextMate, you can install the Avian Missing Bundle and add
>TM_STRIP_WHITESPACE_ON_SAVE = true to your .tm_properties file.

RE: [PATCH v2] doc: add steps to configure VF interface as trusted

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Asaf Penso <asafp@nvidia.com>
> Sent: Monday, February 28, 2022 9:40 AM
> To: dev@dpdk.org
> Cc: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com>
> Subject: [PATCH v2] doc: add steps to configure VF interface as trusted
> 
> Trusted VF is needed to offload rules with rte_flow to a group that is bigger
> than 0.
> The configuration is done in two parts: driver and FW.
> 
> This patch adds the needed steps to configure a VF to be trusted.
> 
> Signed-off-by: Asaf Penso <asafp@nvidia.com>
> Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

RE: [PATCH v3] doc: add steps to configure VF interface as trusted

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Asaf Penso <asafp@nvidia.com>
> Sent: Monday, February 28, 2022 10:11 AM
> To: dev@dpdk.org
> Cc: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com>
> Subject: [PATCH v3] doc: add steps to configure VF interface as trusted
> 
> Trusted VF is needed to offload rules with rte_flow to a group that is bigger
> than 0.
> The configuration is done in two parts: driver and FW.
> 
> This patch adds the needed steps to configure a VF to be trusted.
> 
> Signed-off-by: Asaf Penso <asafp@nvidia.com>
> Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
> ---
> v3: cleanup the patches
> v2: added new line at the end of the file - comment from Stephen
> Hemminger
> ---

Sending the reply to the appropriate version,

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh