netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Alexander Duyck <alexander.duyck@gmail.com>
To: Lennart Sorensen <lsorense@csclub.uwaterloo.ca>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Netdev <netdev@vger.kernel.org>,
	intel-wired-lan <intel-wired-lan@lists.osuosl.org>
Subject: Re: [Intel-wired-lan] i40e X722 RSS problem with NAT-Traversal IPsec packets
Date: Fri, 17 May 2019 09:42:19 -0700	[thread overview]
Message-ID: <CAKgT0Ucd0s_0F5_nwqXknRngwROyuecUt+4bYzWvp1-2cNSg7g@mail.gmail.com> (raw)
In-Reply-To: <CAKgT0UfSa-dM2+7xntK9tB7Zw5N8nDd3U1n4OSK0gbWbkNSKJQ@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 8786 bytes --]

On Thu, May 16, 2019 at 4:32 PM Alexander Duyck
<alexander.duyck@gmail.com> wrote:
>
> On Thu, May 16, 2019 at 11:37 AM Lennart Sorensen
> <lsorense@csclub.uwaterloo.ca> wrote:
> >
> > On Thu, May 16, 2019 at 02:34:08PM -0400, Lennart Sorensen wrote:
> > > Here is what I see:
> > >
> > > i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 2.1.7-k
> > > i40e: Copyright (c) 2013 - 2014 Intel Corporation.
> > > i40e 0000:3d:00.0: fw 3.10.52896 api 1.6 nvm 4.00 0x80001577 1.1767.0
> > > i40e 0000:3d:00.0: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.
> > > i40e 0000:3d:00.0: MAC address: a4:bf:01:4e:0c:87
> > > i40e 0000:3d:00.0: flow_type: 63 input_mask:0x0000000000004000
> > > i40e 0000:3d:00.0: flow_type: 46 input_mask:0x0007fff800000000
> > > i40e 0000:3d:00.0: flow_type: 45 input_mask:0x0007fff800000000
> > > i40e 0000:3d:00.0: flow_type: 44 input_mask:0x0007ffff80000000
> > > i40e 0000:3d:00.0: flow_type: 43 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.0: flow_type: 42 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.0: flow_type: 41 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.0: flow_type: 40 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.0: flow_type: 39 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.0: flow_type: 36 input_mask:0x0006060000000000
> > > i40e 0000:3d:00.0: flow_type: 35 input_mask:0x0006060000000000
> > > i40e 0000:3d:00.0: flow_type: 34 input_mask:0x0006060780000000
> > > i40e 0000:3d:00.0: flow_type: 33 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.0: flow_type: 32 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.0: flow_type: 31 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.0: flow_type: 30 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.0: flow_type: 29 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.0: Features: PF-id[0] VSIs: 34 QP: 12 TXQ: 13 RSS VxLAN Geneve VEPA
> > > i40e 0000:3d:00.1: fw 3.10.52896 api 1.6 nvm 4.00 0x80001577 1.1767.0
> > > i40e 0000:3d:00.1: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.
> > > i40e 0000:3d:00.1: MAC address: a4:bf:01:4e:0c:88
> > > i40e 0000:3d:00.1: flow_type: 63 input_mask:0x0000000000004000
> > > i40e 0000:3d:00.1: flow_type: 46 input_mask:0x0007fff800000000
> > > i40e 0000:3d:00.1: flow_type: 45 input_mask:0x0007fff800000000
> > > i40e 0000:3d:00.1: flow_type: 44 input_mask:0x0007ffff80000000
> > > i40e 0000:3d:00.1: flow_type: 43 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.1: flow_type: 42 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.1: flow_type: 41 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.1: flow_type: 40 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.1: flow_type: 39 input_mask:0x0007fffe00000000
> > > i40e 0000:3d:00.1: flow_type: 36 input_mask:0x0006060000000000
> > > i40e 0000:3d:00.1: flow_type: 35 input_mask:0x0006060000000000
> > > i40e 0000:3d:00.1: flow_type: 34 input_mask:0x0006060780000000
> > > i40e 0000:3d:00.1: flow_type: 33 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.1: flow_type: 32 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.1: flow_type: 31 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.1: flow_type: 30 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.1: flow_type: 29 input_mask:0x0006060600000000
> > > i40e 0000:3d:00.1: Features: PF-id[1] VSIs: 34 QP: 12 TXQ: 13 RSS VxLAN Geneve VEPA
> > > i40e 0000:3d:00.1 eth2: NIC Link is Up, 1000 Mbps Full Duplex, Flow Control: None
> > > i40e_ioctl: power down: eth1
> > > i40e_ioctl: power down: eth2
> >
> > Those last two lines is something I added, so ignore those.
>
> No problem.
>
> So just looking at the data provided I am going to guess that IPv6 w/
> UDP likely works without any issues and it is just going to be IPv4
> that is the problem. When you compare the UDP setup from mine versus
> yours it looks like for some reason somebody swapped around the input
> bits for the L3 src and destination fields. I'm basing that on the
> input set masks in the i40e_txrx.h header:
> /* INPUT SET MASK for RSS, flow director, and flexible payload */
> #define I40E_L3_SRC_SHIFT               47
> #define I40E_L3_SRC_MASK                (0x3ULL << I40E_L3_SRC_SHIFT)
> #define I40E_L3_V6_SRC_SHIFT            43
> #define I40E_L3_V6_SRC_MASK             (0xFFULL << I40E_L3_V6_SRC_SHIFT)
> #define I40E_L3_DST_SHIFT               35
> #define I40E_L3_DST_MASK                (0x3ULL << I40E_L3_DST_SHIFT)
> #define I40E_L3_V6_DST_SHIFT            35
> #define I40E_L3_V6_DST_MASK             (0xFFULL << I40E_L3_V6_DST_SHIFT)
> #define I40E_L4_SRC_SHIFT               34
> #define I40E_L4_SRC_MASK                (0x1ULL << I40E_L4_SRC_SHIFT)
> #define I40E_L4_DST_SHIFT               33
> #define I40E_L4_DST_MASK                (0x1ULL << I40E_L4_DST_SHIFT)
> #define I40E_VERIFY_TAG_SHIFT           31
> #define I40E_VERIFY_TAG_MASK            (0x3ULL << I40E_VERIFY_TAG_SHIFT)
>
> The easiest way to verify would be to rewrite the registers for
> flow_type 29, 30, and 31 to match the value that I had shown earlier
> from my dump:
> [  294.687087] i40e 0000:81:00.1: flow_type: 31 input_mask:0x0001801e00000000
>
> I will take a look at putting together a patch that can be tested to
> verify if this is actually the issue tomorrow.
>
> Thanks.
>
> - Alex

So the patch below/attached should resolve the issues you are seeing
with your system in terms of UDPv4 RSS. What you should see with this
patch is the first function to come up will display some "update input
mask" messages, and then the remaining functions shouldn't make any
noise about it since the registers being updated are global to the
device.

If you can test this and see if it resolves the UDPv4 RSS issues I
would appreciate it.

Thanks.

- Alex

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c
b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 65c2b9d2652b..c0a7f66babd9 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -10998,6 +10998,58 @@ static int i40e_pf_config_rss(struct i40e_pf *pf)
                ((u64)i40e_read_rx_ctl(hw, I40E_PFQF_HENA(1)) << 32);
        hena |= i40e_pf_get_default_rss_hena(pf);

+       for (ret = 64; ret--;) {
+               u64 hash_inset_orig, hash_inset_update;
+
+               if (!(hena & (1ull << ret)))
+                       continue;
+
+               /* Read initial input set value for flow type */
+               hash_inset_orig = i40e_read_rx_ctl(hw,
I40E_GLQF_HASH_INSET(1, ret));
+               hash_inset_orig <<= 32;
+               hash_inset_orig |= i40e_read_rx_ctl(hw,
I40E_GLQF_HASH_INSET(0, ret));
+
+               /* Copy value so we can compare later */
+               hash_inset_update = hash_inset_orig;
+
+               /* We should be looking at either the entire IPv6 or IPv4
+                * mask being set. If only part of the IPv6 mask is set, but
+                * the IPv4 mask is not then we have a garbage mask value
+                * and need to reset it.
+                */
+               switch (hash_inset_orig & I40E_L3_V6_SRC_MASK) {
+               case I40E_L3_V6_SRC_MASK:
+               case I40E_L3_SRC_MASK:
+               case 0:
+                       break;
+               default:
+                       hash_inset_update &= ~I40E_L3_V6_SRC_MASK;
+                       hash_inset_update |= I40E_L3_SRC_MASK;
+               }
+
+               switch (hash_inset_orig & I40E_L3_V6_DST_MASK) {
+               case I40E_L3_V6_DST_MASK:
+               case I40E_L3_DST_MASK:
+               case 0:
+                       break;
+               default:
+                       hash_inset_update &= ~I40E_L3_V6_DST_MASK;
+                       hash_inset_update |= I40E_L3_DST_MASK;
+               }
+
+               if (hash_inset_update != hash_inset_orig) {
+                       dev_warn(&pf->pdev->dev,
+                                "flow type: %d update input mask
from:0x%016llx, to:0x%016llx\n",
+                                ret,
+                                hash_inset_orig, hash_inset_update);
+                       i40e_write_rx_ctl(hw, I40E_GLQF_HASH_INSET(0, ret),
+                                         (u32)hash_inset_update);
+                       hash_inset_update >>= 32;
+                       i40e_write_rx_ctl(hw, I40E_GLQF_HASH_INSET(1, ret),
+                                         (u32)hash_inset_update);
+               }
+       }
+
        i40e_write_rx_ctl(hw, I40E_PFQF_HENA(0), (u32)hena);
        i40e_write_rx_ctl(hw, I40E_PFQF_HENA(1), (u32)(hena >> 32));

[-- Attachment #2: i40e-debug-hash-inputs.patch --]
[-- Type: text/x-patch, Size: 2297 bytes --]

i40e: Debug hash inputs

From: Alexander Duyck <alexander.h.duyck@linux.intel.com>


---
 drivers/net/ethernet/intel/i40e/i40e_main.c |   52 +++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 65c2b9d2652b..c0a7f66babd9 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -10998,6 +10998,58 @@ static int i40e_pf_config_rss(struct i40e_pf *pf)
 		((u64)i40e_read_rx_ctl(hw, I40E_PFQF_HENA(1)) << 32);
 	hena |= i40e_pf_get_default_rss_hena(pf);
 
+	for (ret = 64; ret--;) {
+		u64 hash_inset_orig, hash_inset_update;
+
+		if (!(hena & (1ull << ret)))
+			continue;
+
+		/* Read initial input set value for flow type */
+		hash_inset_orig = i40e_read_rx_ctl(hw, I40E_GLQF_HASH_INSET(1, ret));
+		hash_inset_orig <<= 32;
+		hash_inset_orig |= i40e_read_rx_ctl(hw, I40E_GLQF_HASH_INSET(0, ret));
+
+		/* Copy value so we can compare later */
+		hash_inset_update = hash_inset_orig;
+
+		/* We should be looking at either the entire IPv6 or IPv4
+		 * mask being set. If only part of the IPv6 mask is set, but
+		 * the IPv4 mask is not then we have a garbage mask value
+		 * and need to reset it.
+		 */
+		switch (hash_inset_orig & I40E_L3_V6_SRC_MASK) {
+		case I40E_L3_V6_SRC_MASK:
+		case I40E_L3_SRC_MASK:
+		case 0:
+			break;
+		default:
+			hash_inset_update &= ~I40E_L3_V6_SRC_MASK;
+			hash_inset_update |= I40E_L3_SRC_MASK;
+		}
+
+		switch (hash_inset_orig & I40E_L3_V6_DST_MASK) {
+		case I40E_L3_V6_DST_MASK:
+		case I40E_L3_DST_MASK:
+		case 0:
+			break;
+		default:
+			hash_inset_update &= ~I40E_L3_V6_DST_MASK;
+			hash_inset_update |= I40E_L3_DST_MASK;
+		}
+
+		if (hash_inset_update != hash_inset_orig) {
+			dev_warn(&pf->pdev->dev,
+				 "flow type: %d update input mask from:0x%016llx, to:0x%016llx\n",
+				 ret,
+				 hash_inset_orig, hash_inset_update);
+			i40e_write_rx_ctl(hw, I40E_GLQF_HASH_INSET(0, ret),
+					  (u32)hash_inset_update);
+			hash_inset_update >>= 32;
+			i40e_write_rx_ctl(hw, I40E_GLQF_HASH_INSET(1, ret),
+					  (u32)hash_inset_update);
+		}
+	}
+
 	i40e_write_rx_ctl(hw, I40E_PFQF_HENA(0), (u32)hena);
 	i40e_write_rx_ctl(hw, I40E_PFQF_HENA(1), (u32)(hena >> 32));
 

  reply	other threads:[~2019-05-17 16:42 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-01 20:52 i40e X722 RSS problem with NAT-Traversal IPsec packets Lennart Sorensen
2019-05-01 22:52 ` [Intel-wired-lan] " Alexander Duyck
2019-05-02 15:11   ` Lennart Sorensen
2019-05-02 17:03     ` Alexander Duyck
2019-05-02 17:16       ` Lennart Sorensen
2019-05-02 17:28         ` Alexander Duyck
2019-05-02 17:55           ` Lennart Sorensen
2019-05-02 18:52             ` Lennart Sorensen
2019-05-02 20:59               ` Alexander Duyck
2019-05-03 15:14                 ` Lennart Sorensen
2019-05-03 17:19                   ` Alexander Duyck
2019-05-03 20:59                     ` Lennart Sorensen
2019-05-13 16:55                       ` Lennart Sorensen
2019-05-13 19:04                         ` Alexander Duyck
2019-05-14 16:34                           ` Lennart Sorensen
2019-05-16 17:10                             ` Alexander Duyck
2019-05-16 18:34                               ` Lennart Sorensen
2019-05-16 18:37                                 ` Lennart Sorensen
2019-05-16 23:32                                   ` Alexander Duyck
2019-05-17 16:42                                     ` Alexander Duyck [this message]
2019-05-17 17:23                                       ` Lennart Sorensen
2019-05-17 22:20                                         ` Alexander Duyck
2019-05-21 15:15                                           ` Lennart Sorensen
2019-05-21 16:51                                             ` Alexander Duyck
2019-05-21 17:54                                               ` Lennart Sorensen
2019-05-21 23:22                                                 ` Alexander Duyck
2019-05-22 14:39                                                   ` Lennart Sorensen
2019-06-07 14:39                                                     ` Lennart Sorensen
2019-06-07 19:32                                                       ` Alexander Duyck
2019-06-07 20:49                                                         ` [E1000-devel] " Hisashi T Fujinaka
2019-06-07 22:08                                                           ` Fujinaka, Todd
2019-06-10 19:01                                                             ` Lennart Sorensen
2020-02-07 21:51                                                         ` Lennart Sorensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAKgT0Ucd0s_0F5_nwqXknRngwROyuecUt+4bYzWvp1-2cNSg7g@mail.gmail.com \
    --to=alexander.duyck@gmail.com \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=jeffrey.t.kirsher@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lsorense@csclub.uwaterloo.ca \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).