From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Manev Subject: Re: ixgbe tuning reset when XDP is setup Date: Fri, 15 Dec 2017 17:56:27 +0100 Message-ID: References: <1513333486.28703.9.camel@regit.org> <20171215.105338.2092740911243038177.davem@davemloft.net> Mime-Version: 1.0 (1.0) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: John Fastabend , David Miller , eric@regit.org, Netdev , xdp-newbies@vger.kernel.org, Emil Tantilov To: Alexander Duyck Return-path: Received: from mail-lf0-f52.google.com ([209.85.215.52]:34476 "EHLO mail-lf0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932348AbdLOQ4b (ORCPT ); Fri, 15 Dec 2017 11:56:31 -0500 In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: > On 15 Dec 2017, at 17:51, Alexander Duyck wrot= e: >=20 > On Fri, Dec 15, 2017 at 8:03 AM, John Fastabend > wrote: >> On 12/15/2017 07:53 AM, David Miller wrote: >>> From: Eric Leblond >>> Date: Fri, 15 Dec 2017 11:24:46 +0100 >>>=20 >>>> Hello, >>>>=20 >>>> When using an ixgbe card with Suricata we are using the following >>>> commands to get a symmetric hash on RSS load balancing: >>>>=20 >>>> ./set_irq_affinity 0-15 eth3 >>>> ethtool -X eth3 hkey 6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D= :5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A equal 1= 6 >>>> ethtool -x eth3 >>>> ethtool -n eth3 >>>>=20 >>>> Then we start Suricata. >>>>=20 >>>> In my current experiment on XDP, I have Suricata that inject the eBPF >>>> program when starting. The consequence of that when using an ixgbe card= >>>> is that the load balancing get reset and all interrupts are reaching >>>> the first core. >>>=20 >>> This definitely should _not_ be a side effect of enabling XDP on a devic= e. >>>=20 >>=20 >> Agreed, CC Emil and Alex we should restore these settings after the >> reconfiguration done to support a queue per core. >>=20 >> .John >=20 > So the interrupt configuration has to get reset since we have to > assign 2 Tx queues for every Rx queue instead of the 1-1 that was > previously there. That is a natural consequence of rearranging the > queues as currently happens. The issue is the q_vectors themselves > have to be reallocated. The only way to not make that happen would be > to pre-allocate the Tx queues for XDP always. >=20 > Also just to be clear we are talking about the interrupts being reset, > not the RSS key right? I just want to make sure that is what we are > talking about. >=20 Yes. =46rom the tests we did I only observed the IRQs being all reset to the firs= t CPU after Suricata started. > Thanks. >=20 > - Alex