From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753651AbcEMRTJ (ORCPT ); Fri, 13 May 2016 13:19:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52324 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752063AbcEMRTG (ORCPT ); Fri, 13 May 2016 13:19:06 -0400 Message-ID: <1463159940.11721.37.camel@redhat.com> Subject: Re: [RFC PATCH 0/2] net: threadable napi poll loop From: Paolo Abeni To: Eric Dumazet Cc: Eric Dumazet , Peter Zijlstra , Hannes Frederic Sowa , netdev , "David S. Miller" , Jiri Pirko , Daniel Borkmann , Alexei Starovoitov , Alexander Duyck , Tom Herbert , Ingo Molnar , Rik van Riel , LKML , "Paul E. McKenney" Date: Fri, 13 May 2016 19:19:00 +0200 In-Reply-To: References: <1462890590.23934.68.camel@edumazet-glaptop3.roam.corp.google.com> <90f3db8c-c30c-b204-576a-454939ac93ce@stressinduktion.org> <94f323a9-515e-4d75-cac8-ef0214f0499e@stressinduktion.org> <1462920697.23934.113.camel@edumazet-glaptop3.roam.corp.google.com> <20160511065527.GD3193@twins.programming.kicks-ass.net> <1463003804.23934.154.camel@edumazet-glaptop3.roam.corp.google.com> <1463158203.11721.25.camel@redhat.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 13 May 2016 17:19:05 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2016-05-13 at 10:03 -0700, Eric Dumazet wrote: > On Fri, May 13, 2016 at 9:50 AM, Paolo Abeni wrote: > > >> Indeed, and the patch looks quite simple now ;) > >> > >> diff --git a/kernel/softirq.c b/kernel/softirq.c > >> index 17caf4b63342d7839528f367b283a386413b0362..23c364485d03618773c385d943c0ef39f5931d09 100644 > >> --- a/kernel/softirq.c > >> +++ b/kernel/softirq.c > >> @@ -57,6 +57,11 @@ static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp > >> > >> DEFINE_PER_CPU(struct task_struct *, ksoftirqd); > >> > >> +static inline bool ksoftirqd_running(void) > >> +{ > >> + return __this_cpu_read(ksoftirqd)->state == TASK_RUNNING; > >> +} > >> + > >> const char * const softirq_to_name[NR_SOFTIRQS] = { > >> "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL", > >> "TASKLET", "SCHED", "HRTIMER", "RCU" > >> @@ -313,7 +318,7 @@ asmlinkage __visible void do_softirq(void) > >> > >> pending = local_softirq_pending(); > >> > >> - if (pending) > >> + if (pending && !ksoftirqd_running()) > >> do_softirq_own_stack(); > >> > >> local_irq_restore(flags); > >> @@ -340,6 +345,9 @@ void irq_enter(void) > >> > >> static inline void invoke_softirq(void) > >> { > >> + if (ksoftirqd_running()) > >> + return; > >> + > >> if (!force_irqthreads) { > >> #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK > >> /* > > > > In this version of the path, the chunk affecting __local_bh_enable_ip() > > has been removed. > > > > I think it is beneficial, because it allows avoiding a > > local_irq_save()/local_irq_restore() pairs per local_bh_enable under heavy load. > > > > Interesting, do you have any numbers ? The difference is small, in the noise range: [with this patch applied] super_netperf 100 -H 192.168.122.1 -t UDP_STREAM -l 60 -- -m 1 9.00 [adding the test into __local_bh_enable_ip(), too] super_netperf 100 -H 192.168.122.1 -t UDP_STREAM -l 60 -- -m 1 9.14 but reproducible, in my experiments. I have similar data for different number of flows. > I believe I did this so that we factorize the logic in do_softirq() > and keep the code local to kernel/softirq.c > > Otherwise, netif_rx_ni() could also process softirq while ksoftirqd > was scheduled, > so I would have to 'export' the ksoftirqd_running(void) helper in an > include file. The idea could be to add the test in __local_bh_enable_ip(), maintaining the test also in do_softirq() (as currently done, i.e for local_softirq_pending()) Cheers, Paolo