From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S934069AbeAXOyU (ORCPT <rfc822;w@1wt.eu>);
        Wed, 24 Jan 2018 09:54:20 -0500
Received: from mx1.redhat.com ([209.132.183.28]:43868 "EHLO mx1.redhat.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S933763AbeAXOyS (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 24 Jan 2018 09:54:18 -0500
Message-ID: <1516805645.2476.23.camel@redhat.com>
Subject: Re: [RFC PATCH 0/4] softirq: Per vector threading v3
From: Paolo Abeni <pabeni@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: David Miller <davem@davemloft.net>,
        Frederic Weisbecker <frederic@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Sasha Levin <alexander.levin@verizon.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Mauro Carvalho Chehab <mchehab@s-opensource.com>,
        Hannes Frederic Sowa <hannes@stressinduktion.org>,
        Paul McKenney <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <wanpeng.li@hotmail.com>, Dmitry Safonov <dima@arista.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        Radu Rendec <rrendec@arista.com>, Ingo Molnar <mingo@kernel.org>,
        Stanislaw Gruszka <sgruszka@redhat.com>,
        Rik van Riel <riel@redhat.com>, Eric Dumazet <edumazet@google.com>,
        Niklas Cassel <nks.gnu@gmail.com>
Date: Wed, 24 Jan 2018 15:54:05 +0100
In-Reply-To: <CA+55aFwEMe=jV7pvV2dWFKrQ--mAw3buXhdJ-Qk2u-290rTQQQ@mail.gmail.com>
References: <1516376774-24076-1-git-send-email-frederic@kernel.org>
         <1516702432.2554.37.camel@redhat.com>
         <20180123.112201.1263563609292212852.davem@davemloft.net>
         <1516726652.2554.58.camel@redhat.com>
         <CA+55aFwEMe=jV7pvV2dWFKrQ--mAw3buXhdJ-Qk2u-290rTQQQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, 2018-01-23 at 09:42 -0800, Linus Torvalds wrote:
> On Tue, Jan 23, 2018 at 8:57 AM, Paolo Abeni <pabeni@redhat.com> wrote:
> > 
> > > Or is it that the workqueue execution is simply not yielding for some
> > > reason?
> > 
> > It's like that.
> > 
> > I spent little time on it, so I haven't many data point. I'll try to
> > investigate the scenario later this week.
> 
> Hmm. workqueues seem to use cond_resched_rcu_qs(), which does a
> cond_resched() (and a RCU quiescent note).
> 
> But I wonder if the test triggers the "lets run lots of workqueue
> threads", and then the single-threaded user space just gets blown out
> of the water by many kernel threads. Each thread gets its own "fair"
> amount of CPU, but..

Niklas suggested a possible relation with CONFIG_IRQ_TIME_ACCOUNTING=y
and indeed he was right.

The patched kernel under test had CONFIG_IRQ_TIME_ACCOUNTING set, and
very little CPU time was accounted to the kworker:

[2125 is the relevant kworker's pid]
grep sum_exec_runtime /proc/2125/sched; sleep 10; grep sum_exec_runtime /proc/2125/sched
se.sum_exec_runtime                          :         13408.239286
se.sum_exec_runtime                          :         13456.907197

despite such process was processing a lot of packets and basically
burning a CPU.

Switching CONFIG_IRQ_TIME_ACCOUNTING off I see the expected behaviour:
top reports that the user space process and kworker share the CPU
almost fairly and the user space process is able to receive a
reasonable amount of packets.

Paolo