From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93B7CC43141 for ; Thu, 14 Nov 2019 17:12:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5D7982073A for ; Thu, 14 Nov 2019 17:12:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lca.pw header.i=@lca.pw header.b="qRoRkibL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726953AbfKNRMz (ORCPT ); Thu, 14 Nov 2019 12:12:55 -0500 Received: from mail-qv1-f67.google.com ([209.85.219.67]:44983 "EHLO mail-qv1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726482AbfKNRMy (ORCPT ); Thu, 14 Nov 2019 12:12:54 -0500 Received: by mail-qv1-f67.google.com with SMTP id d3so2634137qvs.11 for ; Thu, 14 Nov 2019 09:12:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lca.pw; s=google; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=SB/N8Ptzig01YWuKDW7xcQj/aLQWm5msLsK8SizfSFo=; b=qRoRkibLDkcQeR+vKYb00RQD2hLN4GpsM8H3Rg9diqaMAGdGQG05Fuldi+pIaVRXch 0zcUzLv2py01YRPPrf7a0x2OrRiYQohi1KokhFEdAO61TLNrTLSv7uNsrKwvC0wdS8JN mTrvLFwjZJQ8Pdp0zENa3s7u+CDf1fJ8dx71b/qyG7OFNI6inJK8NbtqYaMRItkOvgWl NjdgT/e+IQOd1DaHss5gzUJLAh1ff4Cg1KjZf5Tp1ag+3ErPKNo20iYeCv4O2tOSnwNq pzSyaEmOyQVTL9coQiTg36LP7ZrEBXMdHhC3k2RDybvQR0CwVtYW6ebrUhYa9KSr9P1R J4nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=SB/N8Ptzig01YWuKDW7xcQj/aLQWm5msLsK8SizfSFo=; b=Mh/PqmFvQ6t1kNIchso/HHLprgNFnkG68GQLdpggdRwbhb8jexYzljSqMOiwUB0v3f Kpf3r2BTtrjQs+QPG23NZyumpU+ospBFPmt6PO5E3l5YmeVDaTeuKPztMtlpRR5byje/ MDgvxyNsLW9hMJ4yeFue1KAt+DKOFXoJBPEgWZweP1/mmrbt0k01f138gT1mtQJuXR8a hs79EP2saZuznt/qRx7Ha+RKI9eqcRVRJNk7coO96YlaWqNLSNo90TQGCMzLWzV/P93o rMdPAPzJQY1vpMaX/B/uNvN5+EaeB3snpyPEABk/8ehtcfjwN4xAKWQpIZV7TJan13m0 bntA== X-Gm-Message-State: APjAAAXFDLQraJnVfAexqO29HSRi70Rc3ZiNpRcS8dR3m5Sp/5c12RPl BG/6rYKRU/T9cPoPUA7JxUfaWYhygRk= X-Google-Smtp-Source: APXvYqzoFtd1AN7Cola7hUsKJ9xrGflS6FS7qygdW7w7JD2mf/QMeEHLk/uebwsvk//zVKmpVvMrsQ== X-Received: by 2002:a0c:c588:: with SMTP id a8mr9137649qvj.9.1573751573612; Thu, 14 Nov 2019 09:12:53 -0800 (PST) Received: from dhcp-41-57.bos.redhat.com (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id n55sm3444361qta.24.2019.11.14.09.12.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Nov 2019 09:12:52 -0800 (PST) Message-ID: <1573751570.5937.122.camel@lca.pw> Subject: Re: [PATCH] net/skbuff: silence warnings under memory pressure From: Qian Cai To: Sergey Senozhatsky , Steven Rostedt , Petr Mladek Cc: Sergey Senozhatsky , Michal Hocko , Eric Dumazet , davem@davemloft.net, netdev@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Thu, 14 Nov 2019 12:12:50 -0500 In-Reply-To: <20190905113208.GA521@jagdpanzerIV> References: <20190903185305.GA14028@dhcp22.suse.cz> <1567546948.5576.68.camel@lca.pw> <20190904061501.GB3838@dhcp22.suse.cz> <20190904064144.GA5487@jagdpanzerIV> <20190904065455.GE3838@dhcp22.suse.cz> <20190904071911.GB11968@jagdpanzerIV> <20190904074312.GA25744@jagdpanzerIV> <1567599263.5576.72.camel@lca.pw> <20190904144850.GA8296@tigerII.localdomain> <1567629737.5576.87.camel@lca.pw> <20190905113208.GA521@jagdpanzerIV> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6 (3.22.6-10.el7) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2019-09-05 at 20:32 +0900, Sergey Senozhatsky wrote: > On (09/04/19 16:42), Qian Cai wrote: > > > Let me think more. > > > > To summary, those look to me are all good long-term improvement that would > > reduce the likelihood of this kind of livelock in general especially for other > > unknown allocations that happen while processing softirqs, but it is still up to > > the air if it fixes it 100% in all situations as printk() is going to take more > > time > > Well. So. I guess that we don't need irq_work most of the time. > > We need to queue irq_work for "safe" wake_up_interruptible(), when we > know that we can deadlock in scheduler. IOW, only when we are invoked > from the scheduler. Scheduler has printk_deferred(), which tells printk() > that it cannot do wake_up_interruptible(). Otherwise we can just use > normal wake_up_process() and don't need that irq_work->wake_up_interruptible() > indirection. The parts of the scheduler, which by mistake call plain printk() > from under pi_lock or rq_lock have chances to deadlock anyway and should > be switched to printk_deferred(). > > I think we can queue significantly much less irq_work-s from printk(). > > Petr, Steven, what do you think? Sergey, do you still plan to get this patch merged? > > Something like this. Call wake_up_interruptible(), switch to > wake_up_klogd() only when called from sched code. > > --- > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c > index cd51aa7d08a9..89cb47882254 100644 > --- a/kernel/printk/printk.c > +++ b/kernel/printk/printk.c > @@ -2027,8 +2027,11 @@ asmlinkage int vprintk_emit(int facility, int level, > pending_output = (curr_log_seq != log_next_seq); > logbuf_unlock_irqrestore(flags); > > + if (!pending_output) > + return printed_len; > + > /* If called from the scheduler, we can not call up(). */ > - if (!in_sched && pending_output) { > + if (!in_sched) { > /* > * Disable preemption to avoid being preempted while holding > * console_sem which would prevent anyone from printing to > @@ -2043,10 +2046,11 @@ asmlinkage int vprintk_emit(int facility, int level, > if (console_trylock_spinning()) > console_unlock(); > preempt_enable(); > - } > > - if (pending_output) > + wake_up_interruptible(&log_wait); > + } else { > wake_up_klogd(); > + } > return printed_len; > } > EXPORT_SYMBOL(vprintk_emit); > --- > > > and could deal with console hardware that involve irq_exit() anyway. > > printk->console_driver->write() does not involve irq. > > > On the other hand, adding __GPF_NOWARN in the build_skb() allocation will fix > > this known NET_TX_SOFTIRQ case which is common when softirqd involved at least > > in short-term. It even have a benefit to reduce the overall warn_alloc() noise > > out there. > > That's not up to me to decide. > > -ss From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36D89C432C3 for ; Thu, 14 Nov 2019 17:12:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DB8E720724 for ; Thu, 14 Nov 2019 17:12:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lca.pw header.i=@lca.pw header.b="qRoRkibL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB8E720724 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lca.pw Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8EF016B0006; Thu, 14 Nov 2019 12:12:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89F1D6B0008; Thu, 14 Nov 2019 12:12:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B57A6B000A; Thu, 14 Nov 2019 12:12:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 666666B0006 for ; Thu, 14 Nov 2019 12:12:55 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 2805A83E4 for ; Thu, 14 Nov 2019 17:12:55 +0000 (UTC) X-FDA: 76155527910.01.spark58_539a267b4871e X-HE-Tag: spark58_539a267b4871e X-Filterd-Recvd-Size: 6482 Received: from mail-qv1-f65.google.com (mail-qv1-f65.google.com [209.85.219.65]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Thu, 14 Nov 2019 17:12:54 +0000 (UTC) Received: by mail-qv1-f65.google.com with SMTP id v16so2644854qvq.6 for ; Thu, 14 Nov 2019 09:12:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lca.pw; s=google; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=SB/N8Ptzig01YWuKDW7xcQj/aLQWm5msLsK8SizfSFo=; b=qRoRkibLDkcQeR+vKYb00RQD2hLN4GpsM8H3Rg9diqaMAGdGQG05Fuldi+pIaVRXch 0zcUzLv2py01YRPPrf7a0x2OrRiYQohi1KokhFEdAO61TLNrTLSv7uNsrKwvC0wdS8JN mTrvLFwjZJQ8Pdp0zENa3s7u+CDf1fJ8dx71b/qyG7OFNI6inJK8NbtqYaMRItkOvgWl NjdgT/e+IQOd1DaHss5gzUJLAh1ff4Cg1KjZf5Tp1ag+3ErPKNo20iYeCv4O2tOSnwNq pzSyaEmOyQVTL9coQiTg36LP7ZrEBXMdHhC3k2RDybvQR0CwVtYW6ebrUhYa9KSr9P1R J4nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=SB/N8Ptzig01YWuKDW7xcQj/aLQWm5msLsK8SizfSFo=; b=XTsR3yR470YL79m3VeRNTqDnV7dAbsD8FPvLhexBh6NwlwMhDL03JMHyB+OhfpyXk1 HS4jw0SPaImDscHDMPbxwLyf0UfJk+2pg6YK5fK9bJqreVKLsrXM3f4mhKQSnoRFelEj aYrYC2BYX/mH7d6/EYAmwoK1MLEg7JKNenoqaQvnS5PEhN41pkSfaNsIctviJUj09EeS aONFeWNsllGhdJ8E0oSwsKhXJmSzZ9rFmGHRFFJwAJDDeE3vWbh4aKP2gCW1tgWz1IE9 WmTHASmRMJzgw0JTvSrLL7cYCPdM895hR/gXtt4qpfSQabrh7JjCMPSPrq8t/ZdZv3s/ 43RQ== X-Gm-Message-State: APjAAAWARyHqC5ZJqKbpKPagk0WLRojly9kcwRP9e6aA6anzCP4xUgTV e+453bCPXue0wqX9cCTB/OmaNQ== X-Google-Smtp-Source: APXvYqzoFtd1AN7Cola7hUsKJ9xrGflS6FS7qygdW7w7JD2mf/QMeEHLk/uebwsvk//zVKmpVvMrsQ== X-Received: by 2002:a0c:c588:: with SMTP id a8mr9137649qvj.9.1573751573612; Thu, 14 Nov 2019 09:12:53 -0800 (PST) Received: from dhcp-41-57.bos.redhat.com (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id n55sm3444361qta.24.2019.11.14.09.12.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Nov 2019 09:12:52 -0800 (PST) Message-ID: <1573751570.5937.122.camel@lca.pw> Subject: Re: [PATCH] net/skbuff: silence warnings under memory pressure From: Qian Cai To: Sergey Senozhatsky , Steven Rostedt , Petr Mladek Cc: Sergey Senozhatsky , Michal Hocko , Eric Dumazet , davem@davemloft.net, netdev@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Thu, 14 Nov 2019 12:12:50 -0500 In-Reply-To: <20190905113208.GA521@jagdpanzerIV> References: <20190903185305.GA14028@dhcp22.suse.cz> <1567546948.5576.68.camel@lca.pw> <20190904061501.GB3838@dhcp22.suse.cz> <20190904064144.GA5487@jagdpanzerIV> <20190904065455.GE3838@dhcp22.suse.cz> <20190904071911.GB11968@jagdpanzerIV> <20190904074312.GA25744@jagdpanzerIV> <1567599263.5576.72.camel@lca.pw> <20190904144850.GA8296@tigerII.localdomain> <1567629737.5576.87.camel@lca.pw> <20190905113208.GA521@jagdpanzerIV> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6 (3.22.6-10.el7) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 2019-09-05 at 20:32 +0900, Sergey Senozhatsky wrote: > On (09/04/19 16:42), Qian Cai wrote: > > > Let me think more. > > > > To summary, those look to me are all good long-term improvement that would > > reduce the likelihood of this kind of livelock in general especially for other > > unknown allocations that happen while processing softirqs, but it is still up to > > the air if it fixes it 100% in all situations as printk() is going to take more > > time > > Well. So. I guess that we don't need irq_work most of the time. > > We need to queue irq_work for "safe" wake_up_interruptible(), when we > know that we can deadlock in scheduler. IOW, only when we are invoked > from the scheduler. Scheduler has printk_deferred(), which tells printk() > that it cannot do wake_up_interruptible(). Otherwise we can just use > normal wake_up_process() and don't need that irq_work->wake_up_interruptible() > indirection. The parts of the scheduler, which by mistake call plain printk() > from under pi_lock or rq_lock have chances to deadlock anyway and should > be switched to printk_deferred(). > > I think we can queue significantly much less irq_work-s from printk(). > > Petr, Steven, what do you think? Sergey, do you still plan to get this patch merged? > > Something like this. Call wake_up_interruptible(), switch to > wake_up_klogd() only when called from sched code. > > --- > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c > index cd51aa7d08a9..89cb47882254 100644 > --- a/kernel/printk/printk.c > +++ b/kernel/printk/printk.c > @@ -2027,8 +2027,11 @@ asmlinkage int vprintk_emit(int facility, int level, > pending_output = (curr_log_seq != log_next_seq); > logbuf_unlock_irqrestore(flags); > > + if (!pending_output) > + return printed_len; > + > /* If called from the scheduler, we can not call up(). */ > - if (!in_sched && pending_output) { > + if (!in_sched) { > /* > * Disable preemption to avoid being preempted while holding > * console_sem which would prevent anyone from printing to > @@ -2043,10 +2046,11 @@ asmlinkage int vprintk_emit(int facility, int level, > if (console_trylock_spinning()) > console_unlock(); > preempt_enable(); > - } > > - if (pending_output) > + wake_up_interruptible(&log_wait); > + } else { > wake_up_klogd(); > + } > return printed_len; > } > EXPORT_SYMBOL(vprintk_emit); > --- > > > and could deal with console hardware that involve irq_exit() anyway. > > printk->console_driver->write() does not involve irq. > > > On the other hand, adding __GPF_NOWARN in the build_skb() allocation will fix > > this known NET_TX_SOFTIRQ case which is common when softirqd involved at least > > in short-term. It even have a benefit to reduce the overall warn_alloc() noise > > out there. > > That's not up to me to decide. > > -ss