From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B6D3C433B4 for ; Fri, 16 Apr 2021 21:08:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CBDBB613CD for ; Fri, 16 Apr 2021 21:08:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBDBB613CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2D61D6B006C; Fri, 16 Apr 2021 17:08:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 286696B006E; Fri, 16 Apr 2021 17:08:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1268D6B0070; Fri, 16 Apr 2021 17:08:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id EB1CB6B006C for ; Fri, 16 Apr 2021 17:08:52 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A332018017367 for ; Fri, 16 Apr 2021 21:08:52 +0000 (UTC) X-FDA: 78039469704.25.D731664 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) by imf07.hostedemail.com (Postfix) with ESMTP id 163A5A000390 for ; Fri, 16 Apr 2021 21:08:51 +0000 (UTC) Received: by mail-il1-f170.google.com with SMTP id p15so13315877iln.3 for ; Fri, 16 Apr 2021 14:08:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=meQ/nFCSVfgCvCBsW0MHfcVkN8/GFWG8nCSaUoG29Is=; b=L78z1fgYxIdSfeO62hbBHSKAd6IxYmu8J2OLxdPY+LL7VODTL71IPzZR+qSw2sIE1D OFE0WTE3Oq//aVzhYG+rlop/1cLBTka5NLfjY0vwdHUqnPXyvfyJd/u+nLDnxjDj+De8 lpEE+qeEfrSsfR+6oeHInCm7DlN8bieT+IBvl7qeP0FaDEFvHgGaE1GzMKPjabB0H+ZA uIN+IyF5NQKxYL2y+bSF1vxE53qpXgIQBilZY7qIqxn1l1rl4+QoeHFY966JSt/Rckat HJaInfFkQHdWI+fTnIwSlagGmWqguwiuih38q4h+2GFni5qTJAIiIDYMDUAU1mAKYYqj bUIQ== X-Gm-Message-State: AOAM530s8jmHeAmzXCREXN1GOPM5PGh7N7y8iIjfpaZCKCqWydZi2+QR 71tbrkFCJLnE++udD/1wTF0= X-Google-Smtp-Source: ABdhPJyW5tsLpLlSyXpC/DzAfK/igHKXqQsZMuWNnkuYj6DLg1K+DquC0pbtEPEgrguhUcYWjNLQyg== X-Received: by 2002:a92:8e4f:: with SMTP id k15mr8309514ilh.16.1618607331695; Fri, 16 Apr 2021 14:08:51 -0700 (PDT) Received: from google.com (243.199.238.35.bc.googleusercontent.com. [35.238.199.243]) by smtp.gmail.com with ESMTPSA id d2sm3236573ilm.7.2021.04.16.14.08.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:08:51 -0700 (PDT) Date: Fri, 16 Apr 2021 21:08:50 +0000 From: Dennis Zhou To: Roman Gushchin Cc: Tejun Heo , Christoph Lameter , Andrew Morton , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 3/6] percpu: make pcpu_nr_empty_pop_pages per chunk type Message-ID: References: <20210408035736.883861-1-guro@fb.com> <20210408035736.883861-4-guro@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210408035736.883861-4-guro@fb.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 163A5A000390 X-Stat-Signature: 38g3cupuximoouo4qibyxw8dk4w6gnb3 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-il1-f170.google.com; client-ip=209.85.166.170 X-HE-DKIM-Result: none/none X-HE-Tag: 1618607331-204712 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Wed, Apr 07, 2021 at 08:57:33PM -0700, Roman Gushchin wrote: > nr_empty_pop_pages is used to guarantee that there are some free > populated pages to satisfy atomic allocations. Accounted and > non-accounted allocations are using separate sets of chunks, > so both need to have a surplus of empty pages. > > This commit makes pcpu_nr_empty_pop_pages and the corresponding logic > per chunk type. > > Signed-off-by: Roman Gushchin > --- > mm/percpu-internal.h | 2 +- > mm/percpu-stats.c | 9 +++++++-- > mm/percpu.c | 14 +++++++------- > 3 files changed, 15 insertions(+), 10 deletions(-) > > diff --git a/mm/percpu-internal.h b/mm/percpu-internal.h > index 18b768ac7dca..095d7eaa0db4 100644 > --- a/mm/percpu-internal.h > +++ b/mm/percpu-internal.h > @@ -87,7 +87,7 @@ extern spinlock_t pcpu_lock; > > extern struct list_head *pcpu_chunk_lists; > extern int pcpu_nr_slots; > -extern int pcpu_nr_empty_pop_pages; > +extern int pcpu_nr_empty_pop_pages[]; > > extern struct pcpu_chunk *pcpu_first_chunk; > extern struct pcpu_chunk *pcpu_reserved_chunk; > diff --git a/mm/percpu-stats.c b/mm/percpu-stats.c > index c8400a2adbc2..f6026dbcdf6b 100644 > --- a/mm/percpu-stats.c > +++ b/mm/percpu-stats.c > @@ -145,6 +145,7 @@ static int percpu_stats_show(struct seq_file *m, void *v) > int slot, max_nr_alloc; > int *buffer; > enum pcpu_chunk_type type; > + int nr_empty_pop_pages; > > alloc_buffer: > spin_lock_irq(&pcpu_lock); > @@ -165,7 +166,11 @@ static int percpu_stats_show(struct seq_file *m, void *v) > goto alloc_buffer; > } > > -#define PL(X) \ > + nr_empty_pop_pages = 0; > + for (type = 0; type < PCPU_NR_CHUNK_TYPES; type++) > + nr_empty_pop_pages += pcpu_nr_empty_pop_pages[type]; > + > +#define PL(X) \ > seq_printf(m, " %-20s: %12lld\n", #X, (long long int)pcpu_stats_ai.X) > > seq_printf(m, > @@ -196,7 +201,7 @@ static int percpu_stats_show(struct seq_file *m, void *v) > PU(nr_max_chunks); > PU(min_alloc_size); > PU(max_alloc_size); > - P("empty_pop_pages", pcpu_nr_empty_pop_pages); > + P("empty_pop_pages", nr_empty_pop_pages); > seq_putc(m, '\n'); > > #undef PU > diff --git a/mm/percpu.c b/mm/percpu.c > index 7e31e1b8725f..61339b3d9337 100644 > --- a/mm/percpu.c > +++ b/mm/percpu.c > @@ -176,10 +176,10 @@ struct list_head *pcpu_chunk_lists __ro_after_init; /* chunk list slots */ > static LIST_HEAD(pcpu_map_extend_chunks); > > /* > - * The number of empty populated pages, protected by pcpu_lock. The > - * reserved chunk doesn't contribute to the count. > + * The number of empty populated pages by chunk type, protected by pcpu_lock. > + * The reserved chunk doesn't contribute to the count. > */ > -int pcpu_nr_empty_pop_pages; > +int pcpu_nr_empty_pop_pages[PCPU_NR_CHUNK_TYPES]; > > /* > * The number of populated pages in use by the allocator, protected by > @@ -559,7 +559,7 @@ static inline void pcpu_update_empty_pages(struct pcpu_chunk *chunk, int nr) > { > chunk->nr_empty_pop_pages += nr; > if (chunk != pcpu_reserved_chunk) > - pcpu_nr_empty_pop_pages += nr; > + pcpu_nr_empty_pop_pages[pcpu_chunk_type(chunk)] += nr; > } > > /* > @@ -1835,7 +1835,7 @@ static void __percpu *pcpu_alloc(size_t size, size_t align, bool reserved, > mutex_unlock(&pcpu_alloc_mutex); > } > > - if (pcpu_nr_empty_pop_pages < PCPU_EMPTY_POP_PAGES_LOW) > + if (pcpu_nr_empty_pop_pages[type] < PCPU_EMPTY_POP_PAGES_LOW) > pcpu_schedule_balance_work(); > > /* clear the areas and return address relative to base address */ > @@ -2013,7 +2013,7 @@ static void pcpu_balance_populated(enum pcpu_chunk_type type) > pcpu_atomic_alloc_failed = false; > } else { > nr_to_pop = clamp(PCPU_EMPTY_POP_PAGES_HIGH - > - pcpu_nr_empty_pop_pages, > + pcpu_nr_empty_pop_pages[type], > 0, PCPU_EMPTY_POP_PAGES_HIGH); > } > > @@ -2595,7 +2595,7 @@ void __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai, > > /* link the first chunk in */ > pcpu_first_chunk = chunk; > - pcpu_nr_empty_pop_pages = pcpu_first_chunk->nr_empty_pop_pages; > + pcpu_nr_empty_pop_pages[PCPU_CHUNK_ROOT] = pcpu_first_chunk->nr_empty_pop_pages; > pcpu_chunk_relocate(pcpu_first_chunk, -1); > > /* include all regions of the first chunk */ > -- > 2.30.2 > This turns out to have been a more pressing issue. Thanks for fixing this. I ran this to Linus for v5.12-rc7 [1]. https://lore.kernel.org/lkml/YHHs618ESvKhYeeM@google.com/ Thanks, Dennis