From: Mel Gorman <mgorman@suse.de>
To: Nadav Amit <nadav.amit@gmail.com>
Cc: "Huang, Ying" <ying.huang@intel.com>,
Linux-MM <linux-mm@kvack.org>,
Andrew Morton <akpm@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Peter Xu <peterx@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Vlastimil Babka <vbabka@suse.cz>,
Matthew Wilcox <willy@infradead.org>,
Will Deacon <will@kernel.org>,
Michel Lespinasse <walken@google.com>,
Arjun Roy <arjunroy@google.com>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [RFC] NUMA balancing: reduce TLB flush via delaying mapping on hint page fault
Date: Thu, 1 Apr 2021 09:38:09 +0100 [thread overview]
Message-ID: <20210401083809.GX15768@suse.de> (raw)
In-Reply-To: <C42F16F8-646B-448D-A098-E1A98637E28D@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2045 bytes --]
On Wed, Mar 31, 2021 at 09:36:04AM -0700, Nadav Amit wrote:
>
>
> > On Mar 31, 2021, at 6:16 AM, Mel Gorman <mgorman@suse.de> wrote:
> >
> > On Wed, Mar 31, 2021 at 07:20:09PM +0800, Huang, Ying wrote:
> >> Mel Gorman <mgorman@suse.de> writes:
> >>
> >>> On Mon, Mar 29, 2021 at 02:26:51PM +0800, Huang Ying wrote:
> >>>> For NUMA balancing, in hint page fault handler, the faulting page will
> >>>> be migrated to the accessing node if necessary. During the migration,
> >>>> TLB will be shot down on all CPUs that the process has run on
> >>>> recently. Because in the hint page fault handler, the PTE will be
> >>>> made accessible before the migration is tried. The overhead of TLB
> >>>> shooting down is high, so it's better to be avoided if possible. In
> >>>> fact, if we delay mapping the page in PTE until migration, that can be
> >>>> avoided. This is what this patch doing.
> >>>>
> >>>
> >>> Why would the overhead be high? It was previously inaccessibly so it's
> >>> only parallel accesses making forward progress that trigger the need
> >>> for a flush.
> >>
> >> Sorry, I don't understand this. Although the page is inaccessible, the
> >> threads may access other pages, so TLB flushing is still necessary.
> >>
> >
> > You assert the overhead of TLB shootdown is high and yes, it can be
> > very high but you also said "the benchmark score has no visible changes"
> > indicating the TLB shootdown cost is not a major problem for the workload.
> > It does not mean we should ignore it though.
>
> If you are looking for a benchmark that is negatively affected by NUMA
> balancing, then IIRC Parsec???s dedup is such a workload. [1]
>
Few questions;
Is Parsec imparied due to NUMA balancing in general or due to TLB
shootdowns specifically?
Are you using "gcc-pthreads" for parallelisation and the "native" size
for Parsec?
Is there any specific thread count that matters either in
absolute terms or as a precentage of online CPUs?
--
Mel Gorman
SUSE Labs
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2021-04-01 8:38 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-29 6:26 [RFC] NUMA balancing: reduce TLB flush via delaying mapping on hint page fault Huang Ying
2021-03-30 13:33 ` Mel Gorman
2021-03-31 11:20 ` Huang, Ying
2021-03-31 13:16 ` Mel Gorman
2021-03-31 16:36 ` Nadav Amit
2021-04-01 8:38 ` Mel Gorman [this message]
2021-04-01 19:21 ` Nadav Amit
2021-04-01 0:52 ` Huang, Ying
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210401083809.GX15768@suse.de \
--to=mgorman@suse.de \
--cc=akpm@linux-foundation.org \
--cc=arjunroy@google.com \
--cc=hannes@cmpxchg.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nadav.amit@gmail.com \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=vbabka@suse.cz \
--cc=walken@google.com \
--cc=will@kernel.org \
--cc=willy@infradead.org \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).