From: Aaron Lu <aaron.lu@intel.com>
To: "ying.huang@intel.com" <ying.huang@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
Waiman Long <longman@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Will Deacon <will@kernel.org>,
Mel Gorman <mgorman@techsingularity.net>,
"kernel test robot" <oliver.sang@intel.com>,
Vlastimil Babka <vbabka@suse.cz>,
Dave Hansen <dave.hansen@linux.intel.com>,
Jesper Dangaard Brouer <brouer@redhat.com>,
Michal Hocko <mhocko@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>, <lkp@lists.01.org>,
kernel test robot <lkp@intel.com>,
Feng Tang <feng.tang@intel.com>,
Zhengjun Xing <zhengjun.xing@linux.intel.com>,
<fengwei.yin@intel.com>
Subject: Re: [mm/page_alloc] f26b3fa046: netperf.Throughput_Mbps -18.0% regression
Date: Thu, 12 May 2022 20:45:45 +0800 [thread overview]
Message-ID: <Yn0BeRaUC9ailyzz@ziqianlu-desk1> (raw)
In-Reply-To: <dfc98503b11e2e54a5a82c21b8ef6afa10eda9b7.camel@intel.com>
On Thu, May 12, 2022 at 11:17:04AM +0800, ying.huang@intel.com wrote:
> On Wed, 2022-05-11 at 19:04 +0800, Aaron Lu wrote:
> > On Wed, May 11, 2022 at 09:58:23AM +0800, ying.huang@intel.com wrote:
> > > On Tue, 2022-05-10 at 11:05 -0700, Linus Torvalds wrote:
> > > > [ Adding locking people in case they have any input ]
> > > >
> > > > On Mon, May 9, 2022 at 11:23 PM ying.huang@intel.com
> > > > <ying.huang@intel.com> wrote:
> > > > >
> > > > > >
> > > > > > Can you point me to the regression report? I would like to take a look,
> > > > > > thanks.
> > > > >
> > > > > https://lore.kernel.org/all/1425108604.10337.84.camel@linux.intel.com/
> > > >
> > > > Hmm.
> > > >
> > > > That explanation looks believable, except that our qspinlocks
> > > > shouldn't be spinning on the lock itself, but spinning on the mcs node
> > > > it inserts into the lock.
> > >
> > > The referenced regression report is very old (in Feb 2015 for 3.16-
> > > 3.17). The ticket spinlock was still used at that time. I believe that
> > > things become much better after we used qspinlock. We can test that.
> >
> > 'will-it-scale/page_fault1 process mode' can greatly stress both zone
> > lock and LRU lock when nr_process = nr_cpu with thp disabled. So I run
> > it to see if it still makes a difference with qspinlock.
> > https://github.com/antonblanchard/will-it-scale/blob/master/tests/page_fault1.c
> >
> > The result on an Icelake 2 sockets server with a total of 48cores/96cpus:
> >
> > tbox_group/testcase/rootfs/kconfig/compiler/nr_task/mode/test/thp_enabled/cpufreq_governor/ucode:
> > lkp-icl-2sp4/will-it-scale/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3/gcc-11/100%/process/page_fault1/never/performance/0xd000331
> >
> > commit:
> > v5.18-rc4
> > 731a704c0d8760cfd641af4bf57167d8c68f9b99
> >
> > v5.18-rc4 731a704c0d8760cfd641af4bf57
> > ---------------- ---------------------------
> > %stddev %change %stddev
> > \ | \
> > 12323894 -26.0% 9125299 will-it-scale.128.processes
> >
> > 22.33 ± 4% -22.3 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.release_pages.tlb_flush_mmu
> > 9.80 -9.2 0.57 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__pagevec_lru_add.folio_add_lru
> > 36.25 +6.7 42.94 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
> > 4.28 ± 10% +34.6 38.93 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages
> > 75.05 +7.8 82.83 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
> >
> > commit 731a704c0d8760cfd641af4bf57 moves zone's lock back to above
> > free_area by reverting commit a368ab67aa55("mm: move zone lock to a
> > different cache line than order-0 free page lists") on top of v5.18-rc4.
> >
> > The interpretation of the above result is: after the revert, performance
> > dropped 26%, zone lock increased 41% from 40% to 81%, the overall lock
> > contention increased 7.8% from 75% to 82.83%. So it appears it still
> > makes a difference with qspinlock.
>
> The performance impact is larger than what I have thought before. The
> test may be too stressing to be realistic? Can you run the test with
> less process number to check?
What about nr_process=16 and 1?
When nr_process=16, zone lock contention increased about 21% from 6% to
27%, performance dropped 17.8%, overall lock contention increased 14.3%:
commit:
v5.18-rc4
731a704c0d8760cfd641af4bf57167d8c68f9b99
v5.18-rc4 731a704c0d8760cfd641af4bf57
---------------- ---------------------------
\ | \
7179264 -17.8% 5898330 will-it-scale.16.processes
4.01 ± 11% -3.5 0.54 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.release_pages.tlb_flush_mmu
4.53 ± 4% -3.0 1.58 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__pagevec_lru_add.folio_add_lru
4.61 ± 13% +10.3 14.90 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
1.58 ± 14% +11.0 12.62 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages
15.42 ± 2% +14.3 29.75 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
When running single process, there is no measuable performance change
and lock contention change.
next prev parent reply other threads:[~2022-05-12 12:46 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-20 1:35 [mm/page_alloc] f26b3fa046: netperf.Throughput_Mbps -18.0% regression kernel test robot
2022-04-29 11:29 ` Aaron Lu
2022-04-29 13:39 ` Mel Gorman
2022-05-05 8:27 ` Aaron Lu
2022-05-05 11:09 ` Mel Gorman
2022-05-05 14:29 ` Aaron Lu
2022-05-06 8:40 ` ying.huang
2022-05-06 12:17 ` Aaron Lu
2022-05-07 0:54 ` ying.huang
2022-05-07 3:27 ` Aaron Lu
2022-05-07 7:11 ` ying.huang
2022-05-07 7:31 ` Aaron Lu
2022-05-07 7:44 ` ying.huang
2022-05-10 3:43 ` Aaron Lu
2022-05-10 6:23 ` ying.huang
2022-05-10 18:05 ` Linus Torvalds
2022-05-10 18:47 ` Waiman Long
2022-05-10 19:03 ` Linus Torvalds
2022-05-10 19:25 ` Linus Torvalds
2022-05-10 19:46 ` Waiman Long
2022-05-10 19:27 ` Peter Zijlstra
2022-05-11 1:58 ` ying.huang
2022-05-11 2:06 ` Waiman Long
2022-05-11 11:04 ` Aaron Lu
2022-05-12 3:17 ` ying.huang
2022-05-12 12:45 ` Aaron Lu [this message]
2022-05-12 17:42 ` Linus Torvalds
2022-05-12 18:06 ` Andrew Morton
2022-05-12 18:49 ` Linus Torvalds
2022-06-14 2:09 ` Feng Tang
2022-05-13 6:19 ` ying.huang
2022-05-11 3:40 ` Aaron Lu
2022-05-11 7:32 ` ying.huang
2022-05-11 7:53 ` Aaron Lu
2022-06-01 2:19 ` Aaron Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yn0BeRaUC9ailyzz@ziqianlu-desk1 \
--to=aaron.lu@intel.com \
--cc=akpm@linux-foundation.org \
--cc=brouer@redhat.com \
--cc=dave.hansen@linux.intel.com \
--cc=feng.tang@intel.com \
--cc=fengwei.yin@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lkp@intel.com \
--cc=lkp@lists.01.org \
--cc=longman@redhat.com \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=oliver.sang@intel.com \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
--cc=will@kernel.org \
--cc=ying.huang@intel.com \
--cc=zhengjun.xing@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).