From: "Paul E. McKenney" <paulmck@kernel.org> To: Stefan Wahren <stefan.wahren@i2se.com> Cc: Marcelo Tosatti <mtosatti@redhat.com>, Andrew Morton <akpm@linux-foundation.org>, Nicolas Saenz Julienne <nsaenzju@redhat.com>, Borislav Petkov <bp@alien8.de>, Minchan Kim <minchan@kernel.org>, Matthew Wilcox <willy@infradead.org>, Mel Gorman <mgorman@techsingularity.net>, Juri Lelli <juri.lelli@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Sebastian Andrzej Siewior <bigeasy@linutronix.de>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linux ARM <linux-arm-kernel@lists.infradead.org>, Phil Elwell <phil@raspberrypi.com>, regressions@lists.linux.dev, riel@surriel.com, viro@zeniv.linux.org.uk Subject: Re: vchiq: Performance regression since 5.18-rc1 Date: Sun, 22 May 2022 21:48:18 -0700 [thread overview] Message-ID: <20220523044818.GS1790663@paulmck-ThinkPad-P17-Gen-1> (raw) In-Reply-To: <bfad3d08-3268-5528-17c1-c17ef3949d96@i2se.com> On Sun, May 22, 2022 at 05:11:36PM +0200, Stefan Wahren wrote: > Hi Paul, > > Am 22.05.22 um 01:46 schrieb Paul E. McKenney: > > On Sun, May 22, 2022 at 01:22:00AM +0200, Stefan Wahren wrote: > > > Hi, > > > > > > while testing the staging/vc04_services/interface/vchiq_arm driver with my > > > Raspberry Pi 3 B+ (multi_v7_defconfig) i noticed a huge performance > > > regression since [ff042f4a9b050895a42cae893cc01fa2ca81b95c] mm: > > > lru_cache_disable: replace work queue synchronization with synchronize_rcu > > > > > > Usually i run "vchiq_test -f 1" to see the driver is still working [1]. > > > > > > Before commit: > > > > > > real 0m1,500s > > > user 0m0,068s > > > sys 0m0,846s > > > > > > After commit: > > > > > > real 7m11,449s > > > user 0m2,049s > > > sys 0m0,023s > > > > > > Best regards > > > > > > [1] - https://github.com/raspberrypi/userland > > Please feel free to try the patch shown below. Or the pair of patches > > from Rik here: > > > > https://lore.kernel.org/lkml/20220218183114.2867528-2-riel@surriel.com/ > > https://lore.kernel.org/lkml/20220218183114.2867528-3-riel@surriel.com/ > > I tried your patch and Rik's patches but in both cases vchiq_test runs 7 > minutes instead of ~ 1 second. That is surprising. Do you boot with rcupdate.rcu_normal=1? That would nullify my patch, but I would expect that Rik's patch would still provide increased performance even in that case. Could you please characterize where the slowdown is occurring? Thanx, Paul > Best regards > > > > > There is work ongoing to produce something better, but ongoing slowly. > > Especially my part of that work. > > > > Thanx, Paul > > > > ------------------------------------------------------------------------ > > > > From paulmck@kernel.org Mon Feb 14 11:05:49 2022 > > Date: Mon, 14 Feb 2022 11:05:49 -0800 > > From: "Paul E. McKenney" <paulmck@kernel.org> > > To: clm@fb.com > > Cc: riel@surriel.com, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, > > linux-fsdevel@vger.kernel.org, kernel-team@fb.com > > Subject: [PATCH RFC fs/namespace] Make kern_unmount() use > > synchronize_rcu_expedited() > > Message-ID: <20220214190549.GA2815154@paulmck-ThinkPad-P17-Gen-1> > > Reply-To: paulmck@kernel.org > > MIME-Version: 1.0 > > Content-Type: text/plain; charset=us-ascii > > Content-Disposition: inline > > Status: RO > > Content-Length: 1036 > > Lines: 32 > > > > Experimental. Not for inclusion. Yet, anyway. > > > > Freeing large numbers of namespaces in quick succession can result in > > a bottleneck on the synchronize_rcu() invoked from kern_unmount(). > > This patch applies the synchronize_rcu_expedited() hammer to allow > > further testing and fault isolation. > > > > Hey, at least there was no need to change the comment! ;-) > > > > Cc: Alexander Viro <viro@zeniv.linux.org.uk> > > Cc: <linux-fsdevel@vger.kernel.org> > > Cc: <linux-kernel@vger.kernel.org> > > Not-yet-signed-off-by: Paul E. McKenney <paulmck@kernel.org> > > > > --- > > > > namespace.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/namespace.c b/fs/namespace.c > > index 40b994a29e90d..79c50ad0ade5b 100644 > > --- a/fs/namespace.c > > +++ b/fs/namespace.c > > @@ -4389,7 +4389,7 @@ void kern_unmount(struct vfsmount *mnt) > > /* release long term mount so mount point can be released */ > > if (!IS_ERR_OR_NULL(mnt)) { > > real_mount(mnt)->mnt_ns = NULL; > > - synchronize_rcu(); /* yecchhh... */ > > + synchronize_rcu_expedited(); /* yecchhh... */ > > mntput(mnt); > > } > > } > >
WARNING: multiple messages have this Message-ID (diff)
From: "Paul E. McKenney" <paulmck@kernel.org> To: Stefan Wahren <stefan.wahren@i2se.com> Cc: Marcelo Tosatti <mtosatti@redhat.com>, Andrew Morton <akpm@linux-foundation.org>, Nicolas Saenz Julienne <nsaenzju@redhat.com>, Borislav Petkov <bp@alien8.de>, Minchan Kim <minchan@kernel.org>, Matthew Wilcox <willy@infradead.org>, Mel Gorman <mgorman@techsingularity.net>, Juri Lelli <juri.lelli@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Sebastian Andrzej Siewior <bigeasy@linutronix.de>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linux ARM <linux-arm-kernel@lists.infradead.org>, Phil Elwell <phil@raspberrypi.com>, regressions@lists.linux.dev, riel@surriel.com, viro@zeniv.linux.org.uk Subject: Re: vchiq: Performance regression since 5.18-rc1 Date: Sun, 22 May 2022 21:48:18 -0700 [thread overview] Message-ID: <20220523044818.GS1790663@paulmck-ThinkPad-P17-Gen-1> (raw) In-Reply-To: <bfad3d08-3268-5528-17c1-c17ef3949d96@i2se.com> On Sun, May 22, 2022 at 05:11:36PM +0200, Stefan Wahren wrote: > Hi Paul, > > Am 22.05.22 um 01:46 schrieb Paul E. McKenney: > > On Sun, May 22, 2022 at 01:22:00AM +0200, Stefan Wahren wrote: > > > Hi, > > > > > > while testing the staging/vc04_services/interface/vchiq_arm driver with my > > > Raspberry Pi 3 B+ (multi_v7_defconfig) i noticed a huge performance > > > regression since [ff042f4a9b050895a42cae893cc01fa2ca81b95c] mm: > > > lru_cache_disable: replace work queue synchronization with synchronize_rcu > > > > > > Usually i run "vchiq_test -f 1" to see the driver is still working [1]. > > > > > > Before commit: > > > > > > real 0m1,500s > > > user 0m0,068s > > > sys 0m0,846s > > > > > > After commit: > > > > > > real 7m11,449s > > > user 0m2,049s > > > sys 0m0,023s > > > > > > Best regards > > > > > > [1] - https://github.com/raspberrypi/userland > > Please feel free to try the patch shown below. Or the pair of patches > > from Rik here: > > > > https://lore.kernel.org/lkml/20220218183114.2867528-2-riel@surriel.com/ > > https://lore.kernel.org/lkml/20220218183114.2867528-3-riel@surriel.com/ > > I tried your patch and Rik's patches but in both cases vchiq_test runs 7 > minutes instead of ~ 1 second. That is surprising. Do you boot with rcupdate.rcu_normal=1? That would nullify my patch, but I would expect that Rik's patch would still provide increased performance even in that case. Could you please characterize where the slowdown is occurring? Thanx, Paul > Best regards > > > > > There is work ongoing to produce something better, but ongoing slowly. > > Especially my part of that work. > > > > Thanx, Paul > > > > ------------------------------------------------------------------------ > > > > From paulmck@kernel.org Mon Feb 14 11:05:49 2022 > > Date: Mon, 14 Feb 2022 11:05:49 -0800 > > From: "Paul E. McKenney" <paulmck@kernel.org> > > To: clm@fb.com > > Cc: riel@surriel.com, viro@zeniv.linux.org.uk, linux-kernel@vger.kernel.org, > > linux-fsdevel@vger.kernel.org, kernel-team@fb.com > > Subject: [PATCH RFC fs/namespace] Make kern_unmount() use > > synchronize_rcu_expedited() > > Message-ID: <20220214190549.GA2815154@paulmck-ThinkPad-P17-Gen-1> > > Reply-To: paulmck@kernel.org > > MIME-Version: 1.0 > > Content-Type: text/plain; charset=us-ascii > > Content-Disposition: inline > > Status: RO > > Content-Length: 1036 > > Lines: 32 > > > > Experimental. Not for inclusion. Yet, anyway. > > > > Freeing large numbers of namespaces in quick succession can result in > > a bottleneck on the synchronize_rcu() invoked from kern_unmount(). > > This patch applies the synchronize_rcu_expedited() hammer to allow > > further testing and fault isolation. > > > > Hey, at least there was no need to change the comment! ;-) > > > > Cc: Alexander Viro <viro@zeniv.linux.org.uk> > > Cc: <linux-fsdevel@vger.kernel.org> > > Cc: <linux-kernel@vger.kernel.org> > > Not-yet-signed-off-by: Paul E. McKenney <paulmck@kernel.org> > > > > --- > > > > namespace.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/namespace.c b/fs/namespace.c > > index 40b994a29e90d..79c50ad0ade5b 100644 > > --- a/fs/namespace.c > > +++ b/fs/namespace.c > > @@ -4389,7 +4389,7 @@ void kern_unmount(struct vfsmount *mnt) > > /* release long term mount so mount point can be released */ > > if (!IS_ERR_OR_NULL(mnt)) { > > real_mount(mnt)->mnt_ns = NULL; > > - synchronize_rcu(); /* yecchhh... */ > > + synchronize_rcu_expedited(); /* yecchhh... */ > > mntput(mnt); > > } > > } > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-05-23 4:48 UTC|newest] Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-05-21 23:22 vchiq: Performance regression since 5.18-rc1 Stefan Wahren 2022-05-21 23:22 ` Stefan Wahren 2022-05-21 23:46 ` Paul E. McKenney 2022-05-21 23:46 ` Paul E. McKenney 2022-05-22 15:11 ` Stefan Wahren 2022-05-22 15:11 ` Stefan Wahren 2022-05-23 4:48 ` Paul E. McKenney [this message] 2022-05-23 4:48 ` Paul E. McKenney 2022-05-23 6:19 ` Stefan Wahren 2022-05-23 6:19 ` Stefan Wahren 2022-05-23 9:29 ` Phil Elwell 2022-05-23 9:29 ` Phil Elwell 2022-05-23 10:48 ` Stefan Wahren 2022-05-23 10:48 ` Stefan Wahren 2022-05-23 11:01 ` Phil Elwell 2022-05-23 11:01 ` Phil Elwell 2022-05-23 11:15 ` Stefan Wahren 2022-05-23 11:15 ` Stefan Wahren 2022-05-23 11:22 ` Phil Elwell 2022-05-23 11:22 ` Phil Elwell 2022-05-23 7:09 ` Sebastian Andrzej Siewior 2022-05-23 7:09 ` Sebastian Andrzej Siewior 2022-05-25 13:56 ` Marcelo Tosatti 2022-05-25 13:56 ` Marcelo Tosatti 2022-05-25 14:07 ` Stefan Wahren 2022-05-25 14:07 ` Stefan Wahren 2022-05-25 14:26 ` Sebastian Andrzej Siewior 2022-05-25 14:26 ` Sebastian Andrzej Siewior 2022-05-25 15:02 ` Paul E. McKenney 2022-05-25 15:02 ` Paul E. McKenney 2022-05-25 15:37 ` Marcelo Tosatti 2022-05-25 15:37 ` Marcelo Tosatti 2022-05-29 22:47 ` Stefan Wahren 2022-05-29 22:47 ` Stefan Wahren 2022-05-30 9:54 ` Stefan Wahren 2022-05-30 9:54 ` Stefan Wahren 2022-06-01 21:02 ` Stefan Wahren 2022-05-23 9:28 ` Thorsten Leemhuis 2022-05-23 9:28 ` Thorsten Leemhuis 2022-07-04 9:48 ` Thorsten Leemhuis
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220523044818.GS1790663@paulmck-ThinkPad-P17-Gen-1 \ --to=paulmck@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=bigeasy@linutronix.de \ --cc=bp@alien8.de \ --cc=juri.lelli@redhat.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@techsingularity.net \ --cc=minchan@kernel.org \ --cc=mtosatti@redhat.com \ --cc=nsaenzju@redhat.com \ --cc=phil@raspberrypi.com \ --cc=regressions@lists.linux.dev \ --cc=riel@surriel.com \ --cc=stefan.wahren@i2se.com \ --cc=tglx@linutronix.de \ --cc=viro@zeniv.linux.org.uk \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.