Re: [linus:master] [iov_iter] c9eec08bac: vm-scalability.throughput -16.9% regression

From: Borislav Petkov <bp@alien8.de>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: David Howells <dhowells@redhat.com>,
	kernel test robot <oliver.sang@intel.com>,
	oe-lkp@lists.linux.dev, lkp@intel.com,
	linux-kernel@vger.kernel.org,
	Christian Brauner <brauner@kernel.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
	Christian Brauner <christian@brauner.io>,
	Matthew Wilcox <willy@infradead.org>,
	David Laight <David.Laight@aculab.com>,
	ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com,
	linux-toolchains ML <linux-toolchains@vger.kernel.org>
Subject: Re: [linus:master] [iov_iter] c9eec08bac: vm-scalability.throughput -16.9% regression
Date: Fri, 17 Nov 2023 12:44:21 +0100	[thread overview]
Message-ID: <20231117114421.GCZVdSFZ7DKtBol821@fat_crate.local> (raw)
In-Reply-To: <CAHk-=wjGQh3ucZFmFR0evbKu2OyEuue-bOjsrnCvxSQdj8x6aw@mail.gmail.com>

Might as well Cc toolchains...

On Thu, Nov 16, 2023 at 11:48:18AM -0500, Linus Torvalds wrote:
> Hmm. I know about the '-mstringop-strategy' flag because of the fairly
> recently discussed bug where gcc would create a byte-by-byte copy in
> some crazy circumstances with the address space attributes:
> 
>     https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111657

I hear those stringop strategy heuristics are interesting. :)

> But I incorrectly thought that "-mstringop-strategy=libcall" would
> then *always* do library calls.

That's how I understood it too. BUT, reportedly, small and known sizes
are still optimized, which is exactly what we want.

> So I decided to test, and that shows that gcc still ends up doing the
> "expand small constant size copies inline" even with that option, and
> doesn't force library calls for those cases.

And you've confirmed it.

> IOW, my assumption was just broken, and using
> "-mstringop-strategy=libcall" may well be the right thing to do.

And here's where I'm wondering whether we should enable it for x86 only
or globally. I think globally because those stringop heuristics happen,
AFAIU, in the general optimization stage and thus target agnostic.

> Of course, it's also possible that with all the function call overhead
> introduced by the CPU mitigations on older CPU's, we should just say
> "rep movsb" is always correct - if you have a new CPU with FSRM it's
> good, and if you have an old CPU it's no worse than the horrendous CPU
> mitigation overhead for function call/returns.

Yeah, I think we should measure the libcall thing and then try to get
the inlined "rep movsb" working and see which one is better. You do have
a point about that RET overhead after each CALL.

> I really hate the mitigations. Oh well.

Tell me about it.

> Ayway, maybe your patch is the RightThing(tm). Or maybe we should use
> 'rep_byte' instead of 'libcall'. Who knows..

Yeah, lemme keep playing with this.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette