From: Jan Kara <email@example.com> To: Dave Chinner <firstname.lastname@example.org> Cc: Jan Kara <email@example.com>, Dave Hansen <firstname.lastname@example.org>, Andi Kleen <email@example.com>, firstname.lastname@example.org, Mel Gorman <email@example.com>, "Kirill A. Shutemov" <firstname.lastname@example.org>, email@example.com Subject: Re: [PATCH 0/7 v1] Speed up page cache truncation Date: Thu, 12 Oct 2017 16:07:04 +0200 [thread overview] Message-ID: <20171012140704.GH29293@quack2.suse.cz> (raw) In-Reply-To: <20171011212401.GM15067@dastard> On Thu 12-10-17 08:24:01, Dave Chinner wrote: > On Wed, Oct 11, 2017 at 11:06:13PM +0200, Jan Kara wrote: > > On Wed 11-10-17 10:34:47, Dave Hansen wrote: > > > On 10/11/2017 01:06 AM, Jan Kara wrote: > > > >>> when rebasing our enterprise distro to a newer kernel (from 4.4 to 4.12) we > > > >>> have noticed a regression in bonnie++ benchmark when deleting files. > > > >>> Eventually we have tracked this down to a fact that page cache truncation got > > > >>> slower by about 10%. There were both gains and losses in the above interval of > > > >>> kernels but we have been able to identify that commit 83929372f629 "filemap: > > > >>> prepare find and delete operations for huge pages" caused about 10% regression > > > >>> on its own. > > > >> It's odd that just checking if some pages are huge should be that > > > >> expensive, but ok .. > > > > Yeah, I was surprised as well but profiles were pretty clear on this - part > > > > of the slowdown was caused by loads of page->_compound_head (PageTail() > > > > and page_compound() use that) which we previously didn't have to load at > > > > all, part was in hpage_nr_pages() function and its use. > > > > > > Well, page->_compound_head is part of the same cacheline as the rest of > > > the page, and the page is surely getting touched during truncation at > > > _some_ point. The hpage_nr_pages() might cause the cacheline to get > > > loaded earlier than before, but I can't imagine that it's that expensive. > > > > Then my intuition matches yours ;) but profiles disagree. > > Do you get the same benefit across different filesystems? Mel has answered this already. > > That being said > > I'm not really expert in CPU microoptimizations and profiling so feel free > > to gather perf profiles yourself before and after commit 83929372f629 and > > get better explanation of where the cost is - I would be really curious > > what you come up with because the explanation I have disagrees with my > > intuition as well... > > When I see this sort of stuff my immediate thought is "what is the > change in the icache footprint of the hot codepath"? There's a > few IO benchmarks (e.g. IOZone) that are l1/l2 cache footprint > sensitive on XFS, and can see up to 10% differences in performance > from kernel build to kernel build that have no code changes in the > IO paths or l1/l2 dcache footprint. Yeah, icache footprint could be part of the reason commit 83929372f629 makes things slower but it definitely isn't the only reason. I have experimented with modifications of THP handling so that we can discern normal and THP pages from just looking at page flags (currently we have to look at both page flags and page->_compound_head) and it did bring about half of the regression back. But in the end I've discarded that because those changes were likely to slow down splitting of THPs significantly. WRT build-to-build variance of the benchmark: I saw build-to-build variance in the measured truncate times around 2% on that machine. So it is not negligible but small enough so that I'm confident measured differences are not just a noise... Honza -- Jan Kara <firstname.lastname@example.org> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to email@example.com. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"firstname.lastname@example.org"> email@example.com </a>
prev parent reply other threads:[~2017-10-12 14:07 UTC|newest] Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-10-10 15:19 Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 1/7] mm: Speedup cancel_dirty_page() for clean pages Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 2/7] mm: Refactor truncate_complete_page() Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 3/7] mm: Factor out page cache page freeing into a separate function Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 4/7] mm: Move accounting updates before page_cache_tree_delete() Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 5/7] mm: Move clearing of page->mapping to page_cache_tree_delete() Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 6/7] mm: Factor out checks and accounting from __delete_from_page_cache() Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-10 15:19 ` [PATCH 7/7] mm: Batch radix tree operations when truncating pages Jan Kara 2017-10-10 15:19 ` Jan Kara 2017-10-11 7:39 ` Mel Gorman 2017-10-17 23:05 ` Andrew Morton 2017-10-17 23:05 ` Andrew Morton 2017-10-18 10:44 ` Jan Kara 2017-10-10 17:25 ` [PATCH 0/7 v1] Speed up page cache truncation Andi Kleen 2017-10-10 17:25 ` Andi Kleen 2017-10-11 8:06 ` Jan Kara 2017-10-11 16:51 ` Andi Kleen 2017-10-11 17:34 ` Dave Hansen 2017-10-11 17:59 ` Mel Gorman 2017-10-11 18:37 ` Andi Kleen 2017-10-11 21:06 ` Jan Kara 2017-10-11 21:24 ` Dave Chinner 2017-10-12 9:09 ` Mel Gorman 2017-10-12 14:07 ` Jan Kara [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20171012140704.GH29293@quack2.suse.cz \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH 0/7 v1] Speed up page cache truncation' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.