From: "Paul E. McKenney" <paulmck@kernel.org>
To: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Matthew Wilcox <willy@infradead.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Al Viro <viro@kernel.org>, Luis Chamberlain <mcgrof@kernel.org>,
lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org,
linux-mm <linux-mm@kvack.org>,
Daniel Gomez <da.gomez@samsung.com>,
Pankaj Raghav <p.raghav@samsung.com>,
Jens Axboe <axboe@kernel.dk>, Dave Chinner <david@fromorbit.com>,
Christoph Hellwig <hch@lst.de>, Chris Mason <clm@fb.com>,
Johannes Weiner <hannes@cmpxchg.org>
Subject: Re: [LSF/MM/BPF TOPIC] Measuring limits and enhancing buffered IO
Date: Mon, 26 Feb 2024 21:17:41 -0800 [thread overview]
Message-ID: <df68c44e-1ab3-485d-a0d6-0c37a06ab4ff@paulmck-laptop> (raw)
In-Reply-To: <ldpltrnfmf4a3xs43hfjnhrfidrbd7t5k6i5i3ysuzken2zeql@wm2ivk45hitj>
On Mon, Feb 26, 2024 at 08:08:17PM -0500, Kent Overstreet wrote:
> On Mon, Feb 26, 2024 at 04:55:29PM -0800, Paul E. McKenney wrote:
> > On Mon, Feb 26, 2024 at 07:29:04PM -0500, Kent Overstreet wrote:
> > > On Mon, Feb 26, 2024 at 04:05:37PM -0800, Paul E. McKenney wrote:
> > > > On Mon, Feb 26, 2024 at 06:29:43PM -0500, Kent Overstreet wrote:
> > > > > Well, we won't want it getting hammered on continuously - we should be
> > > > > able to tune reclaim so that doesn't happen.
> > > > >
> > > > > I think getting numbers on the amount of memory stranded waiting for RCU
> > > > > is probably first order of business - minor tweak to kfree_rcu() et all
> > > > > for that; there's APIs they can query to maintain that counter.
> > > >
> > > > We can easily tell you the number of blocks of memory waiting to be freed.
> > > > But RCU does not know their size. Yes, we could ferret this on each
> > > > call to kmem_free_rcu(), but that might not be great for performance.
> > > > We could traverse the lists at runtime, but such traversal must be done
> > > > with interrupts disabled, which is also not great.
> > > >
> > > > > then, we can add a heuristic threshhold somewhere, something like
> > > > >
> > > > > if (rcu_stranded * multiplier > reclaimable_memory)
> > > > > kick_rcu()
> > > >
> > > > If it is a heuristic anyway, it sounds best to base the heuristic on
> > > > the number of objects rather than their aggregate size.
> > >
> > > I don't think that'll really work given that object size can very from <
> > > 100 bytes all the way up to 2MB hugepages. The shrinker API works that
> > > way and I positively hate it; it's really helpful for introspection and
> > > debugability later to give good human understandable units to this
> > > stuff.
> >
> > You might well be right, but let's please try it before adding overhead to
> > kfree_rcu() and friends. I bet it will prove to be good and sufficient.
> >
> > > And __ksize() is pretty cheap, and I think there might be room in struct
> > > slab to stick the object size there instead of getting it from the slab
> > > cache - and folio_size() is cheaper still.
> >
> > On __ksize():
> >
> > * This should only be used internally to query the true size of allocations.
> > * It is not meant to be a way to discover the usable size of an allocation
> > * after the fact. Instead, use kmalloc_size_roundup().
> >
> > Except that kmalloc_size_roundup() doesn't look like it is meant for
> > this use case. On __ksize() being used only internally, I would not be
> > at all averse to kfree_rcu() and friends moving to mm.
>
> __ksize() is the right helper to use for this; ksize() is "how much
> usable memory", __ksize() is "how much does this occupy".
>
> > The idea is for kfree_rcu() to invoke __ksize() when given slab memory
> > and folio_size() when given vmalloc() memory?
>
> __ksize() for slab memory, but folio_size() would be for page
> allocations - actually, I think compound_order() is more appropriate
> here, but that's willy's area. IOW, for free_pages_rcu(), which AFAIK we
> don't have yet but it looks like we're going to need.
>
> I'm scanning through vmalloc.c and I don't think we have a helper yet to
> query the allocation size - I can write one tomorrow, giving my brain a
> rest today :)
Again, let's give the straight count of blocks a try first. I do see
that you feel that the added overhead is negligible, but zero added
overhead is even better.
Thanx, Paul
next prev parent reply other threads:[~2024-02-27 5:17 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-23 23:59 [LSF/MM/BPF TOPIC] Measuring limits and enhancing buffered IO Luis Chamberlain
2024-02-24 4:12 ` Matthew Wilcox
2024-02-24 17:31 ` Linus Torvalds
2024-02-24 18:13 ` Matthew Wilcox
2024-02-24 18:24 ` Linus Torvalds
2024-02-24 18:20 ` Linus Torvalds
2024-02-24 19:11 ` Linus Torvalds
2024-02-24 21:42 ` Theodore Ts'o
2024-02-24 22:57 ` Chris Mason
2024-02-24 23:40 ` Linus Torvalds
2024-05-10 23:57 ` Luis Chamberlain
2024-02-25 5:18 ` Kent Overstreet
2024-02-25 6:04 ` Kent Overstreet
2024-02-25 13:10 ` Matthew Wilcox
2024-02-25 17:03 ` Linus Torvalds
2024-02-25 21:14 ` Matthew Wilcox
2024-02-25 23:45 ` Linus Torvalds
2024-02-26 1:02 ` Kent Overstreet
2024-02-26 1:32 ` Linus Torvalds
2024-02-26 1:58 ` Kent Overstreet
2024-02-26 2:06 ` Kent Overstreet
2024-02-26 2:34 ` Linus Torvalds
2024-02-26 2:50 ` Al Viro
2024-02-26 17:17 ` Linus Torvalds
2024-02-26 21:07 ` Matthew Wilcox
2024-02-26 21:17 ` Kent Overstreet
2024-02-26 21:19 ` Kent Overstreet
2024-02-26 21:55 ` Paul E. McKenney
2024-02-26 23:29 ` Kent Overstreet
2024-02-27 0:05 ` Paul E. McKenney
2024-02-27 0:29 ` Kent Overstreet
2024-02-27 0:55 ` Paul E. McKenney
2024-02-27 1:08 ` Kent Overstreet
2024-02-27 5:17 ` Paul E. McKenney [this message]
2024-02-27 6:21 ` Kent Overstreet
2024-02-27 15:32 ` Paul E. McKenney
2024-02-27 15:52 ` Kent Overstreet
2024-02-27 16:06 ` Paul E. McKenney
2024-02-27 15:54 ` Matthew Wilcox
2024-02-27 16:21 ` Paul E. McKenney
2024-02-27 16:34 ` Kent Overstreet
2024-02-27 17:58 ` Paul E. McKenney
2024-02-28 23:55 ` Kent Overstreet
2024-02-29 19:42 ` Paul E. McKenney
2024-02-29 20:51 ` Kent Overstreet
2024-03-05 2:19 ` Paul E. McKenney
2024-02-27 0:43 ` Dave Chinner
2024-02-26 22:46 ` Linus Torvalds
2024-02-26 23:48 ` Linus Torvalds
2024-02-27 7:21 ` Kent Overstreet
2024-02-27 15:39 ` Matthew Wilcox
2024-02-27 15:54 ` Kent Overstreet
2024-02-27 16:34 ` Linus Torvalds
2024-02-27 16:47 ` Kent Overstreet
2024-02-27 17:07 ` Linus Torvalds
2024-02-27 17:20 ` Kent Overstreet
2024-02-27 18:02 ` Linus Torvalds
2024-05-14 11:52 ` Luis Chamberlain
2024-05-14 16:04 ` Linus Torvalds
2024-02-25 21:29 ` Kent Overstreet
2024-02-25 17:32 ` Kent Overstreet
2024-02-24 17:55 ` Luis Chamberlain
2024-02-25 5:24 ` Kent Overstreet
2024-02-26 12:22 ` Dave Chinner
2024-02-27 10:07 ` Kent Overstreet
2024-02-27 14:08 ` Luis Chamberlain
2024-02-27 14:57 ` Kent Overstreet
2024-02-27 22:13 ` Dave Chinner
2024-02-27 22:21 ` Kent Overstreet
2024-02-27 22:42 ` Dave Chinner
2024-02-28 7:48 ` [Lsf-pc] " Amir Goldstein
2024-02-28 14:01 ` Chris Mason
2024-02-29 0:25 ` Dave Chinner
2024-02-29 0:57 ` Kent Overstreet
2024-03-04 0:46 ` Dave Chinner
2024-02-27 22:46 ` Linus Torvalds
2024-02-27 23:00 ` Linus Torvalds
2024-02-28 2:22 ` Kent Overstreet
2024-02-28 3:00 ` Matthew Wilcox
2024-02-28 4:22 ` Matthew Wilcox
2024-02-28 17:34 ` Kent Overstreet
2024-02-28 18:04 ` Matthew Wilcox
2024-02-28 18:18 ` Kent Overstreet
2024-02-28 19:09 ` Linus Torvalds
2024-02-28 19:29 ` Kent Overstreet
2024-02-28 20:17 ` Linus Torvalds
2024-02-28 23:21 ` Kent Overstreet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=df68c44e-1ab3-485d-a0d6-0c37a06ab4ff@paulmck-laptop \
--to=paulmck@kernel.org \
--cc=axboe@kernel.dk \
--cc=clm@fb.com \
--cc=da.gomez@samsung.com \
--cc=david@fromorbit.com \
--cc=hannes@cmpxchg.org \
--cc=hch@lst.de \
--cc=kent.overstreet@linux.dev \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lsf-pc@lists.linux-foundation.org \
--cc=mcgrof@kernel.org \
--cc=p.raghav@samsung.com \
--cc=torvalds@linux-foundation.org \
--cc=viro@kernel.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).