From: James Simmons <jsimmons@casper.infradead.org>
To: Lustre-devel <lustre-devel@lists.lustre.org>
Subject: [lustre-devel] Using generic radix trees for large allocations
Date: Thu, 21 Jan 2021 18:32:24 +0000 [thread overview]
Message-ID: <20210121183224.GA4134678@casper.infradead.org> (raw)
One of the challenging issues for very large scale file systems is the
performance crash when you cross about 670 stripe count. This is due to
the memory allocations going from kmalloc to vmalloc. Once you start to
use vmalloc to allocate the ptlrpc message buffers all the allocating
start to serialize on a global spinlock.
Looking for a solution the best one I found so far have been using the
generic radix tree API. You have to allocate a page worth of data at a
time so its cluncky for use but I think we could make it work. What do
you think?
https://www.kernel.org/doc/html/latest/core-api/generic-radix-tree.html
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
next reply other threads:[~2021-01-21 18:33 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-21 18:32 James Simmons [this message]
2021-01-22 7:14 ` [lustre-devel] Using generic radix trees for large allocations Andreas Dilger
2021-01-27 7:21 ` Alexey Lyashkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210121183224.GA4134678@casper.infradead.org \
--to=jsimmons@casper.infradead.org \
--cc=lustre-devel@lists.lustre.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).