From: Imran Khan <imran.f.khan@oracle.com>
To: Tejun Heo <tj@kernel.org>
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 1/2] kernfs: use hashed mutex and spinlock in place of global ones.
Date: Mon, 14 Feb 2022 23:19:38 +1100 [thread overview]
Message-ID: <d07546ea-0dd3-f795-ee04-e4b40db5354d@oracle.com> (raw)
In-Reply-To: <YgKm5aSCcCYWkck2@slm.duckdns.org>
Hello Tejun,
Thanks for your feedback.
On 9/2/22 4:22 am, Tejun Heo wrote:
> On Sun, Feb 06, 2022 at 12:09:24PM +1100, Imran Khan wrote:
>> +/*
>> + * NR_KERNFS_LOCK_BITS determines size (NR_KERNFS_LOCKS) of hash
>> + * table of locks.
>> + * Having a small hash table would impact scalability, since
>> + * more and more kernfs_node objects will end up using same lock
>> + * and having a very large hash table would waste memory.
>> + *
>> + * At the moment size of hash table of locks is being set based on
>> + * the number of CPUs as follows:
>> + *
>> + * NR_CPU NR_KERNFS_LOCK_BITS NR_KERNFS_LOCKS
>> + * 1 1 2
>> + * 2-3 2 4
>> + * 4-7 4 16
>> + * 8-15 6 64
>> + * 16-31 8 256
>> + * 32 and more 10 1024
>> + */
>> +#ifdef CONFIG_SMP
>> +#define NR_KERNFS_LOCK_BITS (2 * (ilog2(NR_CPUS < 32 ? NR_CPUS : 32)))
>> +#else
>> +#define NR_KERNFS_LOCK_BITS 1
>> +#endif
>> +
>> +#define NR_KERNFS_LOCKS (1 << NR_KERNFS_LOCK_BITS)
>
> I have a couple questions:
>
> * How did you come up with the above numbers? Are they based on some
> experimentation? It'd be nice if the comment explains why the numbers are
> like that.
>
I did some testing using different number of CPUs with qemu and the
above numbers were collected from there. It may not be optimum but this
is what I could think of while doing those internal tests.
Do you think it may be better to make this configurable via Kconfig.
> * IIRC, we split these locks to per kernfs instance recently as a way to
> mitigate lock contention occurring across kernfs instances. I don't think
> it's beneficial to keep these hashed locks separate. It'd be both simpler
> and less contended to double one shared hashtable than splitting the table
> into two separate half sized ones. So, maybe switch to global hashtables
> and increase the size?
I have removed the hash table from kernfs_root. Doubling the size was
not giving any improvements so I have kept the hash table size same as
before. This change is present in v6 of patch at [1]
Thanks
-- Imran
[1]:
https://lore.kernel.org/lkml/20220214120322.2402628-1-imran.f.khan@oracle.com/
>
> Thanks.
>
next prev parent reply other threads:[~2022-02-14 12:20 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-06 1:09 [PATCH v5 0/2] kernfs: use hashed mutex and spinlock in place of global ones Imran Khan
2022-02-06 1:09 ` [PATCH v5 1/2] " Imran Khan
2022-02-08 11:27 ` Greg KH
2022-02-14 12:13 ` Imran Khan
2022-02-08 17:22 ` Tejun Heo
2022-02-14 12:19 ` Imran Khan [this message]
2022-02-14 17:39 ` Tejun Heo
2022-02-06 1:09 ` [PATCH v5 2/2] kernfs: Replace per-fs global rwsem with per-fs hashed rwsem Imran Khan
2022-02-08 18:26 ` Tejun Heo
2022-02-14 12:27 ` Imran Khan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d07546ea-0dd3-f795-ee04-e4b40db5354d@oracle.com \
--to=imran.f.khan@oracle.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).