From: Stephen Hemminger <stephen@networkplumber.org>
To: ebiederm@xmission.com (Eric W. Biederman)
Cc: Alexey Dobriyan <adobriyan@gmail.com>,
Nicolas Dichtel <nicolas.dichtel@6wind.com>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
davem@davemloft.net, akpm@linux-foundation.org,
rui.xiang@huawei.com, viro@zeniv.linux.org.uk, oleg@redhat.com,
gorcunov@openvz.org, kirill.shutemov@linux.intel.com,
grant.likely@secretlab.ca, tytso@mit.edu,
Thierry Herbelot <thierry.herbelot@6wind.com>
Subject: Re: [RFC PATCH linux 2/2] fs/proc: use a hash table for the directory entries
Date: Thu, 2 Oct 2014 14:27:03 -0700 [thread overview]
Message-ID: <20141002142703.10325f4f@urahara> (raw)
In-Reply-To: <87h9zmji3q.fsf@x220.int.ebiederm.org>
On Thu, 02 Oct 2014 14:07:37 -0700
ebiederm@xmission.com (Eric W. Biederman) wrote:
> Alexey Dobriyan <adobriyan@gmail.com> writes:
>
> > On Thu, Oct 02, 2014 at 11:01:50AM -0700, Eric W. Biederman wrote:
> >> Nicolas Dichtel <nicolas.dichtel@6wind.com> writes:
> >>
> >> > From: Thierry Herbelot <thierry.herbelot@6wind.com>
> >> >
> >> > The current implementation for the directories in /proc is using a single
> >> > linked list. This is slow when handling directories with large numbers of
> >> > entries (eg netdevice-related entries when lots of tunnels are opened).
> >> >
> >> > This patch enables multiple linked lists. A hash based on the entry name is
> >> > used to select the linked list for one given entry.
> >> >
> >> > The speed creation of netdevices is faster as shorter linked lists must be
> >> > scanned when adding a new netdevice.
> >>
> >> Is the directory of primary concern /proc/net/dev/snmp6 ?
> >>
> >> Unless I have configured my networking stack weird by mistake that
> >> is the only directory under /proc/net that grows when we add an
> >> interface.
> >>
> >> I just want to make certain I am seeing the same things that you are
> >> seeing.
> >>
> >> I feel silly for overlooking this directory when the rest of the
> >> scalability work was done.
> >
> > Slowdown comes from "duplicate name" check:
> >
> > for (tmp = dir->subdir; tmp; tmp = tmp->next)
> > if (strcmp(tmp->name, dp->name) == 0) {
> > WARN(1, "proc_dir_entry '%s/%s' already registered\n",
> > dir->name, dp->name);
> > break;
> > }
> >
> > Removal can be made O(1) after switching to doubly-linked list.
>
> Yes. There is the however unfortunate fact that proc directories exist
> to be used. If we don't switch to a better data structure than a linked
> list the actual use will then opening of the files under
> /proc/net/dev/snmp6/ will become O(N^2). Which doesn't help much
> (assuming those files are good for something).
>
> If those files aren't actually useful we should just make registering
> them a config option. Deprecate them strongly and let only people who
> need extreme backwards compatibility enable them.
Net-snmp uses them (agent/mibgroup/mibII/kernel_linux.c)
next prev parent reply other threads:[~2014-10-02 21:27 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20131003.150947.2179820478039260398.davem@davemloft.net>
2014-10-02 15:24 ` [RFC PATCH linux 0/2] Optimize network interfaces creation Nicolas Dichtel
2014-10-02 15:25 ` [RFC PATCH linux 1/2] proc_net: declare /proc/net as a directory Nicolas Dichtel
2014-10-02 15:25 ` [RFC PATCH linux 2/2] fs/proc: use a hash table for the directory entries Nicolas Dichtel
2014-10-02 16:46 ` Stephen Hemminger
2014-10-03 13:10 ` Nicolas Dichtel
2014-10-02 17:28 ` Alexey Dobriyan
2014-10-03 13:07 ` Nicolas Dichtel
2014-10-02 18:01 ` Eric W. Biederman
2014-10-02 20:06 ` Alexey Dobriyan
2014-10-02 21:07 ` Eric W. Biederman
2014-10-02 21:27 ` Stephen Hemminger [this message]
2014-10-03 7:28 ` Nicolas Dichtel
2014-10-03 13:09 ` Nicolas Dichtel
2014-10-06 14:30 ` [PATCH linux v2 0/1] Optimize network interfaces creation Nicolas Dichtel
2014-10-06 14:30 ` [PATCH linux v2 1/1] fs/proc: use a rb tree for the directory entries Nicolas Dichtel
2014-10-06 22:14 ` David Miller
2014-10-07 9:02 ` [PATCH linux v3 0/1] Optimize network interfaces creation Nicolas Dichtel
2014-10-07 9:02 ` [PATCH linux v3 1/1] fs/proc: use a rb tree for the directory entries Nicolas Dichtel
2014-10-13 11:14 ` Nicolas Dichtel
2014-10-14 19:30 ` David Miller
2014-10-14 19:56 ` Eric W. Biederman
2014-10-15 9:02 ` Nicolas Dichtel
2014-10-15 21:37 ` Andrew Morton
2014-10-03 10:55 ` [RFC PATCH linux 2/2] fs/proc: use a hash table " Alexey Dobriyan
2014-10-03 13:07 ` Nicolas Dichtel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141002142703.10325f4f@urahara \
--to=stephen@networkplumber.org \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=davem@davemloft.net \
--cc=ebiederm@xmission.com \
--cc=gorcunov@openvz.org \
--cc=grant.likely@secretlab.ca \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nicolas.dichtel@6wind.com \
--cc=oleg@redhat.com \
--cc=rui.xiang@huawei.com \
--cc=thierry.herbelot@6wind.com \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).