linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/69] faster tree-based sysctl implementation
@ 2011-05-01  1:35 Lucian Adrian Grijincu
  2011-05-01  1:35 ` [PATCH 01/69] sysctl: remove .child from dev/parport/default Lucian Adrian Grijincu
                   ` (70 more replies)
  0 siblings, 71 replies; 82+ messages in thread
From: Lucian Adrian Grijincu @ 2011-05-01  1:35 UTC (permalink / raw)
  To: linux-kernel
  Cc: Lucian Adrian Grijincu, Eric W . Biederman, Alexey Dobriyan,
	Octavian Purdila, David S . Miller


Short description: sysctl is slow (bad algorithm); this patch series
makes it faster (without using more memory but with some limitations).


$ time modprobe dummy numdummies=N

Without this patch series :(
- ipv4 only
  -  N=1000  time= 0m 06s
  -  N=2000  time= 0m 30s
  -  N=4000  time= 2m 35s
- ipv4 and ipv6
  -  N=1000  time= 0m 24s
  -  N=2000  time= 2m 14s
  -  N=4000  time=10m 16s
  -  N=5000  time=16m 03s


With this patch series    :)
- ipv4 only
  -  N=1000  time=0.33s
  -  N=2000  time=1.25s
  -  N=4000  time=5.31s
- ipv4 and ipv6
  -  N=1000  time=0.41s
  -  N=2000  time=1.62s
  -  N=4000  time=7.64s
  -  N=5000  time=12.35s
  -  N=8000  time=36.95s


Tests were done with and without ipv6 with the same .config on
2.6.39-rc5 with and without this patch series running:

Note about time stats: adding a netdevice registers
- two sysctl tables for ipv4:
     /proc/sys/net/ipv4/conf/DEVICE
     /proc/sys/net/ipv4/neigh/DEVICE
- two sysctl tables for ipv6:
     /proc/sys/net/ipv6/conf/DEVICE
     /proc/sys/net/ipv6/neigh/DEVICE


Some companies (e.g. IXIACOM which sponsored this work) have use cases
where they need to create 10^3->10^5 (virtual) network devices. They
can't use the current sysctl because of the time it takes to register
so many sysctl tables.



The first patches remove the .child field of ctl_table. This is a
requirement for the new algorithm. These patches are scattered all
over the tree :(

 Some patches make changes in architectures I don't know how to
compile to or drivers for which I don't have devices (there the
patches were at least compiled :).



People interested in the core sysctl changes/networking should read:

   [PATCH 60/69] sysctl: faster tree-based sysctl implementation

which introduces the new algorithm (commit message and comments have
more details), and the next few patches which add some further (simple
and effective) optimisations for networking (and not only).


The last patch tries to replace a rwsem with rcu+spinlock. I'm not
sure about it because I haven't worked with RCU before. If you find
some big ugly monstrosity there, please don't disregard the rest of
the series. :)



Cc: "Eric W . Biederman" <ebiederm@xmission.com>
Cc: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Octavian Purdila <tavi@cs.pub.ro>
Cc: "David S . Miller" <davem@davemloft.net>


Lucian Adrian Grijincu (69):
  sysctl: remove .child from dev/parport/default
  sysctl: parport: reorder .child assignments to simplify review
  sysctl: remove .child from dev/parport/PORT/devices/DEVICE
  sysctl: remove .child from dev/parport/PORT/
  sysctl: remove .child from dev/parport/PORT/devices/
  sysctl: remove .child from kernel/vsyscall64 (x86)
  sysctl: remove .child from abi/vsyscall32 (x86)
  sysctl: remove .child from crypto/fips_enabled
  sysctl: remove .child from dev/cdrom/
  sysctl: remove .child from dev/hpet/
  sysctl: remove .child from dev/ipmi/
  sysctl: remove .child from dev/rtc/
  sysctl: remove .child from dev/mac_hid/
  sysctl: remove .child from dev/raid/
  sysctl: remove .child from xpc/
  sysctl: remove .child from xpc/hb
  sysctl: remove .child from kernel/sclp (s390)
  sysctl: remove .child from dev/scsi
  sysctl: remove .child from kernel/pty
  sysctl: remove .child from coda/
  sysctl: remove .child from fscache/
  sysctl: remove .child from fs/nfs/ nlm_table table
  sysctl: remove .child from fs/nfs/ nfs_cb_table
  sysctl: remove .child from fs/ntfs-debug
  sysctl: remove .child from fs/ocfs2/nm/
  sysctl: remove .child from fs/quota/
  sysctl: remove .child from fs/xfs/
  sysctl: remove .child from kernel/ (ipc)
  sysctl: remove .child from fs/mqueue
  sysctl: sched: add sd_table_template
  sysctl: remove .child from kernel/sched_domain/cpuX/domainY/
  sysctl: remove .child from kernel/ (utsname)
  sysctl: remove .child from sunrpc/
  sysctl: remove .child from sunrpc/svc_rdma
  sysctl: remove .child from sunrpc/ (xprtrdma)
  sysctl: remove .child from sunrpc/ (xprtsock)
  sysctl: remove .child from bus/isa/ (arm)
  sysctl: remove .child from reboot/warm (arm)
  sysctl: remove .child from lasat/ (mips)
  sysctl: remove .child from appldata/ (s390)
  sysctl: remove .child from s390dbf/
  sysctl: remove .child from vm/ (s390)
  sysctl: remove .child from kernel/perfmon/ (ia64)
  sysctl: remove .child from kernel/ (ia64/kdump)
  sysctl: remove .child from kernel/powersave-nap (powerpc)
  sysctl: remove .child from pm/ (frv)
  sysctl: remove .child from frv/
  sysctl: remove .child from sh64/unaligned_fixup/
  sysctl: delete unused register_sysctl_table function
  sysctl: remove .child from ax25 table
  sysctl: remove .child from net/ipv4/route and net/ipv4/neigh tables
  sysctl: remove .child from net/ipv4/neigh table
  sysctl: remove .child from net/ipv6/route, net/ipv6/icmp, net/ipv6 tables
  sysctl: remove .child from net/llc tables
  sysctl: no-child: manually register kernel/random
  sysctl: no-child: manually register kernel/keys
  sysctl: no-child: manually register fs/inotify
  sysctl: no-child: manually register fs/epoll
  sysctl: no-child: manually register root tables
  sysctl: faster tree-based sysctl implementation
  sysctl: single subheader path: optimisation for paths used only once
  sysctl: single subheader path: net/ipv4/conf/DEVICE-NAME/
  sysctl: single subheader path: net/{ipv4|ipv6}/neigh/DEV/
  sysctl: single subheader path: net/ipv6/conf/DEVICE-NAME/
  sysctl: single subheader path: dev/parport/PORT/devices/DEVICE/
  sysctl: single subheader path: net/ax25/DEVICE
  sysctl: single subheader path: kernel/sched_domain/CPU/DOMAIN/
  sysctl: single subheader path: net/decnet/conf/DEVNAME
  RFC: sysctl: convert read-write lock to RCU

 arch/arm/kernel/isa.c              |   31 +-
 arch/arm/mach-bcmring/arch.c       |   25 +-
 arch/frv/kernel/pm.c               |   10 +-
 arch/frv/kernel/sysctl.c           |   12 +-
 arch/ia64/kernel/crash.c           |   13 +-
 arch/ia64/kernel/perfmon.c         |   23 +-
 arch/mips/lasat/sysctl.c           |   13 +-
 arch/powerpc/kernel/idle.c         |   13 +-
 arch/s390/appldata/appldata_base.c |   42 +-
 arch/s390/kernel/debug.c           |   13 +-
 arch/s390/mm/cmm.c                 |   11 +-
 arch/sh/kernel/traps_64.c          |   21 +-
 arch/x86/kernel/vsyscall_64.c      |   25 +-
 arch/x86/vdso/vdso32-setup.c       |   14 +-
 crypto/proc.c                      |   12 +-
 drivers/cdrom/cdrom.c              |   22 +-
 drivers/char/hpet.c                |   38 +--
 drivers/char/ipmi/ipmi_poweroff.c  |   16 +-
 drivers/char/random.c              |   27 ++-
 drivers/char/rtc.c                 |   24 +-
 drivers/macintosh/mac_hid.c        |   26 +-
 drivers/md/md.c                    |   22 +-
 drivers/misc/sgi-xp/xpc_main.c     |   81 ++--
 drivers/parport/procfs.c           |  231 +++++-------
 drivers/s390/char/sclp_async.c     |   13 +-
 drivers/scsi/scsi_sysctl.c         |   28 +-
 drivers/tty/pty.c                  |   23 +-
 fs/coda/sysctl.c                   |   12 +-
 fs/eventpoll.c                     |   22 +-
 fs/fscache/main.c                  |   15 +-
 fs/lockd/svc.c                     |   22 +-
 fs/nfs/sysctl.c                    |   22 +-
 fs/notify/inotify/inotify_user.c   |   22 +-
 fs/ntfs/sysctl.c                   |   15 +-
 fs/ocfs2/stackglue.c               |   36 +--
 fs/proc/inode.c                    |    2 +-
 fs/proc/proc_sysctl.c              |  201 ++++++----
 fs/quota/dquot.c                   |   21 +-
 fs/xfs/linux-2.6/xfs_sysctl.c      |   22 +-
 include/linux/inotify.h            |    2 -
 include/linux/key.h                |    4 +-
 include/linux/poll.h               |    2 -
 include/linux/sysctl.h             |  189 ++++++----
 include/net/ax25.h                 |   10 +-
 include/net/net_namespace.h        |    2 +-
 include/net/netns/ipv6.h           |    4 +-
 init/main.c                        |    2 +
 ipc/ipc_sysctl.c                   |   12 +-
 ipc/mq_sysctl.c                    |   24 +-
 kernel/sched.c                     |  398 +++++++++++++------
 kernel/sysctl.c                    |  798 +++++++++++++++++++++---------------
 kernel/sysctl_check.c              |  253 ++++++------
 kernel/utsname_sysctl.c            |   14 +-
 net/ax25/af_ax25.c                 |   23 +-
 net/ax25/ax25_dev.c                |   10 +-
 net/ax25/sysctl_net_ax25.c         |   82 ++---
 net/core/neighbour.c               |    8 +-
 net/decnet/dn_dev.c                |    8 +-
 net/ipv4/devinet.c                 |    8 +-
 net/ipv4/route.c                   |   15 +-
 net/ipv6/addrconf.c                |    8 +-
 net/ipv6/sysctl_net_ipv6.c         |  119 +++---
 net/llc/sysctl_net_llc.c           |   55 ++--
 net/sunrpc/sysctl.c                |   19 +-
 net/sunrpc/xprtrdma/svc_rdma.c     |   26 +-
 net/sunrpc/xprtrdma/transport.c    |   14 +-
 net/sunrpc/xprtsock.c              |   16 +-
 net/sysctl_net.c                   |   63 ++--
 security/keys/key.c                |    1 +
 security/keys/sysctl.c             |   18 +-
 70 files changed, 1778 insertions(+), 1670 deletions(-)

-- 
1.7.5.134.g1c08b


^ permalink raw reply	[flat|nested] 82+ messages in thread

end of thread, other threads:[~2011-05-02 21:43 UTC | newest]

Thread overview: 82+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-01  1:35 [PATCH 00/69] faster tree-based sysctl implementation Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 01/69] sysctl: remove .child from dev/parport/default Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 02/69] sysctl: parport: reorder .child assignments to simplify review Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 03/69] sysctl: remove .child from dev/parport/PORT/devices/DEVICE Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 04/69] sysctl: remove .child from dev/parport/PORT/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 05/69] sysctl: remove .child from dev/parport/PORT/devices/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 06/69] sysctl: remove .child from kernel/vsyscall64 (x86) Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 07/69] sysctl: remove .child from abi/vsyscall32 (x86) Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 08/69] sysctl: remove .child from crypto/fips_enabled Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 09/69] sysctl: remove .child from dev/cdrom/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 10/69] sysctl: remove .child from dev/hpet/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 11/69] sysctl: remove .child from dev/ipmi/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 12/69] sysctl: remove .child from dev/rtc/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 13/69] sysctl: remove .child from dev/mac_hid/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 14/69] sysctl: remove .child from dev/raid/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 15/69] sysctl: remove .child from xpc/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 16/69] sysctl: remove .child from xpc/hb Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 17/69] sysctl: remove .child from kernel/sclp (s390) Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 18/69] sysctl: remove .child from dev/scsi Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 19/69] sysctl: remove .child from kernel/pty Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 20/69] sysctl: remove .child from coda/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 21/69] sysctl: remove .child from fscache/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 22/69] sysctl: remove .child from fs/nfs/ nlm_table table Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 23/69] sysctl: remove .child from fs/nfs/ nfs_cb_table Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 24/69] sysctl: remove .child from fs/ntfs-debug Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 25/69] sysctl: remove .child from fs/ocfs2/nm/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 26/69] sysctl: remove .child from fs/quota/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 27/69] sysctl: remove .child from fs/xfs/ Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 28/69] sysctl: remove .child from kernel/ (ipc) Lucian Adrian Grijincu
2011-05-01  1:35 ` [PATCH 29/69] sysctl: remove .child from fs/mqueue Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 30/69] sysctl: sched: add sd_table_template Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 31/69] sysctl: remove .child from kernel/sched_domain/cpuX/domainY/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 32/69] sysctl: remove .child from kernel/ (utsname) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 33/69] sysctl: remove .child from sunrpc/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 34/69] sysctl: remove .child from sunrpc/svc_rdma Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 35/69] sysctl: remove .child from sunrpc/ (xprtrdma) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 36/69] sysctl: remove .child from sunrpc/ (xprtsock) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 37/69] sysctl: remove .child from bus/isa/ (arm) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 38/69] sysctl: remove .child from reboot/warm (arm) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 39/69] sysctl: remove .child from lasat/ (mips) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 40/69] sysctl: remove .child from appldata/ (s390) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 41/69] sysctl: remove .child from s390dbf/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 42/69] sysctl: remove .child from vm/ (s390) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 43/69] sysctl: remove .child from kernel/perfmon/ (ia64) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 44/69] sysctl: remove .child from kernel/ (ia64/kdump) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 45/69] sysctl: remove .child from kernel/powersave-nap (powerpc) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 46/69] sysctl: remove .child from pm/ (frv) Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 47/69] sysctl: remove .child from frv/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 48/69] sysctl: remove .child from sh64/unaligned_fixup/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 49/69] sysctl: delete unused register_sysctl_table function Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 50/69] sysctl: remove .child from ax25 table Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 51/69] sysctl: remove .child from net/ipv4/route and net/ipv4/neigh tables Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 52/69] sysctl: remove .child from net/ipv4/neigh table Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 53/69] sysctl: remove .child from net/ipv6/route, net/ipv6/icmp, net/ipv6 tables Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 54/69] sysctl: remove .child from net/llc tables Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 55/69] sysctl: no-child: manually register kernel/random Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 56/69] sysctl: no-child: manually register kernel/keys Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 57/69] sysctl: no-child: manually register fs/inotify Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 58/69] sysctl: no-child: manually register fs/epoll Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 59/69] sysctl: no-child: manually register root tables Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 60/69] sysctl: faster tree-based sysctl implementation Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 61/69] sysctl: single subheader path: optimisation for paths used only once Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 62/69] sysctl: single subheader path: net/ipv4/conf/DEVICE-NAME/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 63/69] sysctl: single subheader path: net/{ipv4|ipv6}/neigh/DEV/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 64/69] sysctl: single subheader path: net/ipv6/conf/DEVICE-NAME/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 65/69] sysctl: single subheader path: dev/parport/PORT/devices/DEVICE/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 66/69] sysctl: single subheader path: net/ax25/DEVICE Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 67/69] sysctl: single subheader path: kernel/sched_domain/CPU/DOMAIN/ Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 68/69] sysctl: single subheader path: net/decnet/conf/DEVNAME Lucian Adrian Grijincu
2011-05-01  1:36 ` [PATCH 69/69] RFC: sysctl: convert read-write lock to RCU Lucian Adrian Grijincu
2011-05-01 11:07 ` [PATCH 00/69] faster tree-based sysctl implementation Lucian Adrian Grijincu
2011-05-01 15:28 ` Eric W. Biederman
2011-05-01 16:20   ` Lucian Adrian Grijincu
2011-05-01 22:49     ` Eric W. Biederman
2011-05-01 23:12       ` Lucian Adrian Grijincu
2011-05-02  3:06         ` Eric W. Biederman
2011-05-02  7:44           ` Lucian Adrian Grijincu
2011-05-02 19:02             ` Eric W. Biederman
2011-05-02 21:43               ` Lucian Adrian Grijincu
2011-05-01 23:34       ` Lucian Adrian Grijincu
2011-05-02  0:03         ` Lucian Adrian Grijincu
2011-05-02  3:26         ` Eric W. Biederman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).