linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Looking at space used by objects in slab allocator
@ 2007-02-12 18:53 William Cohen
  2007-02-13 14:23 ` Theodore Tso
  0 siblings, 1 reply; 2+ messages in thread
From: William Cohen @ 2007-02-12 18:53 UTC (permalink / raw)
  To: Linux Kernel Mailing List

After last week's experiment reducing size of task_struct on I was
curious to see what things are using up memory on the system and which
structs have the largest impact on the space used. /proc/slabinfo
provides information about the number objects allocated and their
sizes. With one line script the amount of space allocated for various
objects can be ranked in the output form object, count, and
accumulated total :

cat /proc/slabinfo | awk '{ print $1 " " $2 " " $4 " " $2*$4}' |sort -nrk 4


Running this script on an x86_64 Fedora Core Rawhide machine with 1GB
of DRAM building a kernel yielded the top ten slab object allocations as
(<name> <number> <active_obj> <space>):

ext3_inode_cache 18824 1400 26353600
buffer_head 163640 128 20945920
radix_tree_node 12744 576 7340544
dentry_cache 19543 272 5315696
selinux_inode_security 20829 192 3999168
avtab_node 80319 48 3855312
size-32 49882 56 2793392
inode_cache 1404 1032 1448928
size-64 11808 88 1039104
size-2048 360 2072 745920


The running kernel is using a bit more space (3 words (24 bytes) per
allocation) because the following slab allocation debugging is turned
on:

CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SLAB_LEAK=y

The following script lists the total slab allocation and amount used
due to 3 words per allocation (on 32-bit machine the 24 below should be
changed to 12):


cat /proc/slabinfo | \
awk 'BEGIN {doverhead=0; rsum = 0 } \
  { rsum += $2*$4; doverhead += $2*24;} END \
  { print "space:   ", rsum; \
   print "overhead:", doverhead; \
   print "percent: ", (doverhead*100)/rsum }'

Looks like the slab debugging adds about 12% to the memory used on the
64-bit machine, but this will vary depending on the objects.

space:    79562544
overhead: 9627048
percent:  12.1


One of the things that would be desirable is to minimize the amount of
wasted space on the slabs. The following script lists out the wasted space
on each kind of slab. The last field is the space wasted for each slab

cat /proc/slabinfo | sed '1,2 d' |\
  awk '{ emptyslab=(4096*$6-$4*$5); \
         slabs=$3/$5; \
      print $1 " " $4 " " $5 " " $5*$4 " " emptyslab  " " slabs*emptyslab}' | \
sort -nrk 6|more

Below is exerpt of the data when building a kernel showing things
wasting more that 50KBytes

<name> <size> <obj/slab> <used_space/slab> <empty_space/slab> <total_waste>
ext3_inode_cache 1400 2 2800 1296 6457968
inode_cache 1032 3 3096 1000 736000
dentry_cache 272 14 3808 288 673632
buffer_head 128 30 3840 256 512000
avtab_node 48 77 3696 400 417600
size-32 56 67 3752 344 257656
size-2048 2072 3 6216 1976 245024
selinux_inode_security 192 20 3840 256 168448
proc_inode_cache 1064 3 3192 904 122944
radix_tree_node 576 7 4032 64 121472
sighand_cache 2128 3 6384 1808 56048
task_struct 3776 1 3776 320 50240

The is a signficant amount of space wasted for ext3_inode_cache. If
the struct used for ext3_inode_cache struct could be made smaller,
three objects rather than two could fit in a slab. Fitting 3 objects
to a page would decrease the space required for ext3_inod_cache from
20.4MB to 13.6MB in this example.

-Will

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Looking at space used by objects in slab allocator
  2007-02-12 18:53 Looking at space used by objects in slab allocator William Cohen
@ 2007-02-13 14:23 ` Theodore Tso
  0 siblings, 0 replies; 2+ messages in thread
From: Theodore Tso @ 2007-02-13 14:23 UTC (permalink / raw)
  To: William Cohen; +Cc: Linux Kernel Mailing List

On Mon, Feb 12, 2007 at 01:53:03PM -0500, William Cohen wrote:
> After last week's experiment reducing size of task_struct on I was
> The is a signficant amount of space wasted for ext3_inode_cache. If
> the struct used for ext3_inode_cache struct could be made smaller,
> three objects rather than two could fit in a slab. Fitting 3 objects
> to a page would decrease the space required for ext3_inod_cache from
> 20.4MB to 13.6MB in this example.

The problem is the size of "struct inode" (which is included in struct
ext3_inode_info).  There is quite a few fields in struct inode which
are only used when the inode has been opened and being referenced by
one or more file descriptors, but which take up a huge amount of
space.  One of the biggies is inode.i_data; struct address_space is
*huge*, and we're keeping a copy of it in every single inode which
we're caching.

It was next on my list on my inode-diet project that I worked on about
six months ago, but it was non-trivial, and my time got sucked away
with other projects, so I never got back to it.  But if you want to
try to save space, that's the next logical step --- we've already done
all or most of the low-hanging fruit to save space wasted by struct
inode and by extension in the ext3_inode_cache.

						- Ted


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2007-02-13 14:24 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-12 18:53 Looking at space used by objects in slab allocator William Cohen
2007-02-13 14:23 ` Theodore Tso

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).