linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* readlink(), the pause that refreshes
@ 2007-12-11 22:16 Andre Majorel
  0 siblings, 0 replies; only message in thread
From: Andre Majorel @ 2007-12-11 22:16 UTC (permalink / raw)
  To: linux-kernel

Kernel 2.6.22.14. Create a tree of directories containing tens or
hundreds of thousands of symlinks and run

  find linkfarm | program-that-readlinks-the-file-names-in-its-stdin

Memory usage will grow and grow until the system starts to swap,
after which you are SOL. The combined effects of thrashing and the
OOM killer will render your system catatonic for a long time and
if it finally recovers, you probably won't be able to log in. With
1 GB of RAM, the tipping point is somewhere above 700,000
symlinks.

So you kill the pipe before it starts to swap. Memory usage does
not go down. Try "sync" or "umount /linkfarm". They will block for
minutes, during which memory usage goes down slowly, on the order
of 1 MB per second. All the while, iostat indicates several
hundred transactions per second on the disk where the link farm
resides.

All this crap appears to be related to the updating of the atimes
after a readlink. I understand that the problem could be
alleviated by mounting the file system noatime or relatime, but
this not just a performance problem. You should not be able to
make your system unusable with just one process making readlink()
calls.

As a stop gap, is there an equivalent of O_NOATIME for readlink() ?

-- 
André Majorel <URL:http://www.teaser.fr/~amajorel/>
Do not use this account for regular correspondence.
See the URL above for contact information.

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2007-12-11 22:48 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-12-11 22:16 readlink(), the pause that refreshes Andre Majorel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).