From: Andy Isaacson <adi@hexapodia.org>
To: Linus Torvalds <torvalds@osdl.org>
Cc: linux-kernel@vger.kernel.org
Subject: Re: sparse file performance (was Re: Is there a "make hole" (truncate in middle) syscall?)
Date: Mon, 8 Dec 2003 14:43:32 -0600 [thread overview]
Message-ID: <20031208144332.A8094@hexapodia.org> (raw)
In-Reply-To: <Pine.LNX.4.58.0312051309390.9125@home.osdl.org>; from torvalds@osdl.org on Fri, Dec 05, 2003 at 01:12:21PM -0800
On Fri, Dec 05, 2003 at 01:12:21PM -0800, Linus Torvalds wrote:
> On Fri, 5 Dec 2003, Andy Isaacson wrote:
> > I got curious enough to run some tests, and was suprised at the results.
> > My machine (Athlon XP 2400+, 2030 MHz, 512 MB, KT400, 2.4.22) can read
> > out of buffer cache at 234 MB/s, and off of its IDE disk at 40 MB/s.
> > I'd assumed that read(2)ing a holey file would go faster than reading
> > out of buffer cache; in theory you could do it completely in L1 cache
> > (with a 4KB buffer, it's just a ton of syscalls, some page table
> > manipulation, and a bunch of memcpy() out of a single zero page). But
> > it turns out that reading a hole is *slower* than reading data from
> > buffer cache, just 195 MB/s.
>
> That's because we actually instantiate the page cache pages even for
> holes. We have to, or we'd have to special-case them no end (and quite
> frankly, "hole read performance" is not something worth special casing,
> since it just isn't done under any real load).
>
> So reading a hole implies creating the page cache entry and _clearing_ it.
> For each page. So while you may read from the L1, you also have to do
> writeback of the _previous_ pages from the L1 into the L2 and eventually
> out to memory.
>
> (And eventually the VM also has to get rid of the pages etc, of course).
Thanks for the explanation, Linus.
I modified my benchmark to use mmap(2) instead of read(2) and the
results are broadly comparable. With a 10MB window, I get 331 MB/s
reading out of buffer cache and 185 MB/s reading a hole. Reading a file
too large to cache is about the same (disk-limited) speed, 43 MB/s.
-andy
next prev parent reply other threads:[~2003-12-08 20:43 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-12-04 20:32 Is there a "make hole" (truncate in middle) syscall? Rob Landley
2003-12-04 20:55 ` Måns Rullgård
2003-12-04 21:10 ` Szakacsits Szabolcs
2003-12-05 0:02 ` Rob Landley
2003-12-04 22:33 ` Szakacsits Szabolcs
2003-12-05 11:22 ` Helge Hafting
2003-12-05 12:11 ` Måns Rullgård
2003-12-05 22:41 ` Mike Fedyk
2003-12-05 23:25 ` Måns Rullgård
2003-12-05 23:33 ` Szakacsits Szabolcs
2003-12-05 23:25 ` Szakacsits Szabolcs
2003-12-04 21:48 ` Mike Fedyk
2003-12-04 23:59 ` Rob Landley
2003-12-05 22:42 ` Olaf Titz
2003-12-04 22:53 ` Peter Chubb
2003-12-05 1:04 ` Philippe Troin
2003-12-05 2:39 ` Peter Chubb
2003-12-08 4:03 ` bill davidsen
2003-12-04 23:23 ` Andy Isaacson
2003-12-04 23:42 ` Szakacsits Szabolcs
2003-12-05 2:03 ` Mike Fedyk
2003-12-05 7:09 ` Ville Herva
2003-12-05 11:22 ` Anton Altaparmakov
2003-12-05 11:44 ` viro
2003-12-05 14:27 ` Anton Altaparmakov
2003-12-05 21:00 ` sparse file performance (was Re: Is there a "make hole" (truncate in middle) syscall?) Andy Isaacson
2003-12-05 21:12 ` Linus Torvalds
2003-12-08 20:43 ` Andy Isaacson [this message]
2003-12-11 5:13 ` Is there a "make hole" (truncate in middle) syscall? Hua Zhong
2003-12-11 6:19 ` Rob Landley
2003-12-11 18:58 ` Andy Isaacson
2003-12-11 19:15 ` Hua Zhong
2003-12-11 19:43 ` Andreas Dilger
2003-12-12 21:37 ` Daniel Phillips
2003-12-11 19:48 ` Jörn Engel
2003-12-11 19:55 ` Hua Zhong
2003-12-11 19:58 ` Andy Isaacson
2003-12-12 12:18 ` Jörn Engel
2003-12-12 15:40 ` Andy Isaacson
2003-12-12 16:03 ` Jörn Engel
2003-12-11 20:32 ` Rob Landley
2003-12-12 12:55 ` Jörn Engel
2003-12-12 13:28 ` Vladimir Saveliev
2003-12-12 13:43 ` Jörn Engel
2003-12-12 13:52 ` Vladimir Saveliev
2003-12-12 14:04 ` Jörn Engel
2003-12-12 13:53 ` Rob Landley
2003-12-12 14:01 ` Vladimir Saveliev
2003-12-12 21:35 ` Rob Landley
2003-12-15 10:00 ` Vladimir Saveliev
2003-12-15 11:52 ` Rob Landley
2003-12-15 13:26 ` Jörn Engel
2003-12-12 13:39 ` Rob Landley
2003-12-12 13:56 ` Jörn Engel
2003-12-12 14:24 ` Jörn Engel
2003-12-12 21:37 ` Rob Landley
2003-12-15 12:47 ` Jörn Engel
2003-12-16 5:43 ` Rob Landley
2003-12-16 11:05 ` Jörn Engel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20031208144332.A8094@hexapodia.org \
--to=adi@hexapodia.org \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).