* 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared for various fs.
@ 2002-10-23 19:42 Steven Cole
2002-10-23 19:57 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Andrew Morton
0 siblings, 1 reply; 11+ messages in thread
From: Steven Cole @ 2002-10-23 19:42 UTC (permalink / raw)
To: Andrew Morton; +Cc: Linux Kernel, Alan Cox
Greetings all,
I performed some timing tests for 2.5.44 mm3 and ac2, using
tar zxf of the 2.5.44 tarball and rm -rf of the resulting tree as the load.
I performed each of these 4 times. There was some variation between runs,
but not as much as between kernel versions. This data comes from the
first of the 4 runs in each case.
The hardware was a single PIII, kernels were UP, PREEMPT.
All partitions are on the same disk, a ST340016A ATA.
For mm3, SHAREPTE was not enabled.
This is not intended as a comparison between filesystems, since each
is on a different part of the disk. But the reputation for reiserfs
to be able to delete files quickly seems deserved. The side-by-side numbers
are intended to show the amount of regression for each fs.
Yes, I know it would be nice to compare plain 2.5.44 too, but there is
only so much time in the day.
Steven
ext3
tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
user 4.42 4.39
system 4.09 4.05
elapsed 00:53.17 00:34.05
% CPU 16 24
rm -rf linux-2.5.44 2.5.44-mm3 2.5.44-ac2
user 0.02 0.02
system 0.58 0.58
elapsed 00:19.73 00:14.13
% CPU 3 4
reiserfs
tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
user 4.57 4.51
system 5.4 5.22
elapsed 00:15.58 00:14.09
% CPU 64 69
rm -rf linux-2.5.44 2.5.44-mm3 2.5.44-ac2
user 0.02 0.02
system 1.65 1.66
elapsed 00:04.38 00:01.92
% CPU 38 88
xfs
tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
user 4.61 4.6
system 6.13 6.08
elapsed 00:58.93 00:40.26
% CPU 18 26
rm -rf linux-2.5.44 2.5.44-mm3 2.5.44-ac2
user 0.07 0.05
system 2.23 2.14
elapsed 00:19.15 00:08.68
% CPU 12 25
jfs
tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
user 4.53 4.4
system 4.01 3.94
elapsed 00:34.71 00:31.67
% CPU 24 26
rm -rf linux-2.5.44 2.5.44-mm3 2.5.44-ac2
user 0.03 0.03
system 0.62 0.55
elapsed 00:14.78 00:08.60
% CPU 4 6
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 19:42 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared for various fs Steven Cole
@ 2002-10-23 19:57 ` Andrew Morton
2002-10-23 20:04 ` Benjamin LaHaise
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Andrew Morton @ 2002-10-23 19:57 UTC (permalink / raw)
To: Steven Cole; +Cc: Linux Kernel, Alan Cox
Steven Cole wrote:
>
> ext3
> tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> user 4.42 4.39
> system 4.09 4.05
> elapsed 00:53.17 00:34.05
> % CPU 16 24
The smaller fifo_batch setting hurts when there are competing
reads and writes on the same disk.
> reiserfs
> xfs
> jfs
ext2?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 19:57 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Andrew Morton
@ 2002-10-23 20:04 ` Benjamin LaHaise
2002-10-23 20:09 ` Andrew Morton
2002-10-23 20:05 ` Steven Cole
2002-10-23 20:32 ` Steven Cole
2 siblings, 1 reply; 11+ messages in thread
From: Benjamin LaHaise @ 2002-10-23 20:04 UTC (permalink / raw)
To: Andrew Morton; +Cc: Steven Cole, Linux Kernel, Alan Cox
On Wed, Oct 23, 2002 at 12:57:24PM -0700, Andrew Morton wrote:
> Steven Cole wrote:
> >
> > ext3
> > tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> > user 4.42 4.39
> > system 4.09 4.05
> > elapsed 00:53.17 00:34.05
> > % CPU 16 24
>
> The smaller fifo_batch setting hurts when there are competing
> reads and writes on the same disk.
Is the ext2/3 allocation heuristic fix in yet? That might swing things
around again too.
-ben
--
"Do you seek knowledge in time travel?"
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 19:57 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Andrew Morton
2002-10-23 20:04 ` Benjamin LaHaise
@ 2002-10-23 20:05 ` Steven Cole
2002-10-23 20:32 ` Steven Cole
2 siblings, 0 replies; 11+ messages in thread
From: Steven Cole @ 2002-10-23 20:05 UTC (permalink / raw)
To: Andrew Morton; +Cc: Linux Kernel, Alan Cox
On Wed, 2002-10-23 at 13:57, Andrew Morton wrote:
> Steven Cole wrote:
> >
> > ext3
> > tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> > user 4.42 4.39
> > system 4.09 4.05
> > elapsed 00:53.17 00:34.05
> > % CPU 16 24
>
> The smaller fifo_batch setting hurts when there are competing
> reads and writes on the same disk.
>
> > reiserfs
> > xfs
> > jfs
>
> ext2?
OK, if ext2 would be of interest.
Here is the result of df -T:
Filesystem Type Size Used Avail Use% Mounted on
/dev/hda1 ext3 236M 77M 148M 35% /
/dev/hda9 ext3 20G 8.0G 12G 41% /home
/dev/hda11 jfs 3.9G 1.8G 2.2G 46% /share_jfs
/dev/hda10
reiserfs 4.0G 73M 3.9G 2% /share_reiser
/dev/hda12 xfs 4.8G 290M 4.5G 6% /share_xfs
/dev/hda8 ext3 236M 4.7M 219M 3% /tmp
/dev/hda6 ext3 2.9G 1.3G 1.5G 47% /usr
/dev/hda7 ext3 479M 63M 392M 14% /var
I'll do the test on /tmp remounted as ext2.
Back in a while,
Steven
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 20:04 ` Benjamin LaHaise
@ 2002-10-23 20:09 ` Andrew Morton
2002-10-24 8:34 ` Daniel Phillips
0 siblings, 1 reply; 11+ messages in thread
From: Andrew Morton @ 2002-10-23 20:09 UTC (permalink / raw)
To: Benjamin LaHaise; +Cc: Steven Cole, Linux Kernel, Alan Cox
Benjamin LaHaise wrote:
>
> On Wed, Oct 23, 2002 at 12:57:24PM -0700, Andrew Morton wrote:
> > Steven Cole wrote:
> > >
> > > ext3
> > > tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> > > user 4.42 4.39
> > > system 4.09 4.05
> > > elapsed 00:53.17 00:34.05
> > > % CPU 16 24
> >
> > The smaller fifo_batch setting hurts when there are competing
> > reads and writes on the same disk.
>
> Is the ext2/3 allocation heuristic fix in yet? That might swing things
> around again too.
>
Only for ext2, only in -mm. I havben't submitted the Orlov allocator
because Ted is playing with it.
ext3's five-second commit interval and limited journal size really
bite in this test. We end up doing most of the writeback within
the measurement period rather than after it.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 19:57 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Andrew Morton
2002-10-23 20:04 ` Benjamin LaHaise
2002-10-23 20:05 ` Steven Cole
@ 2002-10-23 20:32 ` Steven Cole
2002-10-23 20:44 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball comparedforvarious fs Andrew Morton
2002-10-24 3:10 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Hans Reiser
2 siblings, 2 replies; 11+ messages in thread
From: Steven Cole @ 2002-10-23 20:32 UTC (permalink / raw)
To: Andrew Morton; +Cc: Linux Kernel, Alan Cox
On Wed, 2002-10-23 at 13:57, Andrew Morton wrote:
> Steven Cole wrote:
> >
> > ext3
> > tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> > user 4.42 4.39
> > system 4.09 4.05
> > elapsed 00:53.17 00:34.05
> > % CPU 16 24
>
> The smaller fifo_batch setting hurts when there are competing
> reads and writes on the same disk.
>
> > reiserfs
> > xfs
> > jfs
>
> ext2?
OK, here is the ext2 data. This was done on my /tmp partition.
For ext2, the variation between runs was as much as between
mm3 and ac2. This data is from the first of 4 runs as before.
Steven
ext2
tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
user 4.17 4.16
system 2.76 2.7
elapsed 00:08.39 00:08.05
% CPU 82 85
rm -rf linux-2.5.44 2.5.44-mm3 2.5.44-ac2
user 0.01 0.01
system 0.4 0.37
elapsed 00:02.31 00:01.17
% CPU 18 33
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball comparedforvarious fs.
2002-10-23 20:32 ` Steven Cole
@ 2002-10-23 20:44 ` Andrew Morton
2002-10-24 3:10 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Hans Reiser
1 sibling, 0 replies; 11+ messages in thread
From: Andrew Morton @ 2002-10-23 20:44 UTC (permalink / raw)
To: Steven Cole; +Cc: Linux Kernel, Alan Cox
Steven Cole wrote:
>
> ...
> OK, here is the ext2 data. This was done on my /tmp partition.
>
> For ext2, the variation between runs was as much as between
> mm3 and ac2. This data is from the first of 4 runs as before.
>
> Steven
>
> ext2
> tar zxf linux-2.5.44.tar.gz 2.5.44-mm3 2.5.44-ac2
> user 4.17 4.16
> system 2.76 2.7
> elapsed 00:08.39 00:08.05
> % CPU 82 85
>
OK, so I assume what's happening here is that the entire uncompressed
kernel fits into 40% of your memory.
So we see 4 seconds user time from doing the gzip decompression
and three seconds system time; a little from reading the
tarball and most of it is creating a ton of dirty pagecache.
But most of the real cost has not been measured: getting that
dirty pagecache onto disk. It has to happen sometime...
If you include a "sync" in the timing then you'll see the
benefit from the Orlov allocator. You'll get that kernel
tree onto disk in half the time.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 20:32 ` Steven Cole
2002-10-23 20:44 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball comparedforvarious fs Andrew Morton
@ 2002-10-24 3:10 ` Hans Reiser
2002-10-24 9:54 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious Henning P. Schmiedehausen
1 sibling, 1 reply; 11+ messages in thread
From: Hans Reiser @ 2002-10-24 3:10 UTC (permalink / raw)
To: Steven Cole; +Cc: Andrew Morton, Linux Kernel, Alan Cox
Please repeat the test using a tarball created from a reiserfs
partition, you'll get better numbers for reiserfs. Not as good as
reiser4, but still much better than these.
File order from readdir matters a lot.
It is a bit amazing how many obscurities can bite you with seemingly
simple tests like this. We recently ran into one with tar recognizing
that it was writing to /dev/null, and optimizing for it.
Hans
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs.
2002-10-23 20:09 ` Andrew Morton
@ 2002-10-24 8:34 ` Daniel Phillips
0 siblings, 0 replies; 11+ messages in thread
From: Daniel Phillips @ 2002-10-24 8:34 UTC (permalink / raw)
To: Andrew Morton, Benjamin LaHaise; +Cc: Steven Cole, Linux Kernel, Alan Cox
On Wednesday 23 October 2002 22:09, Andrew Morton wrote:
> ext3's five-second commit interval and limited journal size really
> bite in this test. We end up doing most of the writeback within
> the measurement period rather than after it.
The test results should include *both* the time the tar completes
and the time a subsequent sync completes, to tell the whole story.
--
Daniel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious
2002-10-24 3:10 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Hans Reiser
@ 2002-10-24 9:54 ` Henning P. Schmiedehausen
2002-10-24 11:37 ` Padraig Brady
0 siblings, 1 reply; 11+ messages in thread
From: Henning P. Schmiedehausen @ 2002-10-24 9:54 UTC (permalink / raw)
To: linux-kernel
Hans Reiser <reiser@namesys.com> writes:
>simple tests like this. We recently ran into one with tar recognizing
>that it was writing to /dev/null, and optimizing for it.
As stated in the info document. It is there for a reason (Amanda).
--- cut ---
When the archive is being created to `/dev/null', GNU `tar' tries to
minimize input and output operations. The Amanda backup system, when
used with GNU `tar', has an initial sizing pass which uses this feature.
--- cut ---
Regards
Henning
>Hans
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH hps@intermeta.de
Am Schwabachgrund 22 Fon.: 09131 / 50654-0 info@intermeta.de
D-91054 Buckenhof Fax.: 09131 / 50654-20
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious
2002-10-24 9:54 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious Henning P. Schmiedehausen
@ 2002-10-24 11:37 ` Padraig Brady
0 siblings, 0 replies; 11+ messages in thread
From: Padraig Brady @ 2002-10-24 11:37 UTC (permalink / raw)
To: hps; +Cc: linux-kernel
Henning P. Schmiedehausen wrote:
> Hans Reiser <reiser@namesys.com> writes:
>
>>simple tests like this. We recently ran into one with tar recognizing
>>that it was writing to /dev/null, and optimizing for it.
>
> As stated in the info document. It is there for a reason (Amanda).
>
> --- cut ---
> When the archive is being created to `/dev/null', GNU `tar' tries to
> minimize input and output operations. The Amanda backup system, when
> used with GNU `tar', has an initial sizing pass which uses this feature.
> --- cut ---
IMHO /dev/null shouldn't be used for this. What's wrong
with Amanda doing: ln -s /dev/null /dev/drop
Then optimizing tars can use /dev/drop to not write()
and non-optimizing tars will still work as expected?
Pádraig.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2002-10-24 11:32 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-10-23 19:42 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared for various fs Steven Cole
2002-10-23 19:57 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Andrew Morton
2002-10-23 20:04 ` Benjamin LaHaise
2002-10-23 20:09 ` Andrew Morton
2002-10-24 8:34 ` Daniel Phillips
2002-10-23 20:05 ` Steven Cole
2002-10-23 20:32 ` Steven Cole
2002-10-23 20:44 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball comparedforvarious fs Andrew Morton
2002-10-24 3:10 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious fs Hans Reiser
2002-10-24 9:54 ` 2.5.44-[mm3, ac2] time to tar zxf kernel tarball compared forvarious Henning P. Schmiedehausen
2002-10-24 11:37 ` Padraig Brady
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.