Testers wanted: dracut lazy install with cpio

Testers wanted: dracut lazy install with cpio

[Harald Hoyer]

Hi guys,

I made a patch for lazy installing the files with cpio.

I am interested what the difference in speed is, to evaluate, if it is worth the
hazzle.

The patch:
http://git.surfsite.org/?p=dracut.git;a=commit;h=df512139fe9c23b84a2213c934e7a7feb354bdcc

can be found on git://git.surfsite.org/pub/git/dracut.git branch:lazyinstall

I am interested in:

$ time sudo make testimage
$ time sudo make hostimage

Both with and without the patch.

Thanks for testing!

Re: Testers wanted: dracut lazy install with cpio

[John Reiser]

> $ time sudo make testimage
> $ time sudo make hostimage
> 
> Both with and without the patch.

I get (for Fedora 16):

            with patch (lazy)     without patch
   testimage:  24.3 real            25.8 real
               30.2 user            30.7 user
                4.5 sys              4.9 sys

   hostimage:   8.4 real             9.3 real
                8.1 user             8.4 user
                1.5 sys              1.9 real

Did the cpio run in parallel?  cpio reads from a pipe just fine.

cpio needs enhancement: a flag argument for "no overwrite",
and another for "clone the ownership and permissions on
created directories" [else a copied path "a/b/c/d/e" which
must create any of a, b, c, or d will not reproduce the
existing ownership and permissions on those directories.]

-- 

Re: Testers wanted: dracut lazy install with cpio

[Cong Wang]

On Mon, 27 Feb 2012 at 15:50 GMT, Harald Hoyer <harald.hoyer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> I am interested in:
>
> $ time sudo make testimage
> $ time sudo make hostimage
>
> Both with and without the patch.
>

[wangcong@cr0]~/dracut% git branch
  lazyinstall
  * master
[wangcong@cr0]~/dracut% time sudo make testimage
...
sudo make testimage  38.20s user 8.04s system 95% cpu 48.509 total

[wangcong@cr0]~/dracut% time sudo make hostimage
...
sudo make hostimage  11.69s user 4.00s system 101% cpu 15.401 total


[wangcong@cr0]~/dracut% git checkout lazyinstall
Switched to branch 'lazyinstall'
[wangcong@cr0]~/dracut% git branch
* lazyinstall
  master
[wangcong@cr0]~/dracut% time sudo make testimage
...
sudo make testimage  38.03s user 8.11s system 133% cpu 34.474 total

[wangcong@cr0]~/dracut% time sudo make hostimage
...
sudo make hostimage  11.36s user 3.45s system 109% cpu 13.478 total

Re: Testers wanted: dracut lazy install with cpio

[John Reiser]

On 02/28/2012 04:43 AM, Cong Wang wrote:

> [wangcong@cr0]~/dracut% git branch
>   lazyinstall
>   * master
> [wangcong@cr0]~/dracut% time sudo make testimage
> ...
> sudo make testimage  38.20s user 8.04s system 95% cpu 48.509 total

Was that run with a "cold cache"?  (What are the results for the
a second run on testimage, immediately after the first?)
I ask because the difference (48.5 - 34.5) between master and lazy
on testimage is much larger as a percentage than the (15.4 - 13.5)
difference for hostimage; and the testimage case was run first.

-- 

Re: Testers wanted: dracut lazy install with cpio

[Harald Hoyer]

Am 28.02.2012 13:43, schrieb Cong Wang:
> On Mon, 27 Feb 2012 at 15:50 GMT, Harald Hoyer <harald.hoyer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> I am interested in:
>>
>> $ time sudo make testimage
>> $ time sudo make hostimage
>>
>> Both with and without the patch.
>>
> 
> [wangcong@cr0]~/dracut% git branch
>   lazyinstall
>   * master
> [wangcong@cr0]~/dracut% time sudo make testimage
> ...
> sudo make testimage  38.20s user 8.04s system 95% cpu 48.509 total
> 
> [wangcong@cr0]~/dracut% time sudo make hostimage
> ...
> sudo make hostimage  11.69s user 4.00s system 101% cpu 15.401 total
> 
> 
> [wangcong@cr0]~/dracut% git checkout lazyinstall
> Switched to branch 'lazyinstall'
> [wangcong@cr0]~/dracut% git branch
> * lazyinstall
>   master
> [wangcong@cr0]~/dracut% time sudo make testimage
> ...
> sudo make testimage  38.03s user 8.11s system 133% cpu 34.474 total
> 
> [wangcong@cr0]~/dracut% time sudo make hostimage
> ...
> sudo make hostimage  11.36s user 3.45s system 109% cpu 13.478 total
> 

34s vs 48s
13s vs 15s

not sooo bad... but still not convinced

Re: Testers wanted: dracut lazy install with cpio

[Harald Hoyer]

Am 28.02.2012 16:09, schrieb John Reiser:
> On 02/28/2012 04:43 AM, Cong Wang wrote:
> 
>> [wangcong@cr0]~/dracut% git branch
>>   lazyinstall
>>   * master
>> [wangcong@cr0]~/dracut% time sudo make testimage
>> ...
>> sudo make testimage  38.20s user 8.04s system 95% cpu 48.509 total
> 
> Was that run with a "cold cache"?  (What are the results for the
> a second run on testimage, immediately after the first?)
> I ask because the difference (48.5 - 34.5) between master and lazy
> on testimage is much larger as a percentage than the (15.4 - 13.5)
> difference for hostimage; and the testimage case was run first.
> 

yes, "cold cache" should be ruled out.. better do several runs and pick the
fastest one.

Re: Testers wanted: dracut lazy install with cpio

[David Dillow]

On Tue, 2012-02-28 at 16:29 +0100, Harald Hoyer wrote:
> Am 28.02.2012 16:09, schrieb John Reiser:
> > On 02/28/2012 04:43 AM, Cong Wang wrote:
> > Was that run with a "cold cache"?  (What are the results for the
> > a second run on testimage, immediately after the first?)
> > I ask because the difference (48.5 - 34.5) between master and lazy
> > on testimage is much larger as a percentage than the (15.4 - 13.5)
> > difference for hostimage; and the testimage case was run first.
> > 
> 
> yes, "cold cache" should be ruled out.. better do several runs and pick the
> fastest one.

Is the purpose of this work to speed things up for the user, or for the
developers? Your approach is correct for developers, since we're more
likely to be making multiple dracut image with a warm cache. But users
are more likely to hit the cold cache case.

Re: Testers wanted: dracut lazy install with cpio

[John Reiser]

On 02/28/2012 08:27 AM, David Dillow wrote:

> Is the purpose of this work to speed things up for the user, or for the
> developers? Your approach is correct for developers, since we're more
> likely to be making multiple dracut image with a warm cache. But users
> are more likely to hit the cold cache case.

Both users and developers.  However, the cache must be controlled
(or known) for any measurement, else there is no way to attribute
the effects of any measured change: the cache effect could be much
larger than any improvement (or degredation).  The easiest way
is to have a warm cache: discard the first run of a series because
of the unknown cache state.

The cold cache case can be approximated by:
    # echo 1 > /proc/sys/vm/drop_caches
    # time dracut ...
    # echo 1 > /proc/sys/vm/drop_caches    # each time!
    # time dracut ...
but that path /proc/sys/vm/drop_caches is harder to remember
(and harder to get correct in a sudo!)
It is also harder to guarantee consistency on a machine that is
not otherwise idle.

Running dracut after updating kernel and/or drivers is likely
to hit a mixed cache: the updated stuff will tend to be hot,
the not-updated stuff probably will be cold.

-- 

Re: Testers wanted: dracut lazy install with cpio

[David Dillow]

On Tue, 2012-02-28 at 08:58 -0800, John Reiser wrote:
> On 02/28/2012 08:27 AM, David Dillow wrote:
> 
> > Is the purpose of this work to speed things up for the user, or for the
> > developers? Your approach is correct for developers, since we're more
> > likely to be making multiple dracut image with a warm cache. But users
> > are more likely to hit the cold cache case.
> 
> Both users and developers.  However, the cache must be controlled
> (or known) for any measurement, else there is no way to attribute
> the effects of any measured change: the cache effect could be much
> larger than any improvement (or degredation).  The easiest way
> is to have a warm cache: discard the first run of a series because
> of the unknown cache state.

Your commentary on the effects of caching and non-idle machines is true
and a useful reminder, but I think you talked past my point -- Harald
seemed to be wondering if the speedup was worth the complexity, and that
can only be answered by benchmarking it in the appropriate environment.
A fully warmed up cache is easiest and probably correct for developers,
but does not cover the case for users -- though if the speed up is good
there, then it is likely better for the users' case.

Your point about a mixed cache from an updated kernel is well taken;
perhaps a good way to replicate the user's experience would be to put
something like the following in a script -- so you don't have to
remember the path and get it right in a sudo:

for i in {1..10}; do
	sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"
	find /lib/modules/$(uname -r) -type f -print0 | xargs -0 cat > /dev/null
	time ...dracut...
done

and best if you do your testing from single user mode (may not even need
the sudo, then). It won't be perfect, but I think it would be more
meaningful than the easy case.

I tend to agree that the ~2 second improvements reported so far are not
compelling, but I see those as the developers case. I'm more interested
in the users' case -- is that 14 seconds Cong reported real?

Re: Testers wanted: dracut lazy install with cpio

[John Reiser]

On 02/28/2012 09:46 AM, David Dillow wrote:

> I tend to agree that the ~2 second improvements reported so far are not
> compelling, but I see those as the developers case. I'm more interested
> in the users' case -- is that 14 seconds Cong reported real?

I suspect that the 14 seconds difference was between a cold cache (first run)
and a warm cache (third run).  There wasn't anything non-real about it,
but the cache effect was much larger than the lazy/non-lazy algorithm.
The measurements that I reported yesterday were all warm-cache cases,
and I saw much closer improvement in wall-clock time (several percent)
for lazy, in both testimage and hostimage.  This is consistent with
the hypothesis that the improvement was due to the lazy case using one
(or a small handful) cpio of many files at a time, versus the non-lazy
case doing one /bin/cp for each file.  The savings is in reduced
usage of fork[clone]+execve+ld_linux.so+wait.

--