RE: [PATCH][TOOLS] Reducing impact ofdomainsave/restore/dump on Dom0

RE: [PATCH][TOOLS] Reducing impact ofdomainsave/restore/dump on Dom0

[Graham, Simon]

> > Reduce impact of saving/restoring/dumping large domains on Dom0
> memory
> > usage by means of fadvise64() to tell the OS to discard the cache
> pages
> > used for the save/dump file.
> >
> > Signed-off-by: Simon Graham <Simon.Graham@stratus.com>
> 
> Could the fadvise() logic be shared more than it is in this patch?
Also
> you
> sometimes sync-then-fadvise. Is this just a performance enhancement
> (presumably fadvise wouldn't simply discard dirty pages in the buffer
> cache)?
> 

1. You mean putting the fsync/fadvise in a common utility routine? I can
do that.

2. sync-then-advise is only done at the end of writing a file to ensure
that all
   of the cached pages are discarded. Whilst writing the file, I only
fadvise
   which triggers a write back and discards any clean pages up to the
specified offset.
   This is indeed a performance thing -- fsyncing on every write makes
it very slow.

Updated patch will follow...

Re: [PATCH][TOOLS] Reducing impact ofdomainsave/restore/dump on Dom0

[Keir Fraser]

On 21/2/07 15:06, "Graham, Simon" <Simon.Graham@stratus.com> wrote:

> 2. sync-then-advise is only done at the end of writing a file to ensure
> that all
>    of the cached pages are discarded. Whilst writing the file, I only
> fadvise
>    which triggers a write back and discards any clean pages up to the
> specified offset.
>    This is indeed a performance thing -- fsyncing on every write makes
> it very slow.

Do you need the fsync at all? It's possible that the kernel will
launder-then-discard the affected pages automatically, just from the
fadvise() alone.

 -- Keir

Re: [PATCH][TOOLS] Reducing impact ofdomainsave/restore/dump on Dom0

[Robert Read]

Hi,

I also read somewhere (either the man page or in the code) that the  
sync is necessary for exactly these reasons.

robert


On Feb 21, 2007, at 07:22, Graham, Simon wrote:

>
>
>>
>> Do you need the fsync at all? It's possible that the kernel will
>> launder-then-discard the affected pages automatically, just from the
>> fadvise() alone.
>
> To be honest, I'm not completely sure; the implementation of the  
> fadvise
> call triggers a write back and then explicitely discards any clean  
> pages
> -- I don't _think_ it will discard clean pages once the write back
> completes (but it's not entirely clear to me how much of the write  
> back
> is done synchronously).
>
> However, it definitely makes a perf difference if you explicitely  
> fsync
> before each call to fadvise -- therefore I believe that the fadvise  
> call
> is definitely not cleaning all the pages inline (which would be
> equivalent to the fsync()).
>
> So -- my belief is that without the fsync, there will be some (clean)
> pages for the file still in the cache when this is done; my goal  
> was to
> remove ALL the pages, but perhaps leaving a few lying around is OK..
>
> /simgr
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: [PATCH][TOOLS] Reducing impact ofdomainsave/restore/dump on Dom0

[Graham, Simon]

> 
> Do you need the fsync at all? It's possible that the kernel will
> launder-then-discard the affected pages automatically, just from the
> fadvise() alone.

To be honest, I'm not completely sure; the implementation of the fadvise
call triggers a write back and then explicitely discards any clean pages
-- I don't _think_ it will discard clean pages once the write back
completes (but it's not entirely clear to me how much of the write back
is done synchronously).

However, it definitely makes a perf difference if you explicitely fsync
before each call to fadvise -- therefore I believe that the fadvise call
is definitely not cleaning all the pages inline (which would be
equivalent to the fsync()).

So -- my belief is that without the fsync, there will be some (clean)
pages for the file still in the cache when this is done; my goal was to
remove ALL the pages, but perhaps leaving a few lying around is OK..

/simgr