RE: Re: Reducing impact of save/restore/dump on Dom0

RE: Re: Reducing impact of save/restore/dump on Dom0

[Graham, Simon]

> > It's pretty easy for us to arrange that everything is page aligned.
> If
> > there was a measurable performance advantage using o_direct rather
> than
> > madvise/fadvise it probably makes sense to use it -- I can't see
> > o_direct going away any time soon.
> 
> O_DIRECT won't do any write behind, so unless you do very aggressive
IO
> (large IO requests, threads or aio so you do useful work during the
> disk write
> latency) it will be likely slower. Similar for read -- it won't do any
> readahead which you would need to do by yourself.
> 
> It's really not a very good idea for most non database applications.

Well, the dump/save/restore does do large IO requests for most of the
data. Also, it's a non-performance path - it's MUCH more important that
other things in Dom0 happen quickly (such as performing I/O for other
domains) so I would be quite happy with the save/restore/dump process
being a little slow in return for not destroying Dom0 performance (which
is what happens today).

Having said that, I understand that O_DIRECT is deprecated (by Linus at
least) and there is also the problem of it not being available on
Solaris; hence I am trying out the fsync/fadvise(DON'T_NEED) in the loop
after writing/reading a chunk of data.

Simon

Re: Re: Reducing impact of save/restore/dump on Dom0

[Andi Kleen]

> It's pretty easy for us to arrange that everything is page aligned. If
> there was a measurable performance advantage using o_direct rather than
> madvise/fadvise it probably makes sense to use it -- I can't see
> o_direct going away any time soon.

O_DIRECT won't do any write behind, so unless you do very aggressive IO
(large IO requests, threads or aio so you do useful work during the disk write
latency) it will be likely slower. Similar for read -- it won't do any
readahead which you would need to do by yourself.

It's really not a very good idea for most non database applications.

-Andi 

RE: Re: Reducing impact of save/restore/dump on Dom0

[Ian Pratt]

> > Currently, save, restore and dump all used cached I/O in Dom0 to
> > write/read the file containing the memory image of the DomU - when
the
> > memory assigned to the DomU is greater than free memory in Dom0,
this
> > leads to severe memory thrashing and generally the Dom0 performance
goes
> > into the toilet.
> >
> > The 'classic' answer to avoiding this when writing very large files
is,
> > of course, to use non-cached I/O to manipulate the files -
> 
> Otherwise you can just use madvise()/fadvise() to tell the kernel
> to drop the old data [the later might need a fairly recent kernel
> to work]
> 
> It has the advantage that it doesn't need much other changes.

It's pretty easy for us to arrange that everything is page aligned. If
there was a measurable performance advantage using o_direct rather than
madvise/fadvise it probably makes sense to use it -- I can't see
o_direct going away any time soon.

Ian