Re: Reducing impact of save/restore/dump on Dom0

[Iustin Pop <iusty@k1024.org>]

On Tue, Feb 06, 2007 at 12:46:52PM -0500, Graham, Simon wrote:
> > On linux, there is another way beside O_DIRECT: it's possible to
> > reduce
> > the memory used for cached writing using the sysctl vm.dirty_ratio;
> > I've
> > used this to reduce the impact of heavy cached writes on dom0 with
> > good
> > results.
> > 
> > Maybe it helps for save/dump also...
> 
> Oh interesting -- I shall look into this. 
> 
> I just took a quick peek and it is set to 40% in Dom0; I do see free
> memory go to zero during a dump (and save/restore) plus I see the
> waitIO% go to near 100% and Dom0 becomes somewhat unresponsive;
> specifically what we see is that domain boot fails during this time
> because of a XenBus timeout waiting for the hotplug scripts to finish
> adding the VBD in Dom0.

Yes, that's more or less expected. I've used 10% (you can't go below 5%
= harcoded limit in the kernel) and then, for a 512MB dom0, only ~25 MB
of cache will be used. I would hardly say that 25MB is too much.

> I still feel that dump/save/restore files really don't belong in the
> system cache at all since they just pollute the cache for no ggood
> reason.

Then there is also posix_fadvise, which is more or less what you need to
use in case you worry about your cache. I haven't used it, but I've
heard from people that using fadvise with POSIX_FADV_DONTNEED+fsync or
O_SYNC in batches can reduce your cache usage.

Just a few thoughts, as these don't change the way you do writes, as
opposed to O_DIRECT.

Regards,
Iustin