I was made aware of the fact that Xen apparently loses memory as soon as domUs are started. In my testing HVM domUs occupy much more memory than what was configured for them. The expected memory footprint for a PV domU seems to match the value in the domU config file.

My test host has 128GB and 8 cpus, dom0 is started with a fixed amount of memory. Before any domU is started, 1.4G are already "lost".

It seems for each HVM a certain amount of extra memory must be available. For a 100G HVM domU another 809M is required. With just 1 vcpu instead of 8 the amount of extra memory is reduced to 802M. With 32 vcpus it increases to 834M. Apparently each vcpu needs 1M extra memory.


Is there a formula to calculate that amount of extra memory, is this behavior documented somewhere?


Olaf

(XEN) System RAM: 131062MB (134208492kB)
 xl info | grep -i mem
total_memory           : 131062
free_memory            : 125551
xen_commandline        : loglvl=all guest_loglvl=all smt=1 console=com1 com1=57600 dom0_mem=4G

131062M - 125551M = 5511M used for just dom0
5511M - 4096M = 1415M lost?



pv domU, pvgrub2, mem=1024, vcpu=1
free_memory            : 124527
125551M - 124527M = 1024M, matches expectation


pv domU, pvgrub2, mem=65536, vcpu=8
free_memory            : 58990
124527M - 58990M = 65537M
65537M - 65536M = 1M extra?


fv domU, mem=32768, vcpu=8
free_memory            : 25957
58990M - 25957M = 33033M
33033M - 32768M =  256M extra?


stop all domUs
free_memory            : 125551


fv domU, mem=102400, vcpu=8
free_memory            : 22342
125551M - 22342M = 103209M
103209M - 102400M =  809M extra?


fv domU, mem=102400, vcpu=1
free_memory            : 22349
125551M - 22349M = 103202M
103202M - 102400M =  802M extra?