RFC: KMV: nVMX: VMCS12 layout

RFC: KMV: nVMX: VMCS12 layout

[Jim Mattson]

In the five years since the original VMCS12 layout was defined, there
have been four incompatible changes to the layout. Technically, each
of these changes should have also changed the VMCS12_REVISION, but
none did. The original layout included several padding fields for
future expansion, but these fields have only been used in one of five
expansions.

Live migration will be much easier if the source and destination hosts
implement compatible VMCS12 layouts. New fields can either use
existing padding or they can be appended to the end of the existing
layout, but existing fields should not be shifted around. Otherwise,
we have to be more diligent about changing the VMCS12_REVISION when
incompatible changes are made to the layout, and the source and
destination hosts will have to implement the same VMCS12_REVISION. (It
is certainly conceivable that a host could implement multiple layouts,
but that would entail more work than just keeping the layouts
compatible.)

The question I have is whether or not the current VMCS12 layout is the
one that we want to bake in for all time. Other options that come to
mind are (1) matching some existing hardware layout (e.g. Broadwell),
or (2) creating a layout where the field offsets can be calculated
mathematically from the field encoding, rather than through a table
lookup. Perhaps there are other reasons for changing the current
layout that I'm not considering. Or, perhaps the benefits of any
change are too small to warrant the effort.

Comments?

Re: RFC: KMV: nVMX: VMCS12 layout

[Paolo Bonzini]

On 11/07/2016 18:12, Jim Mattson wrote:
> Live migration will be much easier if the source and destination hosts
> implement compatible VMCS12 layouts. New fields can either use
> existing padding or they can be appended to the end of the existing
> layout, but existing fields should not be shifted around. 

Correct.

> The question I have is whether or not the current VMCS12 layout is the
> one that we want to bake in for all time. Other options that come to
> mind are (1) matching some existing hardware layout (e.g. Broadwell),

No, I don't think this is necessary.  Hardware layout can change in the
future, and may not even match the operands of vmread and vmwrite.

> or (2) creating a layout where the field offsets can be calculated
> mathematically from the field encoding, rather than through a table
> lookup. Perhaps there are other reasons for changing the current
> layout that I'm not considering. Or, perhaps the benefits of any
> change are too small to warrant the effort.

This makes sense also because it speeds up vmread and vmwrite.  For
example you could map

	revision id	0x0
	abort		0x4
	launch state	0x8

followed by the VMCS fields:

	index		offset

	0x800-0x87e	0x100-0x17f	(2 bytes per field)
	0xc00-0xc7e	0x180-0x1ff

	0x2000-0x207f	0x200-0x3ff	(8 bytes per field)
	0x2400-0x243f	0x400-0x4ff
	0x2800-0x283f	0x500-0x5ff
	0x2c00-0x2c3f	0x600-0x6ff

	0x4000-0x407e	0x700-0x7ff	(4 bytes per field)
	0x4400-0x447e	0x800-0x8ff
	0x4800-0x487e	0x900-0x9ff
	0x4c00-0x4c7e	0xa00-0xaff

	0x6000-0x603e	0xb00-0xbff	(8 bytes per field)
	0x6400-0x643e	0xc00-0xcff
	0x6800-0x687e	0xd00-0xeff
	0x6c00-0x6c3e	0xf00-0xfff

Because similar fields have close indices, this might even give decent
cache locality.

Possibly, 0x80-0xff could be used for some kind of dirty bitmap to speed
up prepare_vmcs12.

You'd still need some kind of bitmap to distinguish supported VMCS
fields.  The hard part is not the layout, but writing the code so that
adding a new field is not unnecessarily hard.  Perhaps by writing a
small Perl script that generates the struct from vmx.h.  Otherwise it
would possibly be a very large change, because you'd have to remove the
vmcs12 struct completely and convert all accesses to vmcs12 so that they
go through an accessor.

Paolo