xen disk spec and emulation

xen disk spec and emulation

[Paul Durrant]

Hi,

  Currently, when specifying a virtual disk, the only way to determine the emulation model used is by selecting a particular 'vdev' numbering scheme as detailed in xl-disk-configuration in the docs. That document refers the reader to xen-vbd-interface which says:

xvd* -> xen virtual disk (from which one infers PV-only)
hd* -> IDE emulation
sd* -> SCSI emulation
d*p* -> unknown emulation

  The first choice actually results in IDE emulation, which is not stated anywhere AFAIK, and the fourth choice actually results in no emulation, although that's not stated either. These two things can, of course, be fixed in the documentation but I also think that the behaviour of vdev=xvd* is surprising and wrong. But there is another problem...

  Modern OS typically expect to use an NVMe device and we have no way to specify this kind of emulation. The obvious choice might be yet another vdev naming scheme, but this has a problem. The vdev name also dictates the vbd encoding (i.e. enumeration schemed used for PV devices), which seems to have little to do with the emulation model chosen. Adding a new encoding would also mean having to modify all PV frontends in existence to recognise it.

  My proposal to start to dig us out of this mess, is to add a new parameter for disks: emu=<model> where model would be 'ide', 'scsi', 'ahci' (for which we already have half-hearted and completely undocumented support), 'nvme', or 'none'. To maintain compatibility with existing brokenness, I proposed that this parameter only apples if diskspec is of the d*p* form and that we document all other forms as deprecated. I'm fairly sure that all existent PV frontends recognise xen virtual disk vbd encoding so I don't believe it would be necessary, for instance, for 'vdev=d0p0,emu=ide' to force hd encoding and so I think we should also consider deprecating all encodings other than xen virtual disk.

  Furthermore, the 'hvm-emulated-unplug' doc. specifies that emulated nvme devices are unplugged differently (using code 3) from IDE or SCSI (using code 0). I propose this also be changed so that code 0 unplugs all emulated disks (similarly to how 1 unplugs all emulated NICs). The unplug mechanism was never intended (I believe) to offer any finer grained control (although code 2 is somewhat of an anomaly) and I am not aware of any OS/driver that uses code 3. I propose this change, again, because we want to avoid having to modify all PV frontend code, which would be the case if NVMe devices has to be separately unplugged using code 3.

  Thoughts?

    Paul

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: xen disk spec and emulation

[Paul Durrant]

> -----Original Message-----
> From: Paul Durrant
> Sent: 27 March 2019 13:54
> To: xen-devel (xen-devel@lists.xenproject.org) <xen-devel@lists.xenproject.org>
> Subject: xen disk spec and emulation
> 
> Hi,
> 
>   Currently, when specifying a virtual disk, the only way to determine the emulation model used is by
> selecting a particular 'vdev' numbering scheme as detailed in xl-disk-configuration in the docs. That
> document refers the reader to xen-vbd-interface which says:
> 
> xvd* -> xen virtual disk (from which one infers PV-only)
> hd* -> IDE emulation
> sd* -> SCSI emulation
> d*p* -> unknown emulation
> 
>   The first choice actually results in IDE emulation, which is not stated anywhere AFAIK, and the
> fourth choice actually results in no emulation, although that's not stated either.

Actually I'll correct myself... d*p* also results in IDE emulation and so is equivalent to xvd. Thus in my proposal below a missing 'emu' parameter needs to imply 'emu=ide'.

  Paul

> These two things
> can, of course, be fixed in the documentation but I also think that the behaviour of vdev=xvd* is
> surprising and wrong. But there is another problem...
> 
>   Modern OS typically expect to use an NVMe device and we have no way to specify this kind of
> emulation. The obvious choice might be yet another vdev naming scheme, but this has a problem. The
> vdev name also dictates the vbd encoding (i.e. enumeration schemed used for PV devices), which seems
> to have little to do with the emulation model chosen. Adding a new encoding would also mean having to
> modify all PV frontends in existence to recognise it.
> 
>   My proposal to start to dig us out of this mess, is to add a new parameter for disks: emu=<model>
> where model would be 'ide', 'scsi', 'ahci' (for which we already have half-hearted and completely
> undocumented support), 'nvme', or 'none'. To maintain compatibility with existing brokenness, I
> proposed that this parameter only apples if diskspec is of the d*p* form and that we document all
> other forms as deprecated. I'm fairly sure that all existent PV frontends recognise xen virtual disk
> vbd encoding so I don't believe it would be necessary, for instance, for 'vdev=d0p0,emu=ide' to force
> hd encoding and so I think we should also consider deprecating all encodings other than xen virtual
> disk.
> 
>   Furthermore, the 'hvm-emulated-unplug' doc. specifies that emulated nvme devices are unplugged
> differently (using code 3) from IDE or SCSI (using code 0). I propose this also be changed so that
> code 0 unplugs all emulated disks (similarly to how 1 unplugs all emulated NICs). The unplug mechanism
> was never intended (I believe) to offer any finer grained control (although code 2 is somewhat of an
> anomaly) and I am not aware of any OS/driver that uses code 3. I propose this change, again, because
> we want to avoid having to modify all PV frontend code, which would be the case if NVMe devices has to
> be separately unplugged using code 3.
> 
>   Thoughts?
> 
>     Paul

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: xen disk spec and emulation

[Jan Beulich]

>>> On 27.03.19 at 14:54, <Paul.Durrant@citrix.com> wrote:
>   Currently, when specifying a virtual disk, the only way to determine the 
> emulation model used is by selecting a particular 'vdev' numbering scheme as 
> detailed in xl-disk-configuration in the docs. That document refers the 
> reader to xen-vbd-interface which says:
> 
> xvd* -> xen virtual disk (from which one infers PV-only)
> hd* -> IDE emulation
> sd* -> SCSI emulation
> d*p* -> unknown emulation
> 
>   The first choice actually results in IDE emulation, which is not stated 
> anywhere AFAIK, and the fourth choice actually results in no emulation, 
> although that's not stated either. These two things can, of course, be fixed 
> in the documentation but I also think that the behaviour of vdev=xvd* is 
> surprising and wrong.

Wasn't this behavior intended to allow a guest to be installed
without PV drivers, but to have them enabled once they got
installed?

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: xen disk spec and emulation

[Paul Durrant]

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: 27 March 2019 14:27
> To: Paul Durrant <Paul.Durrant@citrix.com>
> Cc: xen-devel <xen-devel@lists.xenproject.org>
> Subject: Re: [Xen-devel] xen disk spec and emulation
> 
> >>> On 27.03.19 at 14:54, <Paul.Durrant@citrix.com> wrote:
> >   Currently, when specifying a virtual disk, the only way to determine the
> > emulation model used is by selecting a particular 'vdev' numbering scheme as
> > detailed in xl-disk-configuration in the docs. That document refers the
> > reader to xen-vbd-interface which says:
> >
> > xvd* -> xen virtual disk (from which one infers PV-only)
> > hd* -> IDE emulation
> > sd* -> SCSI emulation
> > d*p* -> unknown emulation
> >
> >   The first choice actually results in IDE emulation, which is not stated
> > anywhere AFAIK, and the fourth choice actually results in no emulation,
> > although that's not stated either. These two things can, of course, be fixed
> > in the documentation but I also think that the behaviour of vdev=xvd* is
> > surprising and wrong.
> 
> Wasn't this behavior intended to allow a guest to be installed
> without PV drivers, but to have them enabled once they got
> installed?
> 

I don't think so. AIUI that's what hd* and sd* were for. They seem to be the only vdev types that are documented to have emulated equivalents.

I also appear to have been wrong about AHCI being completely undocumented. There is a line in xen-vbd-interface that specifies an extra 'hdtype=ahci' option, to add add ich9 controller emulation to the VM.

  Paul

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel