Guest kernel device compatability auto-detection

Guest kernel device compatability auto-detection

[Sasha Levin]

Hi,

Currently when we run the guest we treat it as a black box, we're not
quite sure what it's going to start and whether it supports the same
features we expect it to support when running it from the host.

This forces us to start the guest with the safest defaults possible, for
example: '-drive file=my_image.qcow2' will be started with slow IDE
emulation even though the guest is capable of virtio.

I'm currently working on a method to try and detect whether the guest
kernel has specific configurations enabled and either warn the user if
we know the kernel is not going to properly work or use better defaults
if we know some advanced features are going to work.

How am I planning to do it? First, we'll try finding which kernel the
guest is going to boot (easy when user does '-kernel', less easy when
the user boots an image). For simplicity sake I'll stick with the
'-kernel' option for now.

Once we have the kernel we can do two things:
 1. See if the kernel was built with CONFIG_IKCONFIG.

 2. Try finding the System.map which belongs to the kernel, it's
provided with all distro kernels so we can expect it to be around. If we
did find it we repeat the same process as in #1.

If we found one of the above, we start matching config sets ("we need
a,b,c,d for virtio, let's see if it's all there"). Once we find a good
config set, we use it for defaults. If we didn't find a good config set
we warn the user and don't even bother starting the guest.

If we couldn't find either, we can just default to whatever we have as
defaults now.


To sum it up, I was wondering if this approach has been considered
before and whether it sounds interesting enough to try.

-- 

Sasha.

[Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

Hi,

Currently when we run the guest we treat it as a black box, we're not
quite sure what it's going to start and whether it supports the same
features we expect it to support when running it from the host.

This forces us to start the guest with the safest defaults possible, for
example: '-drive file=my_image.qcow2' will be started with slow IDE
emulation even though the guest is capable of virtio.

I'm currently working on a method to try and detect whether the guest
kernel has specific configurations enabled and either warn the user if
we know the kernel is not going to properly work or use better defaults
if we know some advanced features are going to work.

How am I planning to do it? First, we'll try finding which kernel the
guest is going to boot (easy when user does '-kernel', less easy when
the user boots an image). For simplicity sake I'll stick with the
'-kernel' option for now.

Once we have the kernel we can do two things:
 1. See if the kernel was built with CONFIG_IKCONFIG.

 2. Try finding the System.map which belongs to the kernel, it's
provided with all distro kernels so we can expect it to be around. If we
did find it we repeat the same process as in #1.

If we found one of the above, we start matching config sets ("we need
a,b,c,d for virtio, let's see if it's all there"). Once we find a good
config set, we use it for defaults. If we didn't find a good config set
we warn the user and don't even bother starting the guest.

If we couldn't find either, we can just default to whatever we have as
defaults now.


To sum it up, I was wondering if this approach has been considered
before and whether it sounds interesting enough to try.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Avi Kivity]

On 08/25/2011 08:21 AM, Sasha Levin wrote:
> Hi,
>
> Currently when we run the guest we treat it as a black box, we're not
> quite sure what it's going to start and whether it supports the same
> features we expect it to support when running it from the host.
>
> This forces us to start the guest with the safest defaults possible, for
> example: '-drive file=my_image.qcow2' will be started with slow IDE
> emulation even though the guest is capable of virtio.
>
> I'm currently working on a method to try and detect whether the guest
> kernel has specific configurations enabled and either warn the user if
> we know the kernel is not going to properly work or use better defaults
> if we know some advanced features are going to work.
>
> How am I planning to do it? First, we'll try finding which kernel the
> guest is going to boot (easy when user does '-kernel', less easy when
> the user boots an image). For simplicity sake I'll stick with the
> '-kernel' option for now.
>
> Once we have the kernel we can do two things:
>   1. See if the kernel was built with CONFIG_IKCONFIG.
>
>   2. Try finding the System.map which belongs to the kernel, it's
> provided with all distro kernels so we can expect it to be around. If we
> did find it we repeat the same process as in #1.
>
> If we found one of the above, we start matching config sets ("we need
> a,b,c,d for virtio, let's see if it's all there"). Once we find a good
> config set, we use it for defaults. If we didn't find a good config set
> we warn the user and don't even bother starting the guest.
>
> If we couldn't find either, we can just default to whatever we have as
> defaults now.
>
>
> To sum it up, I was wondering if this approach has been considered
> before and whether it sounds interesting enough to try.
>

This is a similar problem to p2v or v2v - taking a guest that used to 
run on physical or virtual hardware, and modifying it to run on 
(different) virtual hardware.  The first step is what you're looking for 
- detecting what the guest currently supports.

You can look at http://libguestfs.org/virt-v2v/ for an example.  I'm 
also copying Richard Jones, who maintains libguestfs, which does the 
actual poking around in the guest.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Thu, Aug 25, 2011 at 08:33:04AM +0300, Avi Kivity wrote:
> On 08/25/2011 08:21 AM, Sasha Levin wrote:
> >Hi,
> >
> >Currently when we run the guest we treat it as a black box, we're not
> >quite sure what it's going to start and whether it supports the same
> >features we expect it to support when running it from the host.
> >
> >This forces us to start the guest with the safest defaults possible, for
> >example: '-drive file=my_image.qcow2' will be started with slow IDE
> >emulation even though the guest is capable of virtio.
> >
> >I'm currently working on a method to try and detect whether the guest
> >kernel has specific configurations enabled and either warn the user if
> >we know the kernel is not going to properly work or use better defaults
> >if we know some advanced features are going to work.
> >
> >How am I planning to do it? First, we'll try finding which kernel the
> >guest is going to boot (easy when user does '-kernel', less easy when
> >the user boots an image). For simplicity sake I'll stick with the
> >'-kernel' option for now.
> >
> >Once we have the kernel we can do two things:
> >  1. See if the kernel was built with CONFIG_IKCONFIG.
> >
> >  2. Try finding the System.map which belongs to the kernel, it's
> >provided with all distro kernels so we can expect it to be around. If we
> >did find it we repeat the same process as in #1.
> >
> >If we found one of the above, we start matching config sets ("we need
> >a,b,c,d for virtio, let's see if it's all there"). Once we find a good
> >config set, we use it for defaults. If we didn't find a good config set
> >we warn the user and don't even bother starting the guest.
> >
> >If we couldn't find either, we can just default to whatever we have as
> >defaults now.
> >
> >
> >To sum it up, I was wondering if this approach has been considered
> >before and whether it sounds interesting enough to try.
> >
> 
> This is a similar problem to p2v or v2v - taking a guest that used
> to run on physical or virtual hardware, and modifying it to run on
> (different) virtual hardware.  The first step is what you're looking
> for - detecting what the guest currently supports.
> 
> You can look at http://libguestfs.org/virt-v2v/ for an example.  I'm
> also copying Richard Jones, who maintains libguestfs, which does the
> actual poking around in the guest.

Yes, as Avi says, we do all of the above already.  Including
for Windows guests.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Thu, Aug 25, 2011 at 08:33:04AM +0300, Avi Kivity wrote:
> On 08/25/2011 08:21 AM, Sasha Levin wrote:
> >Hi,
> >
> >Currently when we run the guest we treat it as a black box, we're not
> >quite sure what it's going to start and whether it supports the same
> >features we expect it to support when running it from the host.
> >
> >This forces us to start the guest with the safest defaults possible, for
> >example: '-drive file=my_image.qcow2' will be started with slow IDE
> >emulation even though the guest is capable of virtio.
> >
> >I'm currently working on a method to try and detect whether the guest
> >kernel has specific configurations enabled and either warn the user if
> >we know the kernel is not going to properly work or use better defaults
> >if we know some advanced features are going to work.
> >
> >How am I planning to do it? First, we'll try finding which kernel the
> >guest is going to boot (easy when user does '-kernel', less easy when
> >the user boots an image). For simplicity sake I'll stick with the
> >'-kernel' option for now.
> >
> >Once we have the kernel we can do two things:
> >  1. See if the kernel was built with CONFIG_IKCONFIG.
> >
> >  2. Try finding the System.map which belongs to the kernel, it's
> >provided with all distro kernels so we can expect it to be around. If we
> >did find it we repeat the same process as in #1.
> >
> >If we found one of the above, we start matching config sets ("we need
> >a,b,c,d for virtio, let's see if it's all there"). Once we find a good
> >config set, we use it for defaults. If we didn't find a good config set
> >we warn the user and don't even bother starting the guest.
> >
> >If we couldn't find either, we can just default to whatever we have as
> >defaults now.
> >
> >
> >To sum it up, I was wondering if this approach has been considered
> >before and whether it sounds interesting enough to try.
> >
> 
> This is a similar problem to p2v or v2v - taking a guest that used
> to run on physical or virtual hardware, and modifying it to run on
> (different) virtual hardware.  The first step is what you're looking
> for - detecting what the guest currently supports.
> 
> You can look at http://libguestfs.org/virt-v2v/ for an example.  I'm
> also copying Richard Jones, who maintains libguestfs, which does the
> actual poking around in the guest.

Yes, as Avi says, we do all of the above already.  Including
for Windows guests.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 08:32 +0100, Richard W.M. Jones wrote:
> On Thu, Aug 25, 2011 at 08:33:04AM +0300, Avi Kivity wrote:
> > On 08/25/2011 08:21 AM, Sasha Levin wrote:
> > >Hi,
> > >
> > >Currently when we run the guest we treat it as a black box, we're not
> > >quite sure what it's going to start and whether it supports the same
> > >features we expect it to support when running it from the host.
> > >
> > >This forces us to start the guest with the safest defaults possible, for
> > >example: '-drive file=my_image.qcow2' will be started with slow IDE
> > >emulation even though the guest is capable of virtio.
> > >
> > >I'm currently working on a method to try and detect whether the guest
> > >kernel has specific configurations enabled and either warn the user if
> > >we know the kernel is not going to properly work or use better defaults
> > >if we know some advanced features are going to work.
> > >
> > >How am I planning to do it? First, we'll try finding which kernel the
> > >guest is going to boot (easy when user does '-kernel', less easy when
> > >the user boots an image). For simplicity sake I'll stick with the
> > >'-kernel' option for now.
> > >
> > >Once we have the kernel we can do two things:
> > >  1. See if the kernel was built with CONFIG_IKCONFIG.
> > >
> > >  2. Try finding the System.map which belongs to the kernel, it's
> > >provided with all distro kernels so we can expect it to be around. If we
> > >did find it we repeat the same process as in #1.
> > >
> > >If we found one of the above, we start matching config sets ("we need
> > >a,b,c,d for virtio, let's see if it's all there"). Once we find a good
> > >config set, we use it for defaults. If we didn't find a good config set
> > >we warn the user and don't even bother starting the guest.
> > >
> > >If we couldn't find either, we can just default to whatever we have as
> > >defaults now.
> > >
> > >
> > >To sum it up, I was wondering if this approach has been considered
> > >before and whether it sounds interesting enough to try.
> > >
> > 
> > This is a similar problem to p2v or v2v - taking a guest that used
> > to run on physical or virtual hardware, and modifying it to run on
> > (different) virtual hardware.  The first step is what you're looking
> > for - detecting what the guest currently supports.
> > 
> > You can look at http://libguestfs.org/virt-v2v/ for an example.  I'm
> > also copying Richard Jones, who maintains libguestfs, which does the
> > actual poking around in the guest.
> 
> Yes, as Avi says, we do all of the above already.  Including
> for Windows guests.

>From what I gathered libguestfs only provides access to the guests'
image.

Which part is doing the IKCONFIG or System.map probing? Or is it done in
a different way?

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 08:32 +0100, Richard W.M. Jones wrote:
> On Thu, Aug 25, 2011 at 08:33:04AM +0300, Avi Kivity wrote:
> > On 08/25/2011 08:21 AM, Sasha Levin wrote:
> > >Hi,
> > >
> > >Currently when we run the guest we treat it as a black box, we're not
> > >quite sure what it's going to start and whether it supports the same
> > >features we expect it to support when running it from the host.
> > >
> > >This forces us to start the guest with the safest defaults possible, for
> > >example: '-drive file=my_image.qcow2' will be started with slow IDE
> > >emulation even though the guest is capable of virtio.
> > >
> > >I'm currently working on a method to try and detect whether the guest
> > >kernel has specific configurations enabled and either warn the user if
> > >we know the kernel is not going to properly work or use better defaults
> > >if we know some advanced features are going to work.
> > >
> > >How am I planning to do it? First, we'll try finding which kernel the
> > >guest is going to boot (easy when user does '-kernel', less easy when
> > >the user boots an image). For simplicity sake I'll stick with the
> > >'-kernel' option for now.
> > >
> > >Once we have the kernel we can do two things:
> > >  1. See if the kernel was built with CONFIG_IKCONFIG.
> > >
> > >  2. Try finding the System.map which belongs to the kernel, it's
> > >provided with all distro kernels so we can expect it to be around. If we
> > >did find it we repeat the same process as in #1.
> > >
> > >If we found one of the above, we start matching config sets ("we need
> > >a,b,c,d for virtio, let's see if it's all there"). Once we find a good
> > >config set, we use it for defaults. If we didn't find a good config set
> > >we warn the user and don't even bother starting the guest.
> > >
> > >If we couldn't find either, we can just default to whatever we have as
> > >defaults now.
> > >
> > >
> > >To sum it up, I was wondering if this approach has been considered
> > >before and whether it sounds interesting enough to try.
> > >
> > 
> > This is a similar problem to p2v or v2v - taking a guest that used
> > to run on physical or virtual hardware, and modifying it to run on
> > (different) virtual hardware.  The first step is what you're looking
> > for - detecting what the guest currently supports.
> > 
> > You can look at http://libguestfs.org/virt-v2v/ for an example.  I'm
> > also copying Richard Jones, who maintains libguestfs, which does the
> > actual poking around in the guest.
> 
> Yes, as Avi says, we do all of the above already.  Including
> for Windows guests.

>From what I gathered libguestfs only provides access to the guests'
image.

Which part is doing the IKCONFIG or System.map probing? Or is it done in
a different way?

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Thu, Aug 25, 2011 at 10:40:34AM +0300, Sasha Levin wrote:
> From what I gathered libguestfs only provides access to the guests'
> image.

Correct.

> Which part is doing the IKCONFIG or System.map probing? Or is it done in
> a different way?

You'll have to see what Matt's doing in the virt-v2v code for the
details, but in general we have full access to:

 - grub.conf (to determine which kernel will boot)
 - the kernel image
 - the corresponding System.map and config
 - the modules directory
 - the Xorg config
 - boot.ini or BCD (to determine which NT kernel will boot)
 - the Windows Registry
 - the list of packages installed (to see if VMware-tools or some other
   guest agent is installed)

So working out what drivers are available is just a tedious matter of
iterating across each of these places in the filesystem.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Thu, Aug 25, 2011 at 08:48:25AM +0100, Richard W.M. Jones wrote:
> On Thu, Aug 25, 2011 at 10:40:34AM +0300, Sasha Levin wrote:
> > From what I gathered libguestfs only provides access to the guests'
> > image.
> 
> Correct.
> 
> > Which part is doing the IKCONFIG or System.map probing? Or is it done in
> > a different way?
> 
> You'll have to see what Matt's doing in the virt-v2v code for the
> details, but in general we have full access to:
> 
>  - grub.conf (to determine which kernel will boot)
>  - the kernel image
>  - the corresponding System.map and config
>  - the modules directory
>  - the Xorg config
>  - boot.ini or BCD (to determine which NT kernel will boot)
>  - the Windows Registry
>  - the list of packages installed (to see if VMware-tools or some other
>    guest agent is installed)
> 
> So working out what drivers are available is just a tedious matter of
> iterating across each of these places in the filesystem.

We had some interesting discussion on IRC about this.

Detecting if a guest "supports virtio" is a tricky problem, and it
goes beyond what the guest kernel can do.  For Linux guests you also
need to check what userspace can do.  This means unpacking the initrd
and checking for virtio drivers [in the general case this is
intractable, but you can do it for specific distros].

You also need to check that udev has the correct rules and that LVM is
configured to see VGs on /dev/vd* devices.

Console and Xorg configuration may also need to be checked (for
virtio-console and Cirrus/QXL support resp.)

virt-v2v does quite a lot of work to *enable* virtio drivers
including:

 - possibly installing a new kernel and updating grub

 - rebuilding the initrd to include virtio drivers

 - adjusting many different config files

 - removing other guest tools and Xen drivers

 - reconfiguring SELinux

 - adding viostor driver to Windows and adjusting the Windows Registry
   Critical Device Database

Of course virt-v2v confines itself to specific known guests, and we
test it like crazy.

Here is the code:

http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/RedHat.pm;hb=HEAD
http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/Windows.pm;hb=HEAD

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Re: Guest kernel device compatability auto-detection

[Decker, Schorschi]

>From a security perspective, this not a great idea.  Security isolation in virtualization is gaining ground, so anything that breaches the hypervisor/guest vale is by your typical enterprise/company security team considered completely illegal, a number of firms I have talked with all are talking about how their respective security teams are raising all kinds of hell/red flags, demanding disablement of features that breach the vale.

I would ask two things be done in the design if it goes forward, 1) have an explicit way to disable this feature, where the hypervisor cannot interact with the guest OS directly in any way if disablement is selected.  2) implement the feature as an agent in the guest OS where the hypervisor can only query the guest OS agent, using a standard TCP/IP methodology.  Any under the hood, under the covers methodology, I can tell you for a fact, security teams will quash, or demand the feature is disabled.  This goes for VM to VM communication as well that does not use a formal TCP/IP stack based method as well.  Security teams want true OS isolation, point blank.


Schorschi Decker

VP; Sr. Consultant Engineer
ECT&O Emerging Technologies / Virtualization Platform Engineering Team
Bank of America

Office 213-345-4714



-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Richard W.M. Jones
Sent: Thursday, 25 August, 2011 03:01
To: qemu-devel@nongnu.org
Cc: Avi Kivity; kvm
Subject: Re: [Qemu-devel] Guest kernel device compatability auto-detection

On Thu, Aug 25, 2011 at 08:48:25AM +0100, Richard W.M. Jones wrote:
> On Thu, Aug 25, 2011 at 10:40:34AM +0300, Sasha Levin wrote:
> > From what I gathered libguestfs only provides access to the guests'
> > image.
> 
> Correct.
> 
> > Which part is doing the IKCONFIG or System.map probing? Or is it 
> > done in a different way?
> 
> You'll have to see what Matt's doing in the virt-v2v code for the 
> details, but in general we have full access to:
> 
>  - grub.conf (to determine which kernel will boot)
>  - the kernel image
>  - the corresponding System.map and config
>  - the modules directory
>  - the Xorg config
>  - boot.ini or BCD (to determine which NT kernel will boot)
>  - the Windows Registry
>  - the list of packages installed (to see if VMware-tools or some other
>    guest agent is installed)
> 
> So working out what drivers are available is just a tedious matter of 
> iterating across each of these places in the filesystem.

We had some interesting discussion on IRC about this.

Detecting if a guest "supports virtio" is a tricky problem, and it goes beyond what the guest kernel can do.  For Linux guests you also need to check what userspace can do.  This means unpacking the initrd and checking for virtio drivers [in the general case this is intractable, but you can do it for specific distros].

You also need to check that udev has the correct rules and that LVM is configured to see VGs on /dev/vd* devices.

Console and Xorg configuration may also need to be checked (for virtio-console and Cirrus/QXL support resp.)

virt-v2v does quite a lot of work to *enable* virtio drivers
including:

 - possibly installing a new kernel and updating grub

 - rebuilding the initrd to include virtio drivers

 - adjusting many different config files

 - removing other guest tools and Xen drivers

 - reconfiguring SELinux

 - adding viostor driver to Windows and adjusting the Windows Registry
   Critical Device Database

Of course virt-v2v confines itself to specific known guests, and we test it like crazy.

Here is the code:

http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/RedHat.pm;hb=HEAD
http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/Windows.pm;hb=HEAD

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

----------------------------------------------------------------------
This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. 
Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. 
The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. 

References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: 
http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Decker, Schorschi]

>From a security perspective, this not a great idea.  Security isolation in virtualization is gaining ground, so anything that breaches the hypervisor/guest vale is by your typical enterprise/company security team considered completely illegal, a number of firms I have talked with all are talking about how their respective security teams are raising all kinds of hell/red flags, demanding disablement of features that breach the vale.

I would ask two things be done in the design if it goes forward, 1) have an explicit way to disable this feature, where the hypervisor cannot interact with the guest OS directly in any way if disablement is selected.  2) implement the feature as an agent in the guest OS where the hypervisor can only query the guest OS agent, using a standard TCP/IP methodology.  Any under the hood, under the covers methodology, I can tell you for a fact, security teams will quash, or demand the feature is disabled.  This goes for VM to VM communication as well that does not use a formal TCP/IP stack based method as well.  Security teams want true OS isolation, point blank.


Schorschi Decker

VP; Sr. Consultant Engineer
ECT&O Emerging Technologies / Virtualization Platform Engineering Team
Bank of America

Office 213-345-4714



-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Richard W.M. Jones
Sent: Thursday, 25 August, 2011 03:01
To: qemu-devel@nongnu.org
Cc: Avi Kivity; kvm
Subject: Re: [Qemu-devel] Guest kernel device compatability auto-detection

On Thu, Aug 25, 2011 at 08:48:25AM +0100, Richard W.M. Jones wrote:
> On Thu, Aug 25, 2011 at 10:40:34AM +0300, Sasha Levin wrote:
> > From what I gathered libguestfs only provides access to the guests'
> > image.
> 
> Correct.
> 
> > Which part is doing the IKCONFIG or System.map probing? Or is it 
> > done in a different way?
> 
> You'll have to see what Matt's doing in the virt-v2v code for the 
> details, but in general we have full access to:
> 
>  - grub.conf (to determine which kernel will boot)
>  - the kernel image
>  - the corresponding System.map and config
>  - the modules directory
>  - the Xorg config
>  - boot.ini or BCD (to determine which NT kernel will boot)
>  - the Windows Registry
>  - the list of packages installed (to see if VMware-tools or some other
>    guest agent is installed)
> 
> So working out what drivers are available is just a tedious matter of 
> iterating across each of these places in the filesystem.

We had some interesting discussion on IRC about this.

Detecting if a guest "supports virtio" is a tricky problem, and it goes beyond what the guest kernel can do.  For Linux guests you also need to check what userspace can do.  This means unpacking the initrd and checking for virtio drivers [in the general case this is intractable, but you can do it for specific distros].

You also need to check that udev has the correct rules and that LVM is configured to see VGs on /dev/vd* devices.

Console and Xorg configuration may also need to be checked (for virtio-console and Cirrus/QXL support resp.)

virt-v2v does quite a lot of work to *enable* virtio drivers
including:

 - possibly installing a new kernel and updating grub

 - rebuilding the initrd to include virtio drivers

 - adjusting many different config files

 - removing other guest tools and Xen drivers

 - reconfiguring SELinux

 - adding viostor driver to Windows and adjusting the Windows Registry
   Critical Device Database

Of course virt-v2v confines itself to specific known guests, and we test it like crazy.

Here is the code:

http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/RedHat.pm;hb=HEAD
http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtConvert/Converter/Windows.pm;hb=HEAD

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

----------------------------------------------------------------------
This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. 
Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. 
The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. 

References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: 
http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> I would ask two things be done in the design if it goes forward, 1)
> have an explicit way to disable this feature, where the hypervisor
> cannot interact with the guest OS directly in any way if disablement
> is selected.

I doubt that this (or anything similar) introduced will even be set to
on by default. It has the potential of breaking stuff that would work
otherwise (thats why the default boot is with the safest configuration
possible).

On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> 2) implement the feature as an agent in the guest OS where the
> hypervisor can only query the guest OS agent, using a standard TCP/IP
> methodology.

I was planning to implementing it by probing the image before actually
booting it.
This process is completely offline and doesn't require interaction with
the guest. The guest isn't even running at that point.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> I would ask two things be done in the design if it goes forward, 1)
> have an explicit way to disable this feature, where the hypervisor
> cannot interact with the guest OS directly in any way if disablement
> is selected.

I doubt that this (or anything similar) introduced will even be set to
on by default. It has the potential of breaking stuff that would work
otherwise (thats why the default boot is with the safest configuration
possible).

On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> 2) implement the feature as an agent in the guest OS where the
> hypervisor can only query the guest OS agent, using a standard TCP/IP
> methodology.

I was planning to implementing it by probing the image before actually
booting it.
This process is completely offline and doesn't require interaction with
the guest. The guest isn't even running at that point.

-- 

Sasha.

Re: Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > 2) implement the feature as an agent in the guest OS where the
> > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > methodology.
>
> I was planning to implementing it by probing the image before
> actually booting it.  This process is completely offline and doesn't
> require interaction with the guest. The guest isn't even running at
> that point.

There are still plenty of security issues to be concerned about with
handling an offline guest.  It is quite possible for such a guest to
be booby-trapped in a way that allows an exploit.  I summarised some
of the issues I thought about here, but there are likely to be others:

http://libguestfs.org/guestfs.3.html#security

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > 2) implement the feature as an agent in the guest OS where the
> > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > methodology.
>
> I was planning to implementing it by probing the image before
> actually booting it.  This process is completely offline and doesn't
> require interaction with the guest. The guest isn't even running at
> that point.

There are still plenty of security issues to be concerned about with
handling an offline guest.  It is quite possible for such a guest to
be booby-trapped in a way that allows an exploit.  I summarised some
of the issues I thought about here, but there are likely to be others:

http://libguestfs.org/guestfs.3.html#security

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Fri, 2011-08-26 at 09:04 +0100, Richard W.M. Jones wrote:
> On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> > On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > > 2) implement the feature as an agent in the guest OS where the
> > > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > > methodology.
> >
> > I was planning to implementing it by probing the image before
> > actually booting it.  This process is completely offline and doesn't
> > require interaction with the guest. The guest isn't even running at
> > that point.
> 
> There are still plenty of security issues to be concerned about with
> handling an offline guest.  It is quite possible for such a guest to
> be booby-trapped in a way that allows an exploit.  I summarised some
> of the issues I thought about here, but there are likely to be others:
> 
> http://libguestfs.org/guestfs.3.html#security

That was an interesting read.

Are the concerns still valid if we were going to boot the image anyway
later on?

I'm assuming that probing would happen only before we're trying to boot
a guest, and not just probe any image file we find.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Fri, 2011-08-26 at 09:04 +0100, Richard W.M. Jones wrote:
> On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> > On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > > 2) implement the feature as an agent in the guest OS where the
> > > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > > methodology.
> >
> > I was planning to implementing it by probing the image before
> > actually booting it.  This process is completely offline and doesn't
> > require interaction with the guest. The guest isn't even running at
> > that point.
> 
> There are still plenty of security issues to be concerned about with
> handling an offline guest.  It is quite possible for such a guest to
> be booby-trapped in a way that allows an exploit.  I summarised some
> of the issues I thought about here, but there are likely to be others:
> 
> http://libguestfs.org/guestfs.3.html#security

That was an interesting read.

Are the concerns still valid if we were going to boot the image anyway
later on?

I'm assuming that probing would happen only before we're trying to boot
a guest, and not just probe any image file we find.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 01:18:49PM +0300, Sasha Levin wrote:
> On Fri, 2011-08-26 at 09:04 +0100, Richard W.M. Jones wrote:
> > On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> > > On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > > > 2) implement the feature as an agent in the guest OS where the
> > > > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > > > methodology.
> > >
> > > I was planning to implementing it by probing the image before
> > > actually booting it.  This process is completely offline and doesn't
> > > require interaction with the guest. The guest isn't even running at
> > > that point.
> > 
> > There are still plenty of security issues to be concerned about with
> > handling an offline guest.  It is quite possible for such a guest to
> > be booby-trapped in a way that allows an exploit.  I summarised some
> > of the issues I thought about here, but there are likely to be others:
> > 
> > http://libguestfs.org/guestfs.3.html#security
> 
> That was an interesting read.
> 
> Are the concerns still valid if we were going to boot the image anyway
> later on?

Yes.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 01:18:49PM +0300, Sasha Levin wrote:
> On Fri, 2011-08-26 at 09:04 +0100, Richard W.M. Jones wrote:
> > On Fri, Aug 26, 2011 at 09:22:45AM +0300, Sasha Levin wrote:
> > > On Thu, 2011-08-25 at 16:25 +0000, Decker, Schorschi wrote:
> > > > 2) implement the feature as an agent in the guest OS where the
> > > > hypervisor can only query the guest OS agent, using a standard TCP/IP
> > > > methodology.
> > >
> > > I was planning to implementing it by probing the image before
> > > actually booting it.  This process is completely offline and doesn't
> > > require interaction with the guest. The guest isn't even running at
> > > that point.
> > 
> > There are still plenty of security issues to be concerned about with
> > handling an offline guest.  It is quite possible for such a guest to
> > be booby-trapped in a way that allows an exploit.  I summarised some
> > of the issues I thought about here, but there are likely to be others:
> > 
> > http://libguestfs.org/guestfs.3.html#security
> 
> That was an interesting read.
> 
> Are the concerns still valid if we were going to boot the image anyway
> later on?

Yes.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: Guest kernel device compatability auto-detection

[Anthony Liguori]

On 08/25/2011 12:21 AM, Sasha Levin wrote:
> Hi,
>
> Currently when we run the guest we treat it as a black box, we're not
> quite sure what it's going to start and whether it supports the same
> features we expect it to support when running it from the host.
>
> This forces us to start the guest with the safest defaults possible, for
> example: '-drive file=my_image.qcow2' will be started with slow IDE
> emulation even though the guest is capable of virtio.
>
> I'm currently working on a method to try and detect whether the guest
> kernel has specific configurations enabled and either warn the user if
> we know the kernel is not going to properly work or use better defaults
> if we know some advanced features are going to work.
>
> How am I planning to do it? First, we'll try finding which kernel the
> guest is going to boot (easy when user does '-kernel', less easy when
> the user boots an image). For simplicity sake I'll stick with the
> '-kernel' option for now.

Is the problem you're trying to solve determine whether the guest kernel 
is going to work well under kvm tool or trying to choose the right 
hardware profile to expose to the guest?

If it's the former, I think the path you're heading down is the most 
likely to succeed (trying to guess based on what you can infer about the 
kernel).

If it's the later, there's some interesting possibilities we never fully 
explored in QEMU.

One would be exposing a well supported device (like IDE emulation) and 
having a magic mode that allowed you to basically promote the device 
from IDE emulation to virtio-blk.  Likewise, you could do something like 
that to promote from the e1000 to virtio-net.

It might require some special support in the guest kernel and would 
likely be impossible to do in Windows, but if you primarily care about 
Linux guests, it ought to be possible.

Regards,

Anthony Liguori

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Anthony Liguori]

On 08/25/2011 12:21 AM, Sasha Levin wrote:
> Hi,
>
> Currently when we run the guest we treat it as a black box, we're not
> quite sure what it's going to start and whether it supports the same
> features we expect it to support when running it from the host.
>
> This forces us to start the guest with the safest defaults possible, for
> example: '-drive file=my_image.qcow2' will be started with slow IDE
> emulation even though the guest is capable of virtio.
>
> I'm currently working on a method to try and detect whether the guest
> kernel has specific configurations enabled and either warn the user if
> we know the kernel is not going to properly work or use better defaults
> if we know some advanced features are going to work.
>
> How am I planning to do it? First, we'll try finding which kernel the
> guest is going to boot (easy when user does '-kernel', less easy when
> the user boots an image). For simplicity sake I'll stick with the
> '-kernel' option for now.

Is the problem you're trying to solve determine whether the guest kernel 
is going to work well under kvm tool or trying to choose the right 
hardware profile to expose to the guest?

If it's the former, I think the path you're heading down is the most 
likely to succeed (trying to guess based on what you can infer about the 
kernel).

If it's the later, there's some interesting possibilities we never fully 
explored in QEMU.

One would be exposing a well supported device (like IDE emulation) and 
having a magic mode that allowed you to basically promote the device 
from IDE emulation to virtio-blk.  Likewise, you could do something like 
that to promote from the e1000 to virtio-net.

It might require some special support in the guest kernel and would 
likely be impossible to do in Windows, but if you primarily care about 
Linux guests, it ought to be possible.

Regards,

Anthony Liguori

Re: Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 16:48 -0500, Anthony Liguori wrote:
> On 08/25/2011 12:21 AM, Sasha Levin wrote:
> > Hi,
> >
> > Currently when we run the guest we treat it as a black box, we're not
> > quite sure what it's going to start and whether it supports the same
> > features we expect it to support when running it from the host.
> >
> > This forces us to start the guest with the safest defaults possible, for
> > example: '-drive file=my_image.qcow2' will be started with slow IDE
> > emulation even though the guest is capable of virtio.
> >
> > I'm currently working on a method to try and detect whether the guest
> > kernel has specific configurations enabled and either warn the user if
> > we know the kernel is not going to properly work or use better defaults
> > if we know some advanced features are going to work.
> >
> > How am I planning to do it? First, we'll try finding which kernel the
> > guest is going to boot (easy when user does '-kernel', less easy when
> > the user boots an image). For simplicity sake I'll stick with the
> > '-kernel' option for now.
> 
> Is the problem you're trying to solve determine whether the guest kernel 
> is going to work well under kvm tool or trying to choose the right 
> hardware profile to expose to the guest?
> 
> If it's the former, I think the path you're heading down is the most 
> likely to succeed (trying to guess based on what you can infer about the 
> kernel).
> 
> If it's the later, there's some interesting possibilities we never fully 
> explored in QEMU.
> 

I was thinking about both, I've considered kvm tools to be the 'easy'
case where we only say if it would work or not, and QEMU as the hard one
where we need to build a working configuration.

> One would be exposing a well supported device (like IDE emulation) and 
> having a magic mode that allowed you to basically promote the device 
> from IDE emulation to virtio-blk.  Likewise, you could do something like 
> that to promote from the e1000 to virtio-net.
> 
> It might require some special support in the guest kernel and would 
> likely be impossible to do in Windows, but if you primarily care about 
> Linux guests, it ought to be possible.

You're thinking about trying to expose all interfaces during boot and
seeing which ones the kernel bites?

Another thing that comes to mind is that we could start this project
with a script that given a kernel, it would find the optimal hardware
configuration for it (and the matching QEMU command line).

It would simply work the other way around: try booting with the best
devices first, if it doesn't boot we would 'demote' them one at a time
until the kernel does boot.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Sasha Levin]

On Thu, 2011-08-25 at 16:48 -0500, Anthony Liguori wrote:
> On 08/25/2011 12:21 AM, Sasha Levin wrote:
> > Hi,
> >
> > Currently when we run the guest we treat it as a black box, we're not
> > quite sure what it's going to start and whether it supports the same
> > features we expect it to support when running it from the host.
> >
> > This forces us to start the guest with the safest defaults possible, for
> > example: '-drive file=my_image.qcow2' will be started with slow IDE
> > emulation even though the guest is capable of virtio.
> >
> > I'm currently working on a method to try and detect whether the guest
> > kernel has specific configurations enabled and either warn the user if
> > we know the kernel is not going to properly work or use better defaults
> > if we know some advanced features are going to work.
> >
> > How am I planning to do it? First, we'll try finding which kernel the
> > guest is going to boot (easy when user does '-kernel', less easy when
> > the user boots an image). For simplicity sake I'll stick with the
> > '-kernel' option for now.
> 
> Is the problem you're trying to solve determine whether the guest kernel 
> is going to work well under kvm tool or trying to choose the right 
> hardware profile to expose to the guest?
> 
> If it's the former, I think the path you're heading down is the most 
> likely to succeed (trying to guess based on what you can infer about the 
> kernel).
> 
> If it's the later, there's some interesting possibilities we never fully 
> explored in QEMU.
> 

I was thinking about both, I've considered kvm tools to be the 'easy'
case where we only say if it would work or not, and QEMU as the hard one
where we need to build a working configuration.

> One would be exposing a well supported device (like IDE emulation) and 
> having a magic mode that allowed you to basically promote the device 
> from IDE emulation to virtio-blk.  Likewise, you could do something like 
> that to promote from the e1000 to virtio-net.
> 
> It might require some special support in the guest kernel and would 
> likely be impossible to do in Windows, but if you primarily care about 
> Linux guests, it ought to be possible.

You're thinking about trying to expose all interfaces during boot and
seeing which ones the kernel bites?

Another thing that comes to mind is that we could start this project
with a script that given a kernel, it would find the optimal hardware
configuration for it (and the matching QEMU command line).

It would simply work the other way around: try booting with the best
devices first, if it doesn't boot we would 'demote' them one at a time
until the kernel does boot.

-- 

Sasha.

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 09:08:02AM +0300, Sasha Levin wrote:
> You're thinking about trying to expose all interfaces during boot and
> seeing which ones the kernel bites?

No, that's a bad idea.  A current guest would register that as two
disks.  It might even try to write to them independently.

You need an IDE device which can be promoted to virtio at the request
of the guest (so you know the guest understands this type of device).

> Another thing that comes to mind is that we could start this project
> with a script that given a kernel, it would find the optimal hardware
> configuration for it (and the matching QEMU command line).

It would be better if the guest could communicate what it needs.  But
no current guest works that way.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Fri, Aug 26, 2011 at 09:08:02AM +0300, Sasha Levin wrote:
> You're thinking about trying to expose all interfaces during boot and
> seeing which ones the kernel bites?

No, that's a bad idea.  A current guest would register that as two
disks.  It might even try to write to them independently.

You need an IDE device which can be promoted to virtio at the request
of the guest (so you know the guest understands this type of device).

> Another thing that comes to mind is that we could start this project
> with a script that given a kernel, it would find the optimal hardware
> configuration for it (and the matching QEMU command line).

It would be better if the guest could communicate what it needs.  But
no current guest works that way.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] Guest kernel device compatability auto-detection

[Richard W.M. Jones]

On Thu, Aug 25, 2011 at 04:48:36PM -0500, Anthony Liguori wrote:
> One would be exposing a well supported device (like IDE emulation)
> and having a magic mode that allowed you to basically promote the
> device from IDE emulation to virtio-blk.  Likewise, you could do
> something like that to promote from the e1000 to virtio-net.

+1 .. this is what VMware does AIUI.

Another way might be to invent a "virtio-IDE" device, which at one
level looks like ordinary IDE but has non-standard extensions that are
helpful for paravirt.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw