[Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

Hi,

On last week's call we discussed the issue of splitting non core
features of QEMU into it's own process to reduce the security risks etc.

I wrote up a summary of my thoughts on this to try to cover the various
issues. Feedback welcome and hopefully we can continue the discussion on
a future call - maybe next week?

I would like to be part of the discussion, but it's a public holiday
here March 1st, so I won't be on that call.

Cheers,
Jes


Separating host-side virtagent and other tasks from core QEMU
=============================================================

To improve auditing of the core QEMU code, it would be ideal to be
able to separate the core QEMU functionality from utility
functionality by  moving the utility functionality into its own
process. This process will be referred to as the QEMU client below.

Components which are candidates for moving out of QEMU include:
 - virtagent
 - vnc server (and other graphical displays such as SDL, spice and
   curses)
 - human monitor

The idea is to have QEMU launch as a daemon, and then allow for one of
more client processes to connect to it. These will then offer the
various services. The main issue to discuss is how to handle various
state information, reconnects, and migration.

Security
========

The primary reason for this discussion is security, however there are
other benefits from this split, as will be mentioned below. During a
demo of virtagent I hit a case where a bug in the agent command
handling code caused a crash of the host QEMU process. While it is
probably a simple bug, it shows how adding more complexity to the QEMU
process increases the risk of adding security problems that could
potentially be exploited by a hostile guest.

By splitting non core functionality into a QEMU client process, the
host process will be isolated from a large number of potential
security problems, ie. in case a client process is killed or crashes,
it should not affect the main QEMU process. In addition it makes it
easier to audit the core QEMU functionality.

virtagent
=========

In short virtagent provides a set of simple commands, most of which
do not have state associated with them. These include shutdown, ping,
fsfreeze/fsthaw, etc. Other commands might be multi-stage commands
which could fail if the client is disconnected from the daemon while
the command is in progress. These include copy-paste and file copy.

vnc server
==========

The vnc server simply needs a connection to the video memory of the
QEMU process, video mode state, as well as channels for sending
keyboard and mouse events. It is stateless by nature and supports
reconnects. This applies to the other graphical display engines (SDL,
spice, and curses) as well.

Human monitor
=============

The human monitor is effectively stateless. It issues commands and
prints the result. There is no state in the monitor and it can be
built directly on top of QMP.

An additional benefit here is that it would allow for multiple
monitors.

Disconnects
===========

It must be possible for a client process getting killed or
disconnected from the QEMU process, in which case is should be
possible to launch a new client that connects to the QEMU
process. In this case, commands needs to be provided allowing the
client process to query the QEMU process and virtagent for current
state information. In-progress commands may fail, and will need to be
re-run, such as copy-paste and and file copy. However neither of
these are vital commands and a re-run of such commands is acceptable
behavior.

Migration
=========

Given that migration moves the guest to a new QEMU process, normally
on a different host, any connection from management tools to the
monitor, QMP sockets, virtio-serial, etc. require a new connection
through the new QEMU process. Stateless connections, such as the
monitor, QMP and the vnc-server handles reconnects without problems,
and should not have any issues during migration that are different
from the issues in the current implementation.

The main issues causing problems are stateful events such as
copy-paste, which is handled via a guest agent. The copy-paste problem
can be handled by blocking copy-paste operations during migration,
until the guest agent is reachable through the QEMU process on the new
host. This does mean that copy-paste can block or fail temporarily
(depending on whether is it implemented as -EBUSY or just blocks),
however it is not a mission critical feature, and it can also block or
fail temporarily during normal operation on a non virtual system.

Per the nature of the operation, a file copy via a guest agent is
going to fail and will have to be restarted after migration has
completed, in case it does not complete. This is again a non critical
feature and requiring a restart is acceptable.

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 5958 bytes --]

On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com> wrote:
>
> Hi,
>
> On last week's call we discussed the issue of splitting non core
> features of QEMU into it's own process to reduce the security risks etc.
>
> I wrote up a summary of my thoughts on this to try to cover the various
> issues. Feedback welcome and hopefully we can continue the discussion on
> a future call - maybe next week?
>
> I would like to be part of the discussion, but it's a public holiday
> here March 1st, so I won't be on that call.
>
> Cheers,
> Jes
>
>
> Separating host-side virtagent and other tasks from core QEMU
> =============================================================
>
> To improve auditing of the core QEMU code, it would be ideal to be
> able to separate the core QEMU functionality from utility
> functionality by  moving the utility functionality into its own
> process. This process will be referred to as the QEMU client below.

Separating as in moving code outside of qemu.git, making code optionally
built in, making code optionally built in or loadable as a separate
executable that is automatically launched, or making code always built
outside the main executable?

I'm very nervous about having a large number of daemons necessary to run
QEMU.  I think a reasonable approach would be a single front-end daemond.

Once QAPI is merged, there is a very incremental approach we can take for
this sort of work.  Take your favorite subsystem (like gdbstub or SDL) and
make it only use QMP apis.  Once we're only using QMP internally in a
subsystem, then building it out of the main QEMU and using libqmp should be
fairly easy.

Regards,

Anthony Liguori

> Components which are candidates for moving out of QEMU include:
>  - virtagent
>  - vnc server (and other graphical displays such as SDL, spice and
>   curses)
>  - human monitor

These are all much harder than they may seem.  There's a ton of QMP
functions that would be needed before we could even try to do this.

> The idea is to have QEMU launch as a daemon, and then allow for one of
> more client processes to connect to it. These will then offer the
> various services. The main issue to discuss is how to handle various
> state information, reconnects, and migration.
>
> Security
> ========
>
> The primary reason for this discussion is security, however there are
> other benefits from this split, as will be mentioned below. During a
> demo of virtagent I hit a case where a bug in the agent command
> handling code caused a crash of the host QEMU process. While it is
> probably a simple bug, it shows how adding more complexity to the QEMU
> process increases the risk of adding security problems that could
> potentially be exploited by a hostile guest.
>
> By splitting non core functionality into a QEMU client process, the
> host process will be isolated from a large number of potential
> security problems, ie. in case a client process is killed or crashes,
> it should not affect the main QEMU process. In addition it makes it
> easier to audit the core QEMU functionality.
>
> virtagent
> =========
>
> In short virtagent provides a set of simple commands, most of which
> do not have state associated with them. These include shutdown, ping,
> fsfreeze/fsthaw, etc. Other commands might be multi-stage commands
> which could fail if the client is disconnected from the daemon while
> the command is in progress. These include copy-paste and file copy.
>
> vnc server
> ==========
>
> The vnc server simply needs a connection to the video memory of the
> QEMU process, video mode state, as well as channels for sending
> keyboard and mouse events. It is stateless by nature and supports
> reconnects. This applies to the other graphical display engines (SDL,
> spice, and curses) as well.
>
> Human monitor
> =============
>
> The human monitor is effectively stateless. It issues commands and
> prints the result. There is no state in the monitor and it can be
> built directly on top of QMP.
>
> An additional benefit here is that it would allow for multiple
> monitors.
>
> Disconnects
> ===========
>
> It must be possible for a client process getting killed or
> disconnected from the QEMU process, in which case is should be
> possible to launch a new client that connects to the QEMU
> process. In this case, commands needs to be provided allowing the
> client process to query the QEMU process and virtagent for current
> state information. In-progress commands may fail, and will need to be
> re-run, such as copy-paste and and file copy. However neither of
> these are vital commands and a re-run of such commands is acceptable
> behavior.
>
> Migration
> =========
>
> Given that migration moves the guest to a new QEMU process, normally
> on a different host, any connection from management tools to the
> monitor, QMP sockets, virtio-serial, etc. require a new connection
> through the new QEMU process. Stateless connections, such as the
> monitor, QMP and the vnc-server handles reconnects without problems,
> and should not have any issues during migration that are different
> from the issues in the current implementation.
>
> The main issues causing problems are stateful events such as
> copy-paste, which is handled via a guest agent. The copy-paste problem
> can be handled by blocking copy-paste operations during migration,
> until the guest agent is reachable through the QEMU process on the new
> host. This does mean that copy-paste can block or fail temporarily
> (depending on whether is it implemented as -EBUSY or just blocks),
> however it is not a mission critical feature, and it can also block or
> fail temporarily during normal operation on a non virtual system.
>
> Per the nature of the operation, a file copy via a guest agent is
> going to fail and will have to be restarted after migration has
> completed, in case it does not complete. This is again a non critical
> feature and requiring a restart is acceptable.
>

[-- Attachment #2: Type: text/html, Size: 7053 bytes --]

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Dor Laor]

On 02/28/2011 07:44 PM, Anthony Liguori wrote:
>
> On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com
> <mailto:Jes.Sorensen@redhat.com>> wrote:
>  >
>  > Hi,
>  >
>  > On last week's call we discussed the issue of splitting non core
>  > features of QEMU into it's own process to reduce the security risks etc.
>  >
>  > I wrote up a summary of my thoughts on this to try to cover the various
>  > issues. Feedback welcome and hopefully we can continue the discussion on
>  > a future call - maybe next week?
>  >
>  > I would like to be part of the discussion, but it's a public holiday
>  > here March 1st, so I won't be on that call.
>  >
>  > Cheers,
>  > Jes
>  >
>  >
>  > Separating host-side virtagent and other tasks from core QEMU
>  > =============================================================
>  >
>  > To improve auditing of the core QEMU code, it would be ideal to be
>  > able to separate the core QEMU functionality from utility
>  > functionality by  moving the utility functionality into its own
>  > process. This process will be referred to as the QEMU client below.
>
> Separating as in moving code outside of qemu.git, making code optionally
> built in, making code optionally built in or loadable as a separate
> executable that is automatically launched, or making code always built
> outside the main executable?
>
> I'm very nervous about having a large number of daemons necessary to run
> QEMU.  I think a reasonable approach would be a single front-end daemond.

s/daemon/son processes/
Qemu is the one that should spawn them and they should be transparent 
from the management. This way running qemu stays the same and qemu just 
need to add the logic to get a SIGCHILD and potentially re-execute an 
dead son process.

>
> Once QAPI is merged, there is a very incremental approach we can take
> for this sort of work.  Take your favorite subsystem (like gdbstub or
> SDL) and make it only use QMP apis.  Once we're only using QMP
> internally in a subsystem, then building it out of the main QEMU and
> using libqmp should be fairly easy.
>
> Regards,
>
> Anthony Liguori
>
>  > Components which are candidates for moving out of QEMU include:
>  >  - virtagent
>  >  - vnc server (and other graphical displays such as SDL, spice and
>  >   curses)
>  >  - human monitor
>
> These are all much harder than they may seem.  There's a ton of QMP
> functions that would be needed before we could even try to do this.
>
>  > The idea is to have QEMU launch as a daemon, and then allow for one of
>  > more client processes to connect to it. These will then offer the
>  > various services. The main issue to discuss is how to handle various
>  > state information, reconnects, and migration.
>  >
>  > Security
>  > ========
>  >
>  > The primary reason for this discussion is security, however there are
>  > other benefits from this split, as will be mentioned below. During a
>  > demo of virtagent I hit a case where a bug in the agent command
>  > handling code caused a crash of the host QEMU process. While it is
>  > probably a simple bug, it shows how adding more complexity to the QEMU
>  > process increases the risk of adding security problems that could
>  > potentially be exploited by a hostile guest.
>  >
>  > By splitting non core functionality into a QEMU client process, the
>  > host process will be isolated from a large number of potential
>  > security problems, ie. in case a client process is killed or crashes,
>  > it should not affect the main QEMU process. In addition it makes it
>  > easier to audit the core QEMU functionality.
>  >
>  > virtagent
>  > =========
>  >
>  > In short virtagent provides a set of simple commands, most of which
>  > do not have state associated with them. These include shutdown, ping,
>  > fsfreeze/fsthaw, etc. Other commands might be multi-stage commands
>  > which could fail if the client is disconnected from the daemon while
>  > the command is in progress. These include copy-paste and file copy.
>  >
>  > vnc server
>  > ==========
>  >
>  > The vnc server simply needs a connection to the video memory of the
>  > QEMU process, video mode state, as well as channels for sending
>  > keyboard and mouse events. It is stateless by nature and supports
>  > reconnects. This applies to the other graphical display engines (SDL,
>  > spice, and curses) as well.
>  >
>  > Human monitor
>  > =============
>  >
>  > The human monitor is effectively stateless. It issues commands and
>  > prints the result. There is no state in the monitor and it can be
>  > built directly on top of QMP.
>  >
>  > An additional benefit here is that it would allow for multiple
>  > monitors.
>  >
>  > Disconnects
>  > ===========
>  >
>  > It must be possible for a client process getting killed or
>  > disconnected from the QEMU process, in which case is should be
>  > possible to launch a new client that connects to the QEMU
>  > process. In this case, commands needs to be provided allowing the
>  > client process to query the QEMU process and virtagent for current
>  > state information. In-progress commands may fail, and will need to be
>  > re-run, such as copy-paste and and file copy. However neither of
>  > these are vital commands and a re-run of such commands is acceptable
>  > behavior.
>  >
>  > Migration
>  > =========
>  >
>  > Given that migration moves the guest to a new QEMU process, normally
>  > on a different host, any connection from management tools to the
>  > monitor, QMP sockets, virtio-serial, etc. require a new connection
>  > through the new QEMU process. Stateless connections, such as the
>  > monitor, QMP and the vnc-server handles reconnects without problems,
>  > and should not have any issues during migration that are different
>  > from the issues in the current implementation.
>  >
>  > The main issues causing problems are stateful events such as
>  > copy-paste, which is handled via a guest agent. The copy-paste problem
>  > can be handled by blocking copy-paste operations during migration,
>  > until the guest agent is reachable through the QEMU process on the new
>  > host. This does mean that copy-paste can block or fail temporarily
>  > (depending on whether is it implemented as -EBUSY or just blocks),
>  > however it is not a mission critical feature, and it can also block or
>  > fail temporarily during normal operation on a non virtual system.
>  >
>  > Per the nature of the operation, a file copy via a guest agent is
>  > going to fail and will have to be restarted after migration has
>  > completed, in case it does not complete. This is again a non critical
>  > feature and requiring a restart is acceptable.
>  >
>

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 7126 bytes --]

On Mar 1, 2011 7:07 AM, "Dor Laor" <dlaor@redhat.com> wrote:
>
> On 02/28/2011 07:44 PM, Anthony Liguori wrote:
>>
>>
>> On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com
>> <mailto:Jes.Sorensen@redhat.com>> wrote:
>>  >
>>  > Hi,
>>  >
>>  > On last week's call we discussed the issue of splitting non core
>>  > features of QEMU into it's own process to reduce the security risks
etc.
>>  >
>>  > I wrote up a summary of my thoughts on this to try to cover the
various
>>  > issues. Feedback welcome and hopefully we can continue the discussion
on
>>  > a future call - maybe next week?
>>  >
>>  > I would like to be part of the discussion, but it's a public holiday
>>  > here March 1st, so I won't be on that call.
>>  >
>>  > Cheers,
>>  > Jes
>>  >
>>  >
>>  > Separating host-side virtagent and other tasks from core QEMU
>>  > =============================================================
>>  >
>>  > To improve auditing of the core QEMU code, it would be ideal to be
>>  > able to separate the core QEMU functionality from utility
>>  > functionality by  moving the utility functionality into its own
>>  > process. This process will be referred to as the QEMU client below.
>>
>> Separating as in moving code outside of qemu.git, making code optionally
>> built in, making code optionally built in or loadable as a separate
>> executable that is automatically launched, or making code always built
>> outside the main executable?
>>
>> I'm very nervous about having a large number of daemons necessary to run
>> QEMU.  I think a reasonable approach would be a single front-end daemond.
>
>
> s/daemon/son processes/
> Qemu is the one that should spawn them and they should be transparent from
the management. This way running qemu stays the same and qemu just need to
add the logic to get a SIGCHILD and potentially re-execute an dead son
process.

Spice is the logical place to start, no?  It's the largest single dependency
we have and it does some scary things with qemu_mutex.  I would use spice as
a way to prove the concept.

Regards,

Anthony Liguori

>
>
>>
>> Once QAPI is merged, there is a very incremental approach we can take
>> for this sort of work.  Take your favorite subsystem (like gdbstub or
>> SDL) and make it only use QMP apis.  Once we're only using QMP
>> internally in a subsystem, then building it out of the main QEMU and
>> using libqmp should be fairly easy.
>>
>> Regards,
>>
>> Anthony Liguori
>>
>>  > Components which are candidates for moving out of QEMU include:
>>  >  - virtagent
>>  >  - vnc server (and other graphical displays such as SDL, spice and
>>  >   curses)
>>  >  - human monitor
>>
>> These are all much harder than they may seem.  There's a ton of QMP
>> functions that would be needed before we could even try to do this.
>>
>>  > The idea is to have QEMU launch as a daemon, and then allow for one of
>>  > more client processes to connect to it. These will then offer the
>>  > various services. The main issue to discuss is how to handle various
>>  > state information, reconnects, and migration.
>>  >
>>  > Security
>>  > ========
>>  >
>>  > The primary reason for this discussion is security, however there are
>>  > other benefits from this split, as will be mentioned below. During a
>>  > demo of virtagent I hit a case where a bug in the agent command
>>  > handling code caused a crash of the host QEMU process. While it is
>>  > probably a simple bug, it shows how adding more complexity to the QEMU
>>  > process increases the risk of adding security problems that could
>>  > potentially be exploited by a hostile guest.
>>  >
>>  > By splitting non core functionality into a QEMU client process, the
>>  > host process will be isolated from a large number of potential
>>  > security problems, ie. in case a client process is killed or crashes,
>>  > it should not affect the main QEMU process. In addition it makes it
>>  > easier to audit the core QEMU functionality.
>>  >
>>  > virtagent
>>  > =========
>>  >
>>  > In short virtagent provides a set of simple commands, most of which
>>  > do not have state associated with them. These include shutdown, ping,
>>  > fsfreeze/fsthaw, etc. Other commands might be multi-stage commands
>>  > which could fail if the client is disconnected from the daemon while
>>  > the command is in progress. These include copy-paste and file copy.
>>  >
>>  > vnc server
>>  > ==========
>>  >
>>  > The vnc server simply needs a connection to the video memory of the
>>  > QEMU process, video mode state, as well as channels for sending
>>  > keyboard and mouse events. It is stateless by nature and supports
>>  > reconnects. This applies to the other graphical display engines (SDL,
>>  > spice, and curses) as well.
>>  >
>>  > Human monitor
>>  > =============
>>  >
>>  > The human monitor is effectively stateless. It issues commands and
>>  > prints the result. There is no state in the monitor and it can be
>>  > built directly on top of QMP.
>>  >
>>  > An additional benefit here is that it would allow for multiple
>>  > monitors.
>>  >
>>  > Disconnects
>>  > ===========
>>  >
>>  > It must be possible for a client process getting killed or
>>  > disconnected from the QEMU process, in which case is should be
>>  > possible to launch a new client that connects to the QEMU
>>  > process. In this case, commands needs to be provided allowing the
>>  > client process to query the QEMU process and virtagent for current
>>  > state information. In-progress commands may fail, and will need to be
>>  > re-run, such as copy-paste and and file copy. However neither of
>>  > these are vital commands and a re-run of such commands is acceptable
>>  > behavior.
>>  >
>>  > Migration
>>  > =========
>>  >
>>  > Given that migration moves the guest to a new QEMU process, normally
>>  > on a different host, any connection from management tools to the
>>  > monitor, QMP sockets, virtio-serial, etc. require a new connection
>>  > through the new QEMU process. Stateless connections, such as the
>>  > monitor, QMP and the vnc-server handles reconnects without problems,
>>  > and should not have any issues during migration that are different
>>  > from the issues in the current implementation.
>>  >
>>  > The main issues causing problems are stateful events such as
>>  > copy-paste, which is handled via a guest agent. The copy-paste problem
>>  > can be handled by blocking copy-paste operations during migration,
>>  > until the guest agent is reachable through the QEMU process on the new
>>  > host. This does mean that copy-paste can block or fail temporarily
>>  > (depending on whether is it implemented as -EBUSY or just blocks),
>>  > however it is not a mission critical feature, and it can also block or
>>  > fail temporarily during normal operation on a non virtual system.
>>  >
>>  > Per the nature of the operation, a file copy via a guest agent is
>>  > going to fail and will have to be restarted after migration has
>>  > completed, in case it does not complete. This is again a non critical
>>  > feature and requiring a restart is acceptable.
>>  >
>>
>

[-- Attachment #2: Type: text/html, Size: 9387 bytes --]

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Dor Laor]

On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>
> On Mar 1, 2011 7:07 AM, "Dor Laor" <dlaor@redhat.com
> <mailto:dlaor@redhat.com>> wrote:
>  >
>  > On 02/28/2011 07:44 PM, Anthony Liguori wrote:
>  >>
>  >>
>  >> On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com
> <mailto:Jes.Sorensen@redhat.com>
>  >> <mailto:Jes.Sorensen@redhat.com <mailto:Jes.Sorensen@redhat.com>>>
> wrote:
>  >> >
>  >> > Hi,
>  >> >
>  >> > On last week's call we discussed the issue of splitting non core
>  >> > features of QEMU into it's own process to reduce the security
> risks etc.
>  >> >
>  >> > I wrote up a summary of my thoughts on this to try to cover the
> various
>  >> > issues. Feedback welcome and hopefully we can continue the
> discussion on
>  >> > a future call - maybe next week?
>  >> >
>  >> > I would like to be part of the discussion, but it's a public holiday
>  >> > here March 1st, so I won't be on that call.
>  >> >
>  >> > Cheers,
>  >> > Jes
>  >> >
>  >> >
>  >> > Separating host-side virtagent and other tasks from core QEMU
>  >> > =============================================================
>  >> >
>  >> > To improve auditing of the core QEMU code, it would be ideal to be
>  >> > able to separate the core QEMU functionality from utility
>  >> > functionality by  moving the utility functionality into its own
>  >> > process. This process will be referred to as the QEMU client below.
>  >>
>  >> Separating as in moving code outside of qemu.git, making code optionally
>  >> built in, making code optionally built in or loadable as a separate
>  >> executable that is automatically launched, or making code always built
>  >> outside the main executable?
>  >>
>  >> I'm very nervous about having a large number of daemons necessary to run
>  >> QEMU.  I think a reasonable approach would be a single front-end
> daemond.
>  >
>  >
>  > s/daemon/son processes/
>  > Qemu is the one that should spawn them and they should be transparent
> from the management. This way running qemu stays the same and qemu just
> need to add the logic to get a SIGCHILD and potentially re-execute an
> dead son process.
>
> Spice is the logical place to start, no?  It's the largest single
> dependency we have and it does some scary things with qemu_mutex.  I
> would use spice as a way to prove the concept.

I agree it is desirable to the this for spice but it is allot more 
complex than virtagent isolation. Spice is performance sensitive and 
contains much more state. It needs to access the guest memory for 
reading the surfaces. It can be solved but needs some major changes. 
Adding spice-devel to the discussion.

Will virtagent be kept out of tree till we merge spice?

>
> Regards,
>
> Anthony Liguori
>
>  >
>  >
>  >>
>  >> Once QAPI is merged, there is a very incremental approach we can take
>  >> for this sort of work.  Take your favorite subsystem (like gdbstub or
>  >> SDL) and make it only use QMP apis.  Once we're only using QMP
>  >> internally in a subsystem, then building it out of the main QEMU and
>  >> using libqmp should be fairly easy.
>  >>
>  >> Regards,
>  >>
>  >> Anthony Liguori
>  >>
>  >> > Components which are candidates for moving out of QEMU include:
>  >> >  - virtagent
>  >> >  - vnc server (and other graphical displays such as SDL, spice and
>  >> >   curses)
>  >> >  - human monitor
>  >>
>  >> These are all much harder than they may seem.  There's a ton of QMP
>  >> functions that would be needed before we could even try to do this.
>  >>
>  >> > The idea is to have QEMU launch as a daemon, and then allow for one of
>  >> > more client processes to connect to it. These will then offer the
>  >> > various services. The main issue to discuss is how to handle various
>  >> > state information, reconnects, and migration.
>  >> >
>  >> > Security
>  >> > ========
>  >> >
>  >> > The primary reason for this discussion is security, however there are
>  >> > other benefits from this split, as will be mentioned below. During a
>  >> > demo of virtagent I hit a case where a bug in the agent command
>  >> > handling code caused a crash of the host QEMU process. While it is
>  >> > probably a simple bug, it shows how adding more complexity to the QEMU
>  >> > process increases the risk of adding security problems that could
>  >> > potentially be exploited by a hostile guest.
>  >> >
>  >> > By splitting non core functionality into a QEMU client process, the
>  >> > host process will be isolated from a large number of potential
>  >> > security problems, ie. in case a client process is killed or crashes,
>  >> > it should not affect the main QEMU process. In addition it makes it
>  >> > easier to audit the core QEMU functionality.
>  >> >
>  >> > virtagent
>  >> > =========
>  >> >
>  >> > In short virtagent provides a set of simple commands, most of which
>  >> > do not have state associated with them. These include shutdown, ping,
>  >> > fsfreeze/fsthaw, etc. Other commands might be multi-stage commands
>  >> > which could fail if the client is disconnected from the daemon while
>  >> > the command is in progress. These include copy-paste and file copy.
>  >> >
>  >> > vnc server
>  >> > ==========
>  >> >
>  >> > The vnc server simply needs a connection to the video memory of the
>  >> > QEMU process, video mode state, as well as channels for sending
>  >> > keyboard and mouse events. It is stateless by nature and supports
>  >> > reconnects. This applies to the other graphical display engines (SDL,
>  >> > spice, and curses) as well.
>  >> >
>  >> > Human monitor
>  >> > =============
>  >> >
>  >> > The human monitor is effectively stateless. It issues commands and
>  >> > prints the result. There is no state in the monitor and it can be
>  >> > built directly on top of QMP.
>  >> >
>  >> > An additional benefit here is that it would allow for multiple
>  >> > monitors.
>  >> >
>  >> > Disconnects
>  >> > ===========
>  >> >
>  >> > It must be possible for a client process getting killed or
>  >> > disconnected from the QEMU process, in which case is should be
>  >> > possible to launch a new client that connects to the QEMU
>  >> > process. In this case, commands needs to be provided allowing the
>  >> > client process to query the QEMU process and virtagent for current
>  >> > state information. In-progress commands may fail, and will need to be
>  >> > re-run, such as copy-paste and and file copy. However neither of
>  >> > these are vital commands and a re-run of such commands is acceptable
>  >> > behavior.
>  >> >
>  >> > Migration
>  >> > =========
>  >> >
>  >> > Given that migration moves the guest to a new QEMU process, normally
>  >> > on a different host, any connection from management tools to the
>  >> > monitor, QMP sockets, virtio-serial, etc. require a new connection
>  >> > through the new QEMU process. Stateless connections, such as the
>  >> > monitor, QMP and the vnc-server handles reconnects without problems,
>  >> > and should not have any issues during migration that are different
>  >> > from the issues in the current implementation.
>  >> >
>  >> > The main issues causing problems are stateful events such as
>  >> > copy-paste, which is handled via a guest agent. The copy-paste problem
>  >> > can be handled by blocking copy-paste operations during migration,
>  >> > until the guest agent is reachable through the QEMU process on the new
>  >> > host. This does mean that copy-paste can block or fail temporarily
>  >> > (depending on whether is it implemented as -EBUSY or just blocks),
>  >> > however it is not a mission critical feature, and it can also block or
>  >> > fail temporarily during normal operation on a non virtual system.
>  >> >
>  >> > Per the nature of the operation, a file copy via a guest agent is
>  >> > going to fail and will have to be restarted after migration has
>  >> > completed, in case it does not complete. This is again a non critical
>  >> > feature and requiring a restart is acceptable.
>  >> >
>  >>
>  >
>

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Anthony Liguori]

On 03/01/2011 09:25 AM, Dor Laor wrote:
> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>>
>> On Mar 1, 2011 7:07 AM, "Dor Laor" <dlaor@redhat.com
>> <mailto:dlaor@redhat.com>> wrote:
>> >
>> > On 02/28/2011 07:44 PM, Anthony Liguori wrote:
>> >>
>> >>
>> >> On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com
>> <mailto:Jes.Sorensen@redhat.com>
>> >> <mailto:Jes.Sorensen@redhat.com <mailto:Jes.Sorensen@redhat.com>>>
>> wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > On last week's call we discussed the issue of splitting non core
>> >> > features of QEMU into it's own process to reduce the security
>> risks etc.
>> >> >
>> >> > I wrote up a summary of my thoughts on this to try to cover the
>> various
>> >> > issues. Feedback welcome and hopefully we can continue the
>> discussion on
>> >> > a future call - maybe next week?
>> >> >
>> >> > I would like to be part of the discussion, but it's a public 
>> holiday
>> >> > here March 1st, so I won't be on that call.
>> >> >
>> >> > Cheers,
>> >> > Jes
>> >> >
>> >> >
>> >> > Separating host-side virtagent and other tasks from core QEMU
>> >> > =============================================================
>> >> >
>> >> > To improve auditing of the core QEMU code, it would be ideal to be
>> >> > able to separate the core QEMU functionality from utility
>> >> > functionality by  moving the utility functionality into its own
>> >> > process. This process will be referred to as the QEMU client below.
>> >>
>> >> Separating as in moving code outside of qemu.git, making code 
>> optionally
>> >> built in, making code optionally built in or loadable as a separate
>> >> executable that is automatically launched, or making code always 
>> built
>> >> outside the main executable?
>> >>
>> >> I'm very nervous about having a large number of daemons necessary 
>> to run
>> >> QEMU.  I think a reasonable approach would be a single front-end
>> daemond.
>> >
>> >
>> > s/daemon/son processes/
>> > Qemu is the one that should spawn them and they should be transparent
>> from the management. This way running qemu stays the same and qemu just
>> need to add the logic to get a SIGCHILD and potentially re-execute an
>> dead son process.
>>
>> Spice is the logical place to start, no?  It's the largest single
>> dependency we have and it does some scary things with qemu_mutex.  I
>> would use spice as a way to prove the concept.
>
> I agree it is desirable to the this for spice but it is allot more 
> complex than virtagent isolation. Spice is performance sensitive and 
> contains much more state. It needs to access the guest memory for 
> reading the surfaces. It can be solved but needs some major changes. 
> Adding spice-devel to the discussion.

Yeah, but the viability of this mechanism is dependent on whether it can 
support something that's complex, just like Spice.  Considering that the 
smaller pieces like VNC or virt-agent are at most a couple thousand of 
lines of code, whereas Spice is close to a hundred thousand lines, the 
benefit from moving Spice to a separate address space is significantly 
higher.

> Will virtagent be kept out of tree till we merge spice?

Nothing should be held up from being merged because of an effort to put 
things in a separate address space.  But that's not to say that 
virt-agent is something that's going to get merged tomorrow.  I don't 
know that there is even an agreed upon architecture at the moment.

Regards,

Anthony Liguori

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 02/28/11 18:44, Anthony Liguori wrote:
> On Feb 28, 2011 10:44 AM, "Jes Sorensen" <Jes.Sorensen@redhat.com> wrote:
>> > Separating host-side virtagent and other tasks from core QEMU
>> > =============================================================
>> >
>> > To improve auditing of the core QEMU code, it would be ideal to be
>> > able to separate the core QEMU functionality from utility
>> > functionality by  moving the utility functionality into its own
>> > process. This process will be referred to as the QEMU client below.
> Separating as in moving code outside of qemu.git, making code optionally
> built in, making code optionally built in or loadable as a separate
> executable that is automatically launched, or making code always built
> outside the main executable?
> 
> I'm very nervous about having a large number of daemons necessary to run
> QEMU.  I think a reasonable approach would be a single front-end daemond.
> 
> Once QAPI is merged, there is a very incremental approach we can take for
> this sort of work.  Take your favorite subsystem (like gdbstub or SDL) and
> make it only use QMP apis.  Once we're only using QMP internally in a
> subsystem, then building it out of the main QEMU and using libqmp should be
> fairly easy.

Hi Anthony,

Back from a day off playing with power drills and concrete walls :)

Sorry I should have made it a little more clear, it was obvious to me,
but not written down:

The idea is to keep everything as part of the QEMU package, ie. part of
qemu.git. My idea is really to have one QEMU host daemon and one QEMU
client, which provides the various services. You type make and you get
two binaries instead of one. We could allow other daemons to connect to
the host daemon, but that wouldn't be a primary target for this in my
book, and I am not sure we really want to do this.

It is absolutely vital for me that we do not make things much more
complicated for users with this move. I don't want to get into a
situation where we start forcing external packages or daemons in order
to run basic QEMU. If we start requiring such, we have failed! However,
I think it is a reasonable compromise to have one daemon you launch, and
then a simple client you launch straight after which will then provide
the same/similar views and interfaces that users are used to when they
launch current QEMU (monitor, vnc server etc).

I hope this clarifies things.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/01/11 13:07, Dor Laor wrote:
> On 02/28/2011 07:44 PM, Anthony Liguori wrote:
>> I'm very nervous about having a large number of daemons necessary to run
>> QEMU.  I think a reasonable approach would be a single front-end daemond.
> 
> s/daemon/son processes/
> Qemu is the one that should spawn them and they should be transparent
> from the management. This way running qemu stays the same and qemu just
> need to add the logic to get a SIGCHILD and potentially re-execute an
> dead son process.

I disagree here, I do not want to require QEMU to spawn the new
processes. Having a daemon you can use to connect will provide more
flexibility and isn't unreasonably complex.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/01/11 15:25, Dor Laor wrote:
> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>>
>> On Mar 1, 2011 7:07 AM, "Dor Laor" <dlaor@redhat.com
>>  > Qemu is the one that should spawn them and they should be transparent
>> from the management. This way running qemu stays the same and qemu just
>> need to add the logic to get a SIGCHILD and potentially re-execute an
>> dead son process.
>>
>> Spice is the logical place to start, no?  It's the largest single
>> dependency we have and it does some scary things with qemu_mutex.  I
>> would use spice as a way to prove the concept.
> 
> I agree it is desirable to the this for spice but it is allot more
> complex than virtagent isolation. Spice is performance sensitive and
> contains much more state. It needs to access the guest memory for
> reading the surfaces. It can be solved but needs some major changes.
> Adding spice-devel to the discussion.

I had a few thoughts about this already, which I think will work for
both spice and vnc. What we could do is to expose the video memory via
shared memory. That way a spice or vnc daemon could get direct access to
the memory, this would limit communication to keyboard/mouse events, as
well as video mode info, and possibly notifications to the client about
which ranges of memory have been updated.

Using shared memory this way should allow us to implement the video
clients without performance loss, in fact it should be beneficial since
it would allow them to run fully separate from the host daemon.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/01/11 15:25, Dor Laor wrote:
> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>> Spice is the logical place to start, no?  It's the largest single
>> dependency we have and it does some scary things with qemu_mutex.  I
>> would use spice as a way to prove the concept.
> 
> I agree it is desirable to the this for spice but it is allot more
> complex than virtagent isolation. Spice is performance sensitive and
> contains much more state. It needs to access the guest memory for
> reading the surfaces. It can be solved but needs some major changes.
> Adding spice-devel to the discussion.

Please don't add broken misconfigured mailing lists, which require
moderator or subscription, to discussions on public lists.

Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Dor Laor]

On 03/02/2011 12:28 PM, Jes Sorensen wrote:
> On 03/01/11 15:25, Dor Laor wrote:
>> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>>> Spice is the logical place to start, no?  It's the largest single
>>> dependency we have and it does some scary things with qemu_mutex.  I
>>> would use spice as a way to prove the concept.
>>
>> I agree it is desirable to the this for spice but it is allot more
>> complex than virtagent isolation. Spice is performance sensitive and
>> contains much more state. It needs to access the guest memory for
>> reading the surfaces. It can be solved but needs some major changes.
>> Adding spice-devel to the discussion.
>
> Please don't add broken misconfigured mailing lists, which require
> moderator or subscription, to discussions on public lists.

Isn't it simpler to ask the spice list maintainer (Alon Levy, CCed) to 
do it?


> Jes
>

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/02/11 11:42, Dor Laor wrote:
> On 03/02/2011 12:28 PM, Jes Sorensen wrote:
>> On 03/01/11 15:25, Dor Laor wrote:
>>> I agree it is desirable to the this for spice but it is allot more
>>> complex than virtagent isolation. Spice is performance sensitive and
>>> contains much more state. It needs to access the guest memory for
>>> reading the surfaces. It can be solved but needs some major changes.
>>> Adding spice-devel to the discussion.
>>
>> Please don't add broken misconfigured mailing lists, which require
>> moderator or subscription, to discussions on public lists.
> 
> Isn't it simpler to ask the spice list maintainer (Alon Levy, CCed) to
> do it?

We cannot have a discussion on a mailing list if we constantly have to
wait for the list maintainer to approve it. There is really zero
justification for closing a development list of an open source project.
Spam is handled in other ways, this doesn't solve it.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Dor Laor]

On 03/02/2011 12:25 PM, Jes Sorensen wrote:
> On 03/01/11 15:25, Dor Laor wrote:
>> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>>>
>>> On Mar 1, 2011 7:07 AM, "Dor Laor"<dlaor@redhat.com
>>>   >  Qemu is the one that should spawn them and they should be transparent
>>> from the management. This way running qemu stays the same and qemu just
>>> need to add the logic to get a SIGCHILD and potentially re-execute an
>>> dead son process.
>>>
>>> Spice is the logical place to start, no?  It's the largest single
>>> dependency we have and it does some scary things with qemu_mutex.  I
>>> would use spice as a way to prove the concept.
>>
>> I agree it is desirable to the this for spice but it is allot more
>> complex than virtagent isolation. Spice is performance sensitive and
>> contains much more state. It needs to access the guest memory for
>> reading the surfaces. It can be solved but needs some major changes.
>> Adding spice-devel to the discussion.
>
> I had a few thoughts about this already, which I think will work for
> both spice and vnc. What we could do is to expose the video memory via
> shared memory. That way a spice or vnc daemon could get direct access to
> the memory, this would limit communication to keyboard/mouse events, as
> well as video mode info, and possibly notifications to the client about
> which ranges of memory have been updated.
>
> Using shared memory this way should allow us to implement the video
> clients without performance loss, in fact it should be beneficial since
> it would allow them to run fully separate from the host daemon.

Why do you call it a daemon? Each VM instance should have only one, the 
'host daemon' naming is misleading.

The proper solution long term is to sandbox qemu in a way that there 
privileged mode and non privileged mode. It might be implemented using 
separate address space or not. Most operations like vnc/rpc/spice/usb 
should be run with less privileges.

The main issue is that doing it right will take time and we'll want 
virt-agent be merged before the long term solution is ready. The best 
approach would be gradual development

>
> Cheers,
> Jes
>

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Alon Levy]

On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote:
> On 03/01/11 15:25, Dor Laor wrote:
> > On 03/01/2011 02:40 PM, Anthony Liguori wrote:
> >>
> >> On Mar 1, 2011 7:07 AM, "Dor Laor" <dlaor@redhat.com
> >>  > Qemu is the one that should spawn them and they should be transparent
> >> from the management. This way running qemu stays the same and qemu just
> >> need to add the logic to get a SIGCHILD and potentially re-execute an
> >> dead son process.
> >>
> >> Spice is the logical place to start, no?  It's the largest single
> >> dependency we have and it does some scary things with qemu_mutex.  I
> >> would use spice as a way to prove the concept.
> > 
> > I agree it is desirable to the this for spice but it is allot more
> > complex than virtagent isolation. Spice is performance sensitive and
> > contains much more state. It needs to access the guest memory for
> > reading the surfaces. It can be solved but needs some major changes.
> > Adding spice-devel to the discussion.
> 
> I had a few thoughts about this already, which I think will work for
> both spice and vnc. What we could do is to expose the video memory via
> shared memory. That way a spice or vnc daemon could get direct access to
> the memory, this would limit communication to keyboard/mouse events, as
> well as video mode info, and possibly notifications to the client about
> which ranges of memory have been updated.
> 
> Using shared memory this way should allow us to implement the video
> clients without performance loss, in fact it should be beneficial since
> it would allow them to run fully separate from the host daemon.
> 

I think that would work well for spice. Spice uses shared memory from the
pci device for both the framebuffer and surfaces/commands, but this is
not really relevant at this level. What about IO and irq? that would add
additional latencies, no? because each io exit would need to be ipc'ed over
to the spice/vnc process? and same way in the other direction, requesting
qemu to trigger an interrupt in the next vm entry.

> Cheers,
> Jes
> 

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/02/11 11:56, Dor Laor wrote:
> On 03/02/2011 12:25 PM, Jes Sorensen wrote:
>> On 03/01/11 15:25, Dor Laor wrote:
>> Using shared memory this way should allow us to implement the video
>> clients without performance loss, in fact it should be beneficial since
>> it would allow them to run fully separate from the host daemon.
> 
> Why do you call it a daemon? Each VM instance should have only one, the
> 'host daemon' naming is misleading.

I refer to it as a daemon because it is something the client(s) will
connect to. But yes, there will be a daemon per VM.

> The proper solution long term is to sandbox qemu in a way that there
> privileged mode and non privileged mode. It might be implemented using
> separate address space or not. Most operations like vnc/rpc/spice/usb
> should be run with less privileges.
> 
> The main issue is that doing it right will take time and we'll want
> virt-agent be merged before the long term solution is ready. The best
> approach would be gradual development

Yes I agree, I don't think this will happen overnight, and blocking
virtagent with this would be bad.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Dor Laor]

On 03/02/2011 12:58 PM, Alon Levy wrote:
> On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote:
>> On 03/01/11 15:25, Dor Laor wrote:
>>> On 03/01/2011 02:40 PM, Anthony Liguori wrote:
>>>>
>>>> On Mar 1, 2011 7:07 AM, "Dor Laor"<dlaor@redhat.com
>>>>   >  Qemu is the one that should spawn them and they should be transparent
>>>> from the management. This way running qemu stays the same and qemu just
>>>> need to add the logic to get a SIGCHILD and potentially re-execute an
>>>> dead son process.
>>>>
>>>> Spice is the logical place to start, no?  It's the largest single
>>>> dependency we have and it does some scary things with qemu_mutex.  I
>>>> would use spice as a way to prove the concept.
>>>
>>> I agree it is desirable to the this for spice but it is allot more
>>> complex than virtagent isolation. Spice is performance sensitive and
>>> contains much more state. It needs to access the guest memory for
>>> reading the surfaces. It can be solved but needs some major changes.
>>> Adding spice-devel to the discussion.
>>
>> I had a few thoughts about this already, which I think will work for
>> both spice and vnc. What we could do is to expose the video memory via
>> shared memory. That way a spice or vnc daemon could get direct access to
>> the memory, this would limit communication to keyboard/mouse events, as
>> well as video mode info, and possibly notifications to the client about
>> which ranges of memory have been updated.
>>
>> Using shared memory this way should allow us to implement the video
>> clients without performance loss, in fact it should be beneficial since
>> it would allow them to run fully separate from the host daemon.
>>
>
> I think that would work well for spice. Spice uses shared memory from the
> pci device for both the framebuffer and surfaces/commands, but this is

Is that the only DMA do you do? That's good for this model.

> not really relevant at this level. What about IO and irq? that would add
> additional latencies, no? because each io exit would need to be ipc'ed over
> to the spice/vnc process? and same way in the other direction, requesting
> qemu to trigger an interrupt in the next vm entry.

The qxl device can be in the privileged qemu (as a start) and it will 
handle irqs directly. Even today you need to notify the spice server 
thread, so nothing will change
>
>> Cheers,
>> Jes
>>

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/02/11 11:58, Alon Levy wrote:
> On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote:
>> I had a few thoughts about this already, which I think will work for
>> both spice and vnc. What we could do is to expose the video memory via
>> shared memory. That way a spice or vnc daemon could get direct access to
>> the memory, this would limit communication to keyboard/mouse events, as
>> well as video mode info, and possibly notifications to the client about
>> which ranges of memory have been updated.
>>
>> Using shared memory this way should allow us to implement the video
>> clients without performance loss, in fact it should be beneficial since
>> it would allow them to run fully separate from the host daemon.
>>
> 
> I think that would work well for spice. Spice uses shared memory from the
> pci device for both the framebuffer and surfaces/commands, but this is
> not really relevant at this level. What about IO and irq? that would add
> additional latencies, no? because each io exit would need to be ipc'ed over
> to the spice/vnc process? and same way in the other direction, requesting
> qemu to trigger an interrupt in the next vm entry.

I am glad the shmem approach will work for Spice. There would be
something there with IRQs etc, I agree. but there are methods to help
that too, we could use a shared memory event ring for example.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Alon Levy]

On Wed, Mar 02, 2011 at 01:04:58PM +0200, Dor Laor wrote:
> On 03/02/2011 12:58 PM, Alon Levy wrote:
> >On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote:
> >>On 03/01/11 15:25, Dor Laor wrote:
> >>>On 03/01/2011 02:40 PM, Anthony Liguori wrote:
> >>>>
> >>>>On Mar 1, 2011 7:07 AM, "Dor Laor"<dlaor@redhat.com
> >>>>  >  Qemu is the one that should spawn them and they should be transparent
> >>>>from the management. This way running qemu stays the same and qemu just
> >>>>need to add the logic to get a SIGCHILD and potentially re-execute an
> >>>>dead son process.
> >>>>
> >>>>Spice is the logical place to start, no?  It's the largest single
> >>>>dependency we have and it does some scary things with qemu_mutex.  I
> >>>>would use spice as a way to prove the concept.
> >>>
> >>>I agree it is desirable to the this for spice but it is allot more
> >>>complex than virtagent isolation. Spice is performance sensitive and
> >>>contains much more state. It needs to access the guest memory for
> >>>reading the surfaces. It can be solved but needs some major changes.
> >>>Adding spice-devel to the discussion.
> >>
> >>I had a few thoughts about this already, which I think will work for
> >>both spice and vnc. What we could do is to expose the video memory via
> >>shared memory. That way a spice or vnc daemon could get direct access to
> >>the memory, this would limit communication to keyboard/mouse events, as
> >>well as video mode info, and possibly notifications to the client about
> >>which ranges of memory have been updated.
> >>
> >>Using shared memory this way should allow us to implement the video
> >>clients without performance loss, in fact it should be beneficial since
> >>it would allow them to run fully separate from the host daemon.
> >>
> >
> >I think that would work well for spice. Spice uses shared memory from the
> >pci device for both the framebuffer and surfaces/commands, but this is
> 
> Is that the only DMA do you do? That's good for this model.
> 

Yes. We have two memory bars (another for rom) and an io bar. The bars contain
the framebuffer (primary surface backing store, also used in vga mode), the
non primary surfaces backing store and the rings (display, cursor, and release).

In qxl (native) mode the rings are read/written to all the time, the rest by demand.
In vga mode it's identical to vnc, the framebuffer is written from the guest and
we track the dirty regions.

> >not really relevant at this level. What about IO and irq? that would add
> >additional latencies, no? because each io exit would need to be ipc'ed over
> >to the spice/vnc process? and same way in the other direction, requesting
> >qemu to trigger an interrupt in the next vm entry.
> 
> The qxl device can be in the privileged qemu (as a start) and it
> will handle irqs directly. Even today you need to notify the spice
> server thread, so nothing will change

ok, so qxl remains in qemu process. But all those io exits (well, most) end
up as calls to spice-server, now in another process. So latency wise it's the
same.

> >
> >>Cheers,
> >>Jes
> >>
> 
> 

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Michael Roth]

On 03/02/2011 04:19 AM, Jes Sorensen wrote:
> On 02/28/11 18:44, Anthony Liguori wrote:
>> On Feb 28, 2011 10:44 AM, "Jes Sorensen"<Jes.Sorensen@redhat.com>  wrote:
>>>> Separating host-side virtagent and other tasks from core QEMU
>>>> =============================================================
>>>>
>>>> To improve auditing of the core QEMU code, it would be ideal to be
>>>> able to separate the core QEMU functionality from utility
>>>> functionality by  moving the utility functionality into its own
>>>> process. This process will be referred to as the QEMU client below.
>> Separating as in moving code outside of qemu.git, making code optionally
>> built in, making code optionally built in or loadable as a separate
>> executable that is automatically launched, or making code always built
>> outside the main executable?
>>
>> I'm very nervous about having a large number of daemons necessary to run
>> QEMU.  I think a reasonable approach would be a single front-end daemond.
>>
>> Once QAPI is merged, there is a very incremental approach we can take for
>> this sort of work.  Take your favorite subsystem (like gdbstub or SDL) and
>> make it only use QMP apis.  Once we're only using QMP internally in a
>> subsystem, then building it out of the main QEMU and using libqmp should be
>> fairly easy.
>
> Hi Anthony,
>
> Back from a day off playing with power drills and concrete walls :)
>
> Sorry I should have made it a little more clear, it was obvious to me,
> but not written down:
>
> The idea is to keep everything as part of the QEMU package, ie. part of
> qemu.git. My idea is really to have one QEMU host daemon and one QEMU
> client, which provides the various services. You type make and you get
> two binaries instead of one. We could allow other daemons to connect to
> the host daemon, but that wouldn't be a primary target for this in my
> book, and I am not sure we really want to do this.
>
> It is absolutely vital for me that we do not make things much more
> complicated for users with this move. I don't want to get into a
> situation where we start forcing external packages or daemons in order
> to run basic QEMU. If we start requiring such, we have failed! However,
> I think it is a reasonable compromise to have one daemon you launch, and
> then a simple client you launch straight after which will then provide
> the same/similar views and interfaces that users are used to when they
> launch current QEMU (monitor, vnc server etc).

I think the challenge with this approach is defining an IPC mechanism 
that is robust enough to support it. For instance, on one end of the 
spectrum we have Spice, where shared memory paired with some mechanism 
for IO notification between the client/server might make sense. But then 
we have things like the human monitor where a socket interface or pipe 
is clearly more the more straight-forward route.

We may find that it more desirable to have multiple classes of client 
components, each contain within a client process with it's own IPC 
mechanism. Although, multiple IPC mechanisms doesn't sound particularly 
enticing either...but 2 might not be so unreasonable.

>
> I hope this clarifies things.
>
> Cheers,
> Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/02/11 14:13, Michael Roth wrote:
> On 03/02/2011 04:19 AM, Jes Sorensen wrote:

>> It is absolutely vital for me that we do not make things much more
>> complicated for users with this move. I don't want to get into a
>> situation where we start forcing external packages or daemons in order
>> to run basic QEMU. If we start requiring such, we have failed! However,
>> I think it is a reasonable compromise to have one daemon you launch, and
>> then a simple client you launch straight after which will then provide
>> the same/similar views and interfaces that users are used to when they
>> launch current QEMU (monitor, vnc server etc).
> 
> I think the challenge with this approach is defining an IPC mechanism
> that is robust enough to support it. For instance, on one end of the
> spectrum we have Spice, where shared memory paired with some mechanism
> for IO notification between the client/server might make sense. But then
> we have things like the human monitor where a socket interface or pipe
> is clearly more the more straight-forward route.
> 
> We may find that it more desirable to have multiple classes of client
> components, each contain within a client process with it's own IPC
> mechanism. Although, multiple IPC mechanisms doesn't sound particularly
> enticing either...but 2 might not be so unreasonable.

Hi Michael,

I think we need two types for sure, even for the video case, we will
still need a control channel as well. However, I don't think it is
desirable to split things up more than we have to, so if we can keep it
within one client process that is good. Maybe there are cases where it
makes sense to split it into more processes, I could be convinced, but I
think we really need to be careful making it too much of a complex mess
either.

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Michael Roth]

On 03/02/2011 07:18 AM, Jes Sorensen wrote:
> On 03/02/11 14:13, Michael Roth wrote:
>> On 03/02/2011 04:19 AM, Jes Sorensen wrote:
>
>>> It is absolutely vital for me that we do not make things much more
>>> complicated for users with this move. I don't want to get into a
>>> situation where we start forcing external packages or daemons in order
>>> to run basic QEMU. If we start requiring such, we have failed! However,
>>> I think it is a reasonable compromise to have one daemon you launch, and
>>> then a simple client you launch straight after which will then provide
>>> the same/similar views and interfaces that users are used to when they
>>> launch current QEMU (monitor, vnc server etc).
>>
>> I think the challenge with this approach is defining an IPC mechanism
>> that is robust enough to support it. For instance, on one end of the
>> spectrum we have Spice, where shared memory paired with some mechanism
>> for IO notification between the client/server might make sense. But then
>> we have things like the human monitor where a socket interface or pipe
>> is clearly more the more straight-forward route.
>>
>> We may find that it more desirable to have multiple classes of client
>> components, each contain within a client process with it's own IPC
>> mechanism. Although, multiple IPC mechanisms doesn't sound particularly
>> enticing either...but 2 might not be so unreasonable.
>
> Hi Michael,
>
> I think we need two types for sure, even for the video case, we will
> still need a control channel as well. However, I don't think it is
> desirable to split things up more than we have to, so if we can keep it
> within one client process that is good. Maybe there are cases where it
> makes sense to split it into more processes, I could be convinced, but I
> think we really need to be careful making it too much of a complex mess
> either.

Yup, if it's doable I'd prefer a single client process as well. Just 
hard to predict how difficult it'll be to support 2 or more mechanisms. 
Although, I'd imagine we'd end up with something like qemu's io loop, 
with event-driven shmem and fd-based work, which does seem doable.

>
> Cheers,
> Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Jes Sorensen]

On 03/02/11 14:49, Michael Roth wrote:
> On 03/02/2011 07:18 AM, Jes Sorensen wrote:
>> I think we need two types for sure, even for the video case, we will
>> still need a control channel as well. However, I don't think it is
>> desirable to split things up more than we have to, so if we can keep it
>> within one client process that is good. Maybe there are cases where it
>> makes sense to split it into more processes, I could be convinced, but I
>> think we really need to be careful making it too much of a complex mess
>> either.
> 
> Yup, if it's doable I'd prefer a single client process as well. Just
> hard to predict how difficult it'll be to support 2 or more mechanisms.
> Although, I'd imagine we'd end up with something like qemu's io loop,
> with event-driven shmem and fd-based work, which does seem doable.

That is pretty much what I had in mind. Will have to see how it works
out, but I think it is very feasible :)

Cheers,
Jes

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Gerd Hoffmann]

   Hi,

>> I think that would work well for spice. Spice uses shared memory from the
>> pci device for both the framebuffer and surfaces/commands, but this is
>
> Is that the only DMA do you do? That's good for this model.

Yes.  Spice does both reads and writes though, so a way to tag pages as 
dirty is needed.

cheers,
   Gerd

Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features

[Anthony Liguori]

On 04/26/2011 04:14 AM, Gerd Hoffmann wrote:
> Hi,
>
>>> I think that would work well for spice. Spice uses shared memory from
>>> the
>>> pci device for both the framebuffer and surfaces/commands, but this is
>>
>> Is that the only DMA do you do? That's good for this model.
>
> Yes. Spice does both reads and writes though, so a way to tag pages as
> dirty is needed.

Just implementing Spice as it currently is in a separate process isn't 
going to be useful IMHO.

I would think that the best approach would be to parse all of the ring 
requests in QEMU itself, and issue higher level commands to a separate 
process.  You can still have the video memory segment mapped in a 
separate process but QEMU should know enough about what's going on to 
take care of dirtying the memory.

Sort of like how we deal with SCSI passthrough.  We interpret enough of 
the command to hand it off to something else and then handle the return 
logic.

Having QEMU as an intermediary is important to preserve our current 
security model.  We shouldn't be passing unsanitized guest input to an 
external process.

Regards,

Anthony Liguori


> cheers,
> Gerd
>
>