All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [RFC][PATCH v6 00/23] virtagent: host/guest RPC communication agent
@ 2011-01-17 13:14 Michael Roth
  2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 01/23] Move code related to fd handlers into utility functions Michael Roth
                   ` (24 more replies)
  0 siblings, 25 replies; 70+ messages in thread
From: Michael Roth @ 2011-01-17 13:14 UTC (permalink / raw)
  To: qemu-devel
  Cc: agl, stefanha, Jes.Sorensen, marcel.mittelstaedt, mdroth,
	markus_mueller, aliguori, ryanh, abeekhof

These patches apply to master (1-14-2011), and can also be obtained from:
git://repo.or.cz/qemu/mdroth.git virtagent_v6

CHANGES IN V6:

 - Added a sentinel value to reliably detect the start of an "http" hdr. Used to skip past partially sent http content from previous "sessions"
 - Added http hdr tag (currently hardcoded for testing, will switch to uuid) to filter out valid-but-unexpected content in channel from previous "sessions"
 - Added timeout mechanism to avoid hanging monitor when agent isn't running
 - Added timed back-off on read's from a virtio-serial that result in ret=0 to avoid spinning if host isn't connected. 
 - Added daemonize flags to qemu-va
 - Added sane defaults for channel type and virtio-serial port path
 - Various bug fixes for state machine/job handling logic

CHANGES IN V5:

 - Dependency on virtproxy dropped, virtagent now handles transport and multiplexing of bi-directional RPCs internally
 - Removed duplification of qemu_set_fd_handler()-centered i/o code. Support for interacting with objects that use qemu_set_fd_handler() now available to tools via qemu-tools.c and a set of generalized utility functions
 - Fixed memory leaks in client/monitor functions
 - Various cleanups

CHANGES IN V4:

 - Added guest agent capabilities negotiation
 - Added RPC/monitor command to invoke guest shutdown/reboot/powerdown
 - Added RPC/monitor command to the guest agent
 - Added guest startup notification ("hello")
 - Added syslog()'ing of guest agent RPCs
 - Various cleanups

CHANGES IN V3:

 - Integrated virtagent server into virtproxy chardev. Usage examples below.
 - Consolidated RPC server/client setup into a pair of init routines
 - Fixed buffer overflow in agent_viewfile() and various memory leaks

CHANGES IN V2:

 - All RPC communication is now done using asynchronous/non-blocking read/write handlers
 - Previously fork()'d RPC server loop is now integrated into qemu-vp/virtproxy i/o loop
 - Cleanups/suggestions from previous RFC

OVERVIEW:

There are a wide range of use cases motivating the need for a guest agent of some sort to extend the functionality/usability/control offered by QEMU. Some examples include graceful guest shutdown/reboot and notifications thereof, copy/paste syncing between host/guest, guest statistics gathering, file access, etc.

Ideally these would all be served by a single, easilly extensible agent that can be deployed in a wide range of guests. Virtagent is an XMLRPC server, integrated into QEMU and a simple guest daemon, aimed at providing this type of functionality.

DESIGN:

There are actually 2 RPC servers:

1) a server in the guest agent which handles RPC requests from QEMU
2) a server in the host, integrated into the virtagent chardev, to handle RPC requests sent by the guest agent (mainly for handling asynchronous events reported by the agent).

Communication is done via RPCs (XMLRPC/HTTP between host and guest), albeit with a non-standard implementation that allows for multiplexing server/client RPC over a single virtio/isa serial channel.

EXAMPLE USAGE:

 - Configure guest agent to talk to host via virtio-serial
    # start guest with virtio-serial/virtagent. for example (RHEL6):
    qemu \
    -chardev virtagent,id=test0 \
    -device virtio-serial \
    -device virtserialport,chardev=test0,name=virtagent0 \
    -monitor stdio
    ...
    # in the guest:
    ./qemu-va -c virtio-serial -p /dev/virtio-ports/virtagent0
    ...
    # monitor commands
    (qemu) agent_viewdmesg
    [139311.710326] wlan0: deauthenticating from 00:30:bd:f7:12:d5 by local choice (reason=3)
    [139323.469857] wlan0: deauthenticating from 00:21:29:cd:41:ee by local choice (reason=3)
    ...
    [257683.375646] wlan0: authenticated
    [257683.375684] wlan0: associate with AP 00:30:bd:f7:12:d5 (try 1)
    [257683.377932] wlan0: RX AssocResp from 00:30:bd:f7:12:d5 (capab=0x411 status=0 aid=4)
    [257683.377940] wlan0: associated
    
    (qemu) agent_viewfile /proc/meminfo
    MemTotal:        3985488 kB
    MemFree:          400524 kB
    Buffers:          220556 kB
    Cached:          2073160 kB
    SwapCached:            0 kB
    ...
    Hugepagesize:       2048 kB
    DirectMap4k:        8896 kB
    DirectMap2M:     4110336 kB
    (qemu) agent_shutdown powerdown
    (qemu)

KNOWN ISSUES/PLANS:
 - Implement RPC for guest script/command execution
 - Use UUIDs for generating unique tags for each http request/response
 - Scan for sentinel value while reading http content as well to immediately detect truncated requests/responses and avoid accidentally consuming new ones
 - switch to standard logging/trace mechanisms
 - outstanding requests should be reset if we get a hello notification (this implies guest/guest agent restart)
 - the client socket that qemu connects to send RPCs is a hardcoded filepath. This is unacceptable as the socket is channel/process specific and things will break when multiple guests are started.

[RFC][PATCH v6 01/23] Move code related to fd handlers into utility functions                                                                            
[RFC][PATCH v6 02/23] Add qemu_set_fd_handler() wrappers to qemu-tools.c
[RFC][PATCH v6 03/23] Make qemu timers available for tools
[RFC][PATCH v6 04/23] virtagent: common code for managing client/server rpc jobs
[RFC][PATCH v6 05/23] virtagent: transport definitions read/send callback functions
[RFC][PATCH v6 06/23] virtagent: base client definitions
[RFC][PATCH v6 07/23] virtagent: base server definitions
[RFC][PATCH v6 08/23] virtagent: add va.getfile RPC
[RFC][PATCH v6 09/23] virtagent: add agent_viewfile qmp/hmp command
[RFC][PATCH v6 10/23] virtagent: add va.getdmesg RPC
[RFC][PATCH v6 11/23] virtagent: add agent_viewdmesg qmp/hmp commands
[RFC][PATCH v6 12/23] virtagent: add va.shutdown RPC
[RFC][PATCH v6 13/23] virtagent: add agent_shutdown qmp/hmp commands
[RFC][PATCH v6 14/23] virtagent: add va.ping RPC
[RFC][PATCH v6 15/23] virtagent: add agent_ping qmp/hmp commands
[RFC][PATCH v6 16/23] virtagent: add agent_capabilities qmp/hmp commands
[RFC][PATCH v6 17/23] virtagent: add client capabilities init function
[RFC][PATCH v6 18/23] virtagent: add va.hello RPC
[RFC][PATCH v6 19/23] virtagent: add "hello" notification function for guest agent
[RFC][PATCH v6 20/23] virtagent: add va.capabilities RPC
[RFC][PATCH v6 21/23] virtagent: add virtagent guest daemon
[RFC][PATCH v6 22/23] virtagent: integrate virtagent server/client via chardev
[RFC][PATCH v6 23/23] virtagent: various bits to build QEMU with virtagent

 Makefile           |    4 +-
 Makefile.objs      |    2 +-
 Makefile.target    |    2 +-
 configure          |   32 ++
 cpus.c             |   83 +----
 hmp-commands.hx    |   80 ++++
 monitor.c          |    1 +
 qemu-char.c        |   44 +++
 qemu-char.h        |    4 +
 qemu-ioh.c         |  208 +++++++++++
 qemu-ioh.h         |   43 +++
 qemu-tool.c        |  115 ++++++-
 qemu-tool.h        |   26 ++
 qemu-va.c          |  238 ++++++++++++
 qerror.c           |    8 +
 qerror.h           |    6 +
 qmp-commands.hx    |  164 +++++++++
 virtagent-common.c | 1028 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 virtagent-common.h |   73 ++++
 virtagent-server.c |  350 ++++++++++++++++++
 virtagent-server.h |   34 ++
 virtagent.c        |  642 ++++++++++++++++++++++++++++++++
 virtagent.h        |   50 +++
 vl.c               |   86 +----
 25 files changed, 3177 insertions(+), 148 deletions(-)

^ permalink raw reply	[flat|nested] 70+ messages in thread

end of thread, other threads:[~2011-02-21 13:38 UTC | newest]

Thread overview: 70+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-17 13:14 [Qemu-devel] [RFC][PATCH v6 00/23] virtagent: host/guest RPC communication agent Michael Roth
2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 01/23] Move code related to fd handlers into utility functions Michael Roth
2011-01-17 13:56   ` Gerd Hoffmann
2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 02/23] Add qemu_set_fd_handler() wrappers to qemu-tools.c Michael Roth
2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 03/23] Make qemu timers available for tools Michael Roth
2011-01-21 16:30   ` [Qemu-devel] " Jes Sorensen
2011-01-21 17:26     ` Michael Roth
2011-01-24  7:56       ` Jes Sorensen
2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 04/23] virtagent: common code for managing client/server rpc jobs Michael Roth
2011-01-17 13:14 ` [Qemu-devel] [RFC][PATCH v6 05/23] virtagent: transport definitions read/send callback functions Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 06/23] virtagent: base client definitions Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 07/23] virtagent: base server definitions Michael Roth
2011-01-21 16:38   ` [Qemu-devel] " Jes Sorensen
2011-01-21 17:55     ` Michael Roth
2011-01-24 10:16       ` Jes Sorensen
2011-01-24 16:51         ` Michael Roth
2011-01-24 17:04           ` Jes Sorensen
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 08/23] virtagent: add va.getfile RPC Michael Roth
2011-01-21 16:40   ` [Qemu-devel] " Jes Sorensen
2011-01-21 17:20     ` Daniel P. Berrange
2011-01-21 18:23       ` Michael Roth
2011-01-24 22:08         ` Richard W.M. Jones
2011-01-24 22:20           ` Richard W.M. Jones
2011-01-24 22:26             ` Anthony Liguori
2011-01-24 22:48               ` Richard W.M. Jones
2011-01-24 23:40                 ` Anthony Liguori
2011-01-25  0:22                   ` Michael Roth
2011-01-25  0:25                     ` Anthony Liguori
2011-01-25  9:21                   ` Richard W.M. Jones
2011-01-25 15:12                     ` Anthony Liguori
2011-01-25 15:43                       ` Richard W.M. Jones
2011-01-26 13:01                         ` Richard W.M. Jones
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 09/23] virtagent: add agent_viewfile qmp/hmp command Michael Roth
2011-01-21 16:41   ` [Qemu-devel] " Jes Sorensen
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 10/23] virtagent: add va.getdmesg RPC Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 11/23] virtagent: add agent_viewdmesg qmp/hmp commands Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 12/23] virtagent: add va.shutdown RPC Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 13/23] virtagent: add agent_shutdown qmp/hmp commands Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 14/23] virtagent: add va.ping RPC Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 15/23] virtagent: add agent_ping qmp/hmp commands Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 16/23] virtagent: add agent_capabilities " Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 17/23] virtagent: add client capabilities init function Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 18/23] virtagent: add va.hello RPC Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 19/23] virtagent: add "hello" notification function for guest agent Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 20/23] virtagent: add va.capabilities RPC Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 21/23] virtagent: add virtagent guest daemon Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 22/23] virtagent: integrate virtagent server/client via chardev Michael Roth
2011-01-17 13:15 ` [Qemu-devel] [RFC][PATCH v6 23/23] virtagent: various bits to build QEMU with virtagent Michael Roth
2011-01-24 10:24   ` [Qemu-devel] " Jes Sorensen
2011-01-17 13:53 ` [Qemu-devel] [RFC][PATCH v6 00/23] virtagent: host/guest RPC communication agent Gerd Hoffmann
2011-01-17 14:53   ` Michael Roth
2011-01-18 14:02     ` Gerd Hoffmann
2011-01-18 14:13       ` Anthony Liguori
2011-01-31 14:41         ` Michael Roth
2011-02-01 22:18           ` Michael Roth
2011-02-14  9:49             ` Gerd Hoffmann
2011-02-16 16:04 ` Jes Sorensen
2011-02-16 17:22   ` Michael Roth
2011-02-17  8:26     ` Jes Sorensen
2011-02-17  9:08       ` Dor Laor
2011-02-17 14:39       ` Michael Roth
2011-02-18 12:45         ` Jes Sorensen
2011-02-18 14:07           ` Anthony Liguori
2011-02-18 14:30             ` Jes Sorensen
2011-02-18 14:57               ` Anthony Liguori
2011-02-21  8:32                 ` Jes Sorensen
2011-02-21 13:36                   ` Michael Roth
2011-02-21 13:38                     ` Jes Sorensen
2011-02-18 15:22             ` Gerd Hoffmann
2011-02-18 15:25               ` Anthony Liguori

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.