[Qemu-devel] bidirectional data exchange between guest and host without network

[Qemu-devel] bidirectional data exchange between guest and host without network

[Anthony Lannuzel]

Dear All,

I want the host to be able to communicate with the host without using the
guest networking interfaces, is there a way this can be bidirectional ?
Here is what I have tested so far (command line options):

-drive file=fat::rw:/tmp/qemuShare
This allows the guest to read the host data, but when it writes to its
mounted device, the host doesn't see the data.

-fda file=fat::rw:/tmp/qemuShare
Apparently the guest has to eject the floppy for the host to see the data,
and I have read that the transfer speed is very low.

-smb : this requires guest networking (with 10.0.2.4 by default)

Has anyone already tried to do something similar ?

Regards
Anthony

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Anthony Lannuzel wrote:
> I want the host to be able to communicate with the host without using the
> guest networking interfaces, is there a way this can be bidirectional ?
> Here is what I have tested so far (command line options):

virtio-serial was designed for exactly this sort of thing.
Patches appear on the list from time to time.

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Anthony Lannuzel]

> virtio-serial was designed for exactly this sort of thing.
> Patches appear on the list from time to time.
>
> -- Jamie

Hi, thanks for replying.

I have just tried the following:

On the host I create a fat partition on /dev/sda8. Then I run

kvm -hda $IMG -drive file=/dev/sda8,if=virtio -cdrom /dev/cdrom -m 192

In the guest in do the following:

sudo mount /dev/vda /mnt/shared
sudo touch /mnt/shared/guestFile

On the host I mount the partition :

sudo mount /dev/sda8 /tmp/qemuShare

I can see the file guestFile.

Then both the host and guest add a file in the shared directory
(newHostFile and newGuestFile), but the other one does not see the change.
And when the guest unmounts /dev/vda, the hosts sees the 2 files written
by the guest (guestFile and newGuestFile) and loses its own file
(newHostFile).

It looks like I can not mount this block device from two OSs at the same
time, but isn't that virtio's purpose ?

I see a dirty workaround: sharing two partitions, one for the host to
write and one for the guest to write. But it still seems the writer has to
unmount /dev/sda8 for the data to be written on the other side. Am I wrong
?

Thanks
Anthony

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Lennart Sorensen]

On Tue, Jul 07, 2009 at 03:11:30PM +0200, Anthony Lannuzel wrote:
> Hi, thanks for replying.
> 
> I have just tried the following:
> 
> On the host I create a fat partition on /dev/sda8. Then I run
> 
> kvm -hda $IMG -drive file=/dev/sda8,if=virtio -cdrom /dev/cdrom -m 192
> 
> In the guest in do the following:
> 
> sudo mount /dev/vda /mnt/shared
> sudo touch /mnt/shared/guestFile
> 
> On the host I mount the partition :
> 
> sudo mount /dev/sda8 /tmp/qemuShare
> 
> I can see the file guestFile.
> 
> Then both the host and guest add a file in the shared directory
> (newHostFile and newGuestFile), but the other one does not see the change.
> And when the guest unmounts /dev/vda, the hosts sees the 2 files written
> by the guest (guestFile and newGuestFile) and loses its own file
> (newHostFile).
> 
> It looks like I can not mount this block device from two OSs at the same
> time, but isn't that virtio's purpose ?
> 
> I see a dirty workaround: sharing two partitions, one for the host to
> write and one for the guest to write. But it still seems the writer has to
> unmount /dev/sda8 for the data to be written on the other side. Am I wrong

Most filesystems do not support simultanious mounting.  Besides, allowing
multiple mounts at once would make any use of caching imposible.
Almost certainly what you are seeing is caused by caching since each
OS has no reason to believe what it read before isn't still valid so it
uses the cache.  After all it is normally safe to assume that the data
on disk can't change unless you write to it.

Some filesystems probably exist for high availability use to allow a
SAN to be mounted in multiple places at once.

-- 
Len Sorensen

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Lennart Sorensen wrote:
> Most filesystems do not support simultanious mounting.  Besides, allowing
> multiple mounts at once would make any use of caching imposible.
> Almost certainly what you are seeing is caused by caching since each
> OS has no reason to believe what it read before isn't still valid so it
> uses the cache.  After all it is normally safe to assume that the data
> on disk can't change unless you write to it.
> 
> Some filesystems probably exist for high availability use to allow a
> SAN to be mounted in multiple places at once.

Those filesystems generally communicate with each other over the
network, so that cache coherency is managed by that communication.

Another way to do it is add a cache coherency protocol to the "disk",
in addition to the usual read and write I/O operations.

I'm not aware of any SAN or shared storage which provides a cache
coherency protocol (though I'd be surprised if there isn't one).

There are several Linux filesystems designed for clusters which can
share a SAN provided they can communicate cache coherency & locking
information with each other over a network.

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Anthony Lannuzel]

> Lennart Sorensen wrote:
>> Most filesystems do not support simultanious mounting.  Besides,
>> allowing
>> multiple mounts at once would make any use of caching imposible.
>> Almost certainly what you are seeing is caused by caching since each
>> OS has no reason to believe what it read before isn't still valid so it
>> uses the cache.  After all it is normally safe to assume that the data
>> on disk can't change unless you write to it.
>>
>> Some filesystems probably exist for high availability use to allow a
>> SAN to be mounted in multiple places at once.
>
> Those filesystems generally communicate with each other over the
> network, so that cache coherency is managed by that communication.
>
> Another way to do it is add a cache coherency protocol to the "disk",
> in addition to the usual read and write I/O operations.
>
> I'm not aware of any SAN or shared storage which provides a cache
> coherency protocol (though I'd be surprised if there isn't one).
>
> There are several Linux filesystems designed for clusters which can
> share a SAN provided they can communicate cache coherency & locking
> information with each other over a network.

Yes, the thing is, I can not use the network, as it is used for another
purpose, so I think I have to content myself with the basic linux
filesystems.

I tried using a fat partition with the "sync" mount option, to avoid
caching on both sides (still using virtio on /dev/sda8 on the host and
/dev/vda on the guest).

The host now reads data written by the guest but the guest does not see
the host data.

Is there anything related to virtio that prevents this from working, as I
think the mount option provides me with a filesystem that fulfils the
conditions you just told me ?

Thanks
Anthony

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Lennart Sorensen]

On Wed, Jul 08, 2009 at 12:03:20PM +0200, Anthony Lannuzel wrote:
> Yes, the thing is, I can not use the network, as it is used for another
> purpose, so I think I have to content myself with the basic linux
> filesystems.
> 
> I tried using a fat partition with the "sync" mount option, to avoid
> caching on both sides (still using virtio on /dev/sda8 on the host and
> /dev/vda on the guest).

sync only affects write caching, it does nothing for read caching,
which is where your real problem is.

> The host now reads data written by the guest but the guest does not see
> the host data.
> 
> Is there anything related to virtio that prevents this from working, as I
> think the mount option provides me with a filesystem that fulfils the
> conditions you just told me ?

Just the fact any sane OS does read caching is what makes it not work.
Filesystems are NOT going to do this for you.  You can not share
filesystems like that unless explicitly designed for it.

I remember many years ago in highschool we had a small set of machines
that had a shared scsi drive between 5 machines (all daisy chained).
You would shutdown all the machines, then start up the master machine,
make the drive read/write, add new software and updates, then shutdown,
switch the drive to read only, and boot all the systems.  They could
then all use the software on the drive, but anything you wanted to save
had to be done to floppy disks on the individual machines.  Writing a
shared drive was simply not an option.

-- 
Len Sorensen

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Paul Brook]

>Yes, the thing is, I can not use the network, as it is used for another
>purpose, so I think I have to content myself with the basic linux
>filesystems.

That's just not going to work. Block devices don't provide the features 
necessary to implement a shared filesystem, you need some other mechanism 
(i.e. network) for handling locking and coherence. You can't take a regular 
filesystem and expect it to work. You have to use a proper network/clustered 
filesystem.

In theory SCSI has locking facilities, however this is not implemented by qemu 
(and probably not by your underlying storage), and AFAIK aren't used by any of 
the current cluster filesystems.

> I tried using a fat partition with the "sync" mount option, to avoid
> caching on both sides (still using virtio on /dev/sda8 on the host and
> /dev/vda on the guest).
>...
> Is there anything related to virtio that prevents this from working, as I
> think the mount option provides me with a filesystem that fulfils the
> conditions you just told me ?

I believe "sync" only disables the write cache.

Paul

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Anthony Lannuzel wrote:
> Yes, the thing is, I can not use the network, as it is used for another
> purpose, so I think I have to content myself with the basic linux
> filesystems.

Can you not create _another_ network device and use that?

QEMU lets you create lots of network devices.

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Anthony Lannuzel]

> Just the fact any sane OS does read caching is what makes it not work.
> Filesystems are NOT going to do this for you.  You can not share
> filesystems like that unless explicitly designed for it.
>
> I remember many years ago in highschool we had a small set of machines
> that had a shared scsi drive between 5 machines (all daisy chained).
> You would shutdown all the machines, then start up the master machine,
> make the drive read/write, add new software and updates, then shutdown,
> switch the drive to read only, and boot all the systems.  They could
> then all use the software on the drive, but anything you wanted to save
> had to be done to floppy disks on the individual machines.  Writing a
> shared drive was simply not an option.
>
> --
> Len Sorensen
>

> I believe "sync" only disables the write cache.
>
> Paul

Ok I got your point, this is clearly not possible with a regular filesystem.

> Can you not create _another_ network device and use that?
>
> QEMU lets you create lots of network devices.
>
> -- Jamie
>

No, I do not want the guest to be able to communicate with the host
network, so that is not an option.

Virtualbox provides a samba-like sharing feature, which does not uses the
network, but needs the virtualbox guest additions. Has this feature been
discussed for qemu ?

Thanks for your replies
Regards
Anthony

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Lennart Sorensen wrote:
> On Wed, Jul 08, 2009 at 12:03:20PM +0200, Anthony Lannuzel wrote:
> > I tried using a fat partition with the "sync" mount option, to avoid
> > caching on both sides (still using virtio on /dev/sda8 on the host and
> > /dev/vda on the guest).
> 
> sync only affects write caching, it does nothing for read caching,
> which is where your real problem is.
> 
> > The host now reads data written by the guest but the guest does not see
> > the host data.
> > 
> > Is there anything related to virtio that prevents this from working, as I
> > think the mount option provides me with a filesystem that fulfils the
> > conditions you just told me ?
> 
> Just the fact any sane OS does read caching is what makes it not work.
> Filesystems are NOT going to do this for you.  You can not share
> filesystems like that unless explicitly designed for it.

Many guest OSes have O_DIRECT or equivalent.  That bypasses read and
write caching, and can be used for direct access to the underlying
block device.

You can share data over a block device that way, by _not_ using a
filesystem, only using your own protocol on the block filesystem.

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Anthony Lannuzel wrote:
> > Can you not create _another_ network device and use that?
> > QEMU lets you create lots of network devices.
>
> No, I do not want the guest to be able to communicate with the host
> network, so that is not an option.

So create a network device that is only used for the private
communication, and isolate from the rest of the host network with
firewall rules.

I'll admit that can be a lot of work, if the host has a complex
network, or a dynamic one where IP addresses are unpredictable.

For that, on the latest Linux hosts we have network cgroups :-)

> Virtualbox provides a samba-like sharing feature, which does not uses the
> network, but needs the virtualbox guest additions. Has this feature been
> discussed for qemu ?

You can just use Samba.  See the "-smb" option.

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Richard W.M. Jones]

On Sat, Jul 11, 2009 at 01:04:00AM +0100, Jamie Lokier wrote:
> Anthony Lannuzel wrote:
> > > Can you not create _another_ network device and use that?
> > > QEMU lets you create lots of network devices.
> >
> > No, I do not want the guest to be able to communicate with the host
> > network, so that is not an option.
> 
> So create a network device that is only used for the private
> communication, and isolate from the rest of the host network with
> firewall rules.
> 
> I'll admit that can be a lot of work, if the host has a complex
> network, or a dynamic one where IP addresses are unpredictable.

This is precisely the reason why vmchannel is a good thing.

There is no IPv4 address you can give to the new interface which won't
have the potential to conflict with some other IPv4 address already in
use.  Extra network devices require special handling in firewall rules
and changes to the configuration of every network daemon in the
system.

If you don't add an extra network device to the guest, then you rely
on the host and guest being on the same network, which is not always
true.

While it's possible to configure the guest specially to avoid this,
that doesn't look much like our promise to run any vanilla guest as a
virtual machine.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Jamie Lokier]

Richard W.M. Jones wrote:
> On Sat, Jul 11, 2009 at 01:04:00AM +0100, Jamie Lokier wrote:
> > Anthony Lannuzel wrote:
> > > > Can you not create _another_ network device and use that?
> > > > QEMU lets you create lots of network devices.
> > >
> > > No, I do not want the guest to be able to communicate with the host
> > > network, so that is not an option.
> > 
> > So create a network device that is only used for the private
> > communication, and isolate from the rest of the host network with
> > firewall rules.
> > 
> > I'll admit that can be a lot of work, if the host has a complex
> > network, or a dynamic one where IP addresses are unpredictable.
> 
> This is precisely the reason why vmchannel is a good thing.
> 
> There is no IPv4 address you can give to the new interface which won't
> have the potential to conflict with some other IPv4 address already in
> use.  Extra network devices require special handling in firewall rules
> and changes to the configuration of every network daemon in the
> system.

A network interface with no IPv4 or IPv6 addresses assigned would
avoid most daemon problems.  Using a non-ethernet MAC type and
point-to-point would probably avoid the rest, even routing daemons,
NetworkManager and the like.  Use raw sockets over the interfaces.

Just a thought.

> If you don't add an extra network device to the guest, then you rely
> on the host and guest being on the same network, which is not always
> true.
> 
> While it's possible to configure the guest specially to avoid this,
> that doesn't look much like our promise to run any vanilla guest as a
> virtual machine.

I don't see how "any vanilla guest" can use vmchannel, since it
requires specific kernel support doesn't it, and therefore the guest
tools can only use vmchannel on new guest kernels built for it?

-- Jamie

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Richard W.M. Jones]

On Mon, Jul 13, 2009 at 11:38:32PM +0100, Jamie Lokier wrote:
> Richard W.M. Jones wrote:
> > There is no IPv4 address you can give to the new interface which won't
> > have the potential to conflict with some other IPv4 address already in
> > use.  Extra network devices require special handling in firewall rules
> > and changes to the configuration of every network daemon in the
> > system.
> 
> A network interface with no IPv4 or IPv6 addresses assigned would
> avoid most daemon problems.  Using a non-ethernet MAC type and
> point-to-point would probably avoid the rest, even routing daemons,
> NetworkManager and the like.  Use raw sockets over the interfaces.

Can you give a precise description of how to configure this with qemu?

> > While it's possible to configure the guest specially to avoid this,
> > that doesn't look much like our promise to run any vanilla guest as a
> > virtual machine.
> 
> I don't see how "any vanilla guest" can use vmchannel, since it
> requires specific kernel support doesn't it, and therefore the guest
> tools can only use vmchannel on new guest kernels built for it?

Point taken, but a vanilla guest + a support package, as with
VMWare Tools or Linux fullvirt + virtio drivers.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://et.redhat.com/~rjones/libguestfs/
See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html

Re: [Qemu-devel] bidirectional data exchange between guest and host without network

[Amit Shah]

On (Tue) Jul 07 2009 [15:11:30], Anthony Lannuzel wrote:
> 
> > virtio-serial was designed for exactly this sort of thing.
> > Patches appear on the list from time to time.
> >
> > -- Jamie
> 
> Hi, thanks for replying.
> 
> I have just tried the following:
> 
> On the host I create a fat partition on /dev/sda8. Then I run
> 
> kvm -hda $IMG -drive file=/dev/sda8,if=virtio -cdrom /dev/cdrom -m 192

What Jamie meant is virtio-serial, that's not yet included in qemu. I'm
working on the patches and the last snapshot I sent out is available
from here:

https://lists.linux-foundation.org/pipermail/virtualization/2009-July/013250.html

		Amit