* specifying rbd images in libvirt
@ 2011-08-27 19:19 Sage Weil
2011-09-01 12:46 ` [libvirt] " Daniel P. Berrange
0 siblings, 1 reply; 6+ messages in thread
From: Sage Weil @ 2011-08-27 19:19 UTC (permalink / raw)
To: libvir-list; +Cc: ceph-devel
Hi all,
Currently, you can specify an rbd (or nbd, sheepdog) image with xml
that looks like so:
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<source protocol='rbd' name='mypool/myimage'>
<host name='monhost1.mydomain.com' port='6789'/>
<host name='monhost2.mydomain.com' port='6789'/>
<host name='monhost3.mydomain.com' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
</disk>
This works okay if you have authentication disabled and all of the default
settings are okay. Usually, though, there are other options you need to
specify to librbd to make it do what you want.
The current schema can be abused by adding options after the image name
like so:
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'>
<host name='monhost1.mydomain.com' port='6789'/>
<host name='monhost2.mydomain.com' port='6789'/>
<host name='monhost3.mydomain.com' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
</disk>
This works only because that's what the qemu incantation looks like. In
general, though, this is ugly. I also doesn't generalize well to the
kernel-level rbd driver, which we'd like to also support, as that will
work with hypervisors other than qemu.
What about something more like this?
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<source protocol='rbd' name='mypool/myimage'>
<option name='conf'>/etc/ceph/ceph.conf</option>
<option name='id'>myusername</option>
<option name='foo'>bar</option>
<host name='monhost1.mydomain.com' port='6789'/>
<host name='monhost2.mydomain.com' port='6789'/>
<host name='monhost3.mydomain.com' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
</disk>
I'm not married to any particular syntax/schema, as long as there is a
generic way to specify name/value pairs to configure the driver. I think
the above would generalize well to other network block devices as well,
which presumably also want a way to feed in information other than a
server address (e.g. for authentication).
Does that look reasonable? If there are no objections we can work up some
patches and send them along!
Thanks-
sage
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] specifying rbd images in libvirt
2011-08-27 19:19 specifying rbd images in libvirt Sage Weil
@ 2011-09-01 12:46 ` Daniel P. Berrange
2011-09-01 21:07 ` Sage Weil
0 siblings, 1 reply; 6+ messages in thread
From: Daniel P. Berrange @ 2011-09-01 12:46 UTC (permalink / raw)
To: Sage Weil; +Cc: libvir-list, ceph-devel
On Sat, Aug 27, 2011 at 12:19:33PM -0700, Sage Weil wrote:
> Hi all,
>
> Currently, you can specify an rbd (or nbd, sheepdog) image with xml
> that looks like so:
>
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='writeback'/>
> <source protocol='rbd' name='mypool/myimage'>
> <host name='monhost1.mydomain.com' port='6789'/>
> <host name='monhost2.mydomain.com' port='6789'/>
> <host name='monhost3.mydomain.com' port='6789'/>
> </source>
> <target dev='vda' bus='virtio'/>
> </disk>
>
> This works okay if you have authentication disabled and all of the default
> settings are okay. Usually, though, there are other options you need to
> specify to librbd to make it do what you want.
>
> The current schema can be abused by adding options after the image name
> like so:
>
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='writeback'/>
> <source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'>
> <host name='monhost1.mydomain.com' port='6789'/>
> <host name='monhost2.mydomain.com' port='6789'/>
> <host name='monhost3.mydomain.com' port='6789'/>
> </source>
> <target dev='vda' bus='virtio'/>
> </disk>
>
> This works only because that's what the qemu incantation looks like. In
> general, though, this is ugly. I also doesn't generalize well to the
> kernel-level rbd driver, which we'd like to also support, as that will
> work with hypervisors other than qemu.
Also, we should be doing some validation on the content of
the 'name' attribute to prevent abuse like that.
> What about something more like this?
>
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='writeback'/>
> <source protocol='rbd' name='mypool/myimage'>
> <option name='conf'>/etc/ceph/ceph.conf</option>
> <option name='id'>myusername</option>
> <option name='foo'>bar</option>
> <host name='monhost1.mydomain.com' port='6789'/>
> <host name='monhost2.mydomain.com' port='6789'/>
> <host name='monhost3.mydomain.com' port='6789'/>
> </source>
> <target dev='vda' bus='virtio'/>
> </disk>
>
> I'm not married to any particular syntax/schema, as long as there is a
> generic way to specify name/value pairs to configure the driver. I think
> the above would generalize well to other network block devices as well,
> which presumably also want a way to feed in information other than a
> server address (e.g. for authentication).
>
> Does that look reasonable? If there are no objections we can work up some
> patches and send them along!
We generally prefer to add explicit models for attributes, rather than
just do a generic key/value passthrough.
For authentication credentials, we also need to consider security
implications of keeping them in the XML. For qcow2 encryption
keys, we leverage the separate libvirt secrets management APIs
to provide the keys outside the XML. IMHO we should likely do the
same for any keys required to login to network block devices.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] specifying rbd images in libvirt
2011-09-01 12:46 ` [libvirt] " Daniel P. Berrange
@ 2011-09-01 21:07 ` Sage Weil
2011-09-07 23:54 ` Sage Weil
0 siblings, 1 reply; 6+ messages in thread
From: Sage Weil @ 2011-09-01 21:07 UTC (permalink / raw)
To: Daniel P. Berrange; +Cc: libvir-list, ceph-devel
On Thu, 1 Sep 2011, Daniel P. Berrange wrote:
> On Sat, Aug 27, 2011 at 12:19:33PM -0700, Sage Weil wrote:
> > Hi all,
> >
> > Currently, you can specify an rbd (or nbd, sheepdog) image with xml
> > that looks like so:
> >
> > <disk type='network' device='disk'>
> > <driver name='qemu' type='raw' cache='writeback'/>
> > <source protocol='rbd' name='mypool/myimage'>
> > <host name='monhost1.mydomain.com' port='6789'/>
> > <host name='monhost2.mydomain.com' port='6789'/>
> > <host name='monhost3.mydomain.com' port='6789'/>
> > </source>
> > <target dev='vda' bus='virtio'/>
> > </disk>
> >
> > This works okay if you have authentication disabled and all of the default
> > settings are okay. Usually, though, there are other options you need to
> > specify to librbd to make it do what you want.
> >
> > The current schema can be abused by adding options after the image name
> > like so:
> >
> > <disk type='network' device='disk'>
> > <driver name='qemu' type='raw' cache='writeback'/>
> > <source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'>
> > <host name='monhost1.mydomain.com' port='6789'/>
> > <host name='monhost2.mydomain.com' port='6789'/>
> > <host name='monhost3.mydomain.com' port='6789'/>
> > </source>
> > <target dev='vda' bus='virtio'/>
> > </disk>
> >
> > This works only because that's what the qemu incantation looks like. In
> > general, though, this is ugly. I also doesn't generalize well to the
> > kernel-level rbd driver, which we'd like to also support, as that will
> > work with hypervisors other than qemu.
>
> Also, we should be doing some validation on the content of
> the 'name' attribute to prevent abuse like that.
>
> > What about something more like this?
> >
> > <disk type='network' device='disk'>
> > <driver name='qemu' type='raw' cache='writeback'/>
> > <source protocol='rbd' name='mypool/myimage'>
> > <option name='conf'>/etc/ceph/ceph.conf</option>
> > <option name='id'>myusername</option>
> > <option name='foo'>bar</option>
> > <host name='monhost1.mydomain.com' port='6789'/>
> > <host name='monhost2.mydomain.com' port='6789'/>
> > <host name='monhost3.mydomain.com' port='6789'/>
> > </source>
> > <target dev='vda' bus='virtio'/>
> > </disk>
> >
> > I'm not married to any particular syntax/schema, as long as there is a
> > generic way to specify name/value pairs to configure the driver. I think
> > the above would generalize well to other network block devices as well,
> > which presumably also want a way to feed in information other than a
> > server address (e.g. for authentication).
> >
> > Does that look reasonable? If there are no objections we can work up some
> > patches and send them along!
>
> We generally prefer to add explicit models for attributes, rather than
> just do a generic key/value passthrough.
Fair enough. For the qemu/librbd and kernel drivers, the key fields we
need are:
id - ceph userid to authenticate as
snap - snapshot name (optional; used if mapping a read-only snapshot)
The goal would be to have libvirt describe rbd images in a generic way,
and let you managed them as a storage pool via librbd. Then, when it
comes time to spin up an actual VM, either use the native qemu support or
map a kernel block device. To control that it might make sense to have an
option like
driver - librbd (qemu-only), kernel, or auto (default)
to let users control which driver to use in the qemu case.
> For authentication credentials, we also need to consider security
> implications of keeping them in the XML. For qcow2 encryption
> keys, we leverage the separate libvirt secrets management APIs
> to provide the keys outside the XML. IMHO we should likely do the
> same for any keys required to login to network block devices.
Usually we keep secrets in a file on disk and reference the file when
starting up librbd or mapping a kernel rbd device. It's possible to
specify it explicitly but that is discouraged. So by specifying id=foo
above, librbd would normally look for a key called foo in its default
keyring file.
We can definitely use the libvirt secrets API to store these keys there,
too. In that case, we would need something like
keyname - name of the libvirt secret to use for authentication
What do you think?
sage
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] specifying rbd images in libvirt
2011-09-01 21:07 ` Sage Weil
@ 2011-09-07 23:54 ` Sage Weil
2011-09-16 10:28 ` Daniel Veillard
0 siblings, 1 reply; 6+ messages in thread
From: Sage Weil @ 2011-09-07 23:54 UTC (permalink / raw)
To: Daniel P. Berrange; +Cc: libvir-list, ceph-devel
On Thu, 1 Sep 2011, Sage Weil wrote:
> On Thu, 1 Sep 2011, Daniel P. Berrange wrote:
> > On Sat, Aug 27, 2011 at 12:19:33PM -0700, Sage Weil wrote:
> > > Hi all,
> > >
> > > Currently, you can specify an rbd (or nbd, sheepdog) image with xml
> > > that looks like so:
> > >
> > > <disk type='network' device='disk'>
> > > <driver name='qemu' type='raw' cache='writeback'/>
> > > <source protocol='rbd' name='mypool/myimage'>
> > > <host name='monhost1.mydomain.com' port='6789'/>
> > > <host name='monhost2.mydomain.com' port='6789'/>
> > > <host name='monhost3.mydomain.com' port='6789'/>
> > > </source>
> > > <target dev='vda' bus='virtio'/>
> > > </disk>
> > >
> > > This works okay if you have authentication disabled and all of the default
> > > settings are okay. Usually, though, there are other options you need to
> > > specify to librbd to make it do what you want.
> > >
> > > The current schema can be abused by adding options after the image name
> > > like so:
> > >
> > > <disk type='network' device='disk'>
> > > <driver name='qemu' type='raw' cache='writeback'/>
> > > <source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'>
> > > <host name='monhost1.mydomain.com' port='6789'/>
> > > <host name='monhost2.mydomain.com' port='6789'/>
> > > <host name='monhost3.mydomain.com' port='6789'/>
> > > </source>
> > > <target dev='vda' bus='virtio'/>
> > > </disk>
> > >
> > > This works only because that's what the qemu incantation looks like. In
> > > general, though, this is ugly. I also doesn't generalize well to the
> > > kernel-level rbd driver, which we'd like to also support, as that will
> > > work with hypervisors other than qemu.
> >
> > Also, we should be doing some validation on the content of
> > the 'name' attribute to prevent abuse like that.
> >
> > > What about something more like this?
> > >
> > > <disk type='network' device='disk'>
> > > <driver name='qemu' type='raw' cache='writeback'/>
> > > <source protocol='rbd' name='mypool/myimage'>
> > > <option name='conf'>/etc/ceph/ceph.conf</option>
> > > <option name='id'>myusername</option>
> > > <option name='foo'>bar</option>
> > > <host name='monhost1.mydomain.com' port='6789'/>
> > > <host name='monhost2.mydomain.com' port='6789'/>
> > > <host name='monhost3.mydomain.com' port='6789'/>
> > > </source>
> > > <target dev='vda' bus='virtio'/>
> > > </disk>
> > >
> > > I'm not married to any particular syntax/schema, as long as there is a
> > > generic way to specify name/value pairs to configure the driver. I think
> > > the above would generalize well to other network block devices as well,
> > > which presumably also want a way to feed in information other than a
> > > server address (e.g. for authentication).
> > >
> > > Does that look reasonable? If there are no objections we can work up some
> > > patches and send them along!
> >
> > We generally prefer to add explicit models for attributes, rather than
> > just do a generic key/value passthrough.
>
> Fair enough. For the qemu/librbd and kernel drivers, the key fields we
> need are:
>
> id - ceph userid to authenticate as
> snap - snapshot name (optional; used if mapping a read-only snapshot)
>
> The goal would be to have libvirt describe rbd images in a generic way,
> and let you managed them as a storage pool via librbd. Then, when it
> comes time to spin up an actual VM, either use the native qemu support or
> map a kernel block device. To control that it might make sense to have an
> option like
>
> driver - librbd (qemu-only), kernel, or auto (default)
>
> to let users control which driver to use in the qemu case.
>
> > For authentication credentials, we also need to consider security
> > implications of keeping them in the XML. For qcow2 encryption
> > keys, we leverage the separate libvirt secrets management APIs
> > to provide the keys outside the XML. IMHO we should likely do the
> > same for any keys required to login to network block devices.
>
> Usually we keep secrets in a file on disk and reference the file when
> starting up librbd or mapping a kernel rbd device. It's possible to
> specify it explicitly but that is discouraged. So by specifying id=foo
> above, librbd would normally look for a key called foo in its default
> keyring file.
>
> We can definitely use the libvirt secrets API to store these keys there,
> too. In that case, we would need something like
>
> keyname - name of the libvirt secret to use for authentication
>
> What do you think?
Shall I take silence as general agreement and start putting together
patches?
I don't think there's anything controversial here. I suspect the main
concern is that these fields are named in a sufficiently general fashion
so that they can be used for other backend storage types.
sage
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] specifying rbd images in libvirt
2011-09-07 23:54 ` Sage Weil
@ 2011-09-16 10:28 ` Daniel Veillard
2011-09-20 4:13 ` Sage Weil
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Veillard @ 2011-09-16 10:28 UTC (permalink / raw)
To: Sage Weil; +Cc: Daniel P. Berrange, libvir-list, ceph-devel
On Wed, Sep 07, 2011 at 04:54:37PM -0700, Sage Weil wrote:
> On Thu, 1 Sep 2011, Sage Weil wrote:
> > On Thu, 1 Sep 2011, Daniel P. Berrange wrote:
> > > On Sat, Aug 27, 2011 at 12:19:33PM -0700, Sage Weil wrote:
> > > > Hi all,
> > > >
> > > > Currently, you can specify an rbd (or nbd, sheepdog) image with xml
> > > > that looks like so:
> > > >
> > > > <disk type='network' device='disk'>
> > > > <driver name='qemu' type='raw' cache='writeback'/>
> > > > <source protocol='rbd' name='mypool/myimage'>
> > > > <host name='monhost1.mydomain.com' port='6789'/>
> > > > <host name='monhost2.mydomain.com' port='6789'/>
> > > > <host name='monhost3.mydomain.com' port='6789'/>
> > > > </source>
> > > > <target dev='vda' bus='virtio'/>
> > > > </disk>
> > > >
> > > > This works okay if you have authentication disabled and all of the default
> > > > settings are okay. Usually, though, there are other options you need to
> > > > specify to librbd to make it do what you want.
> > > >
> > > > The current schema can be abused by adding options after the image name
> > > > like so:
> > > >
> > > > <disk type='network' device='disk'>
> > > > <driver name='qemu' type='raw' cache='writeback'/>
> > > > <source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'>
> > > > <host name='monhost1.mydomain.com' port='6789'/>
> > > > <host name='monhost2.mydomain.com' port='6789'/>
> > > > <host name='monhost3.mydomain.com' port='6789'/>
> > > > </source>
> > > > <target dev='vda' bus='virtio'/>
> > > > </disk>
> > > >
> > > > This works only because that's what the qemu incantation looks like. In
> > > > general, though, this is ugly. I also doesn't generalize well to the
> > > > kernel-level rbd driver, which we'd like to also support, as that will
> > > > work with hypervisors other than qemu.
> > >
> > > Also, we should be doing some validation on the content of
> > > the 'name' attribute to prevent abuse like that.
> > >
> > > > What about something more like this?
> > > >
> > > > <disk type='network' device='disk'>
> > > > <driver name='qemu' type='raw' cache='writeback'/>
> > > > <source protocol='rbd' name='mypool/myimage'>
> > > > <option name='conf'>/etc/ceph/ceph.conf</option>
> > > > <option name='id'>myusername</option>
> > > > <option name='foo'>bar</option>
> > > > <host name='monhost1.mydomain.com' port='6789'/>
> > > > <host name='monhost2.mydomain.com' port='6789'/>
> > > > <host name='monhost3.mydomain.com' port='6789'/>
> > > > </source>
> > > > <target dev='vda' bus='virtio'/>
> > > > </disk>
> > > >
> > > > I'm not married to any particular syntax/schema, as long as there is a
> > > > generic way to specify name/value pairs to configure the driver. I think
> > > > the above would generalize well to other network block devices as well,
> > > > which presumably also want a way to feed in information other than a
> > > > server address (e.g. for authentication).
> > > >
> > > > Does that look reasonable? If there are no objections we can work up some
> > > > patches and send them along!
> > >
> > > We generally prefer to add explicit models for attributes, rather than
> > > just do a generic key/value passthrough.
> >
> > Fair enough. For the qemu/librbd and kernel drivers, the key fields we
> > need are:
> >
> > id - ceph userid to authenticate as
> > snap - snapshot name (optional; used if mapping a read-only snapshot)
> >
> > The goal would be to have libvirt describe rbd images in a generic way,
> > and let you managed them as a storage pool via librbd. Then, when it
> > comes time to spin up an actual VM, either use the native qemu support or
> > map a kernel block device. To control that it might make sense to have an
> > option like
> >
> > driver - librbd (qemu-only), kernel, or auto (default)
> >
> > to let users control which driver to use in the qemu case.
> >
> > > For authentication credentials, we also need to consider security
> > > implications of keeping them in the XML. For qcow2 encryption
> > > keys, we leverage the separate libvirt secrets management APIs
> > > to provide the keys outside the XML. IMHO we should likely do the
> > > same for any keys required to login to network block devices.
> >
> > Usually we keep secrets in a file on disk and reference the file when
> > starting up librbd or mapping a kernel rbd device. It's possible to
> > specify it explicitly but that is discouraged. So by specifying id=foo
> > above, librbd would normally look for a key called foo in its default
> > keyring file.
> >
> > We can definitely use the libvirt secrets API to store these keys there,
> > too. In that case, we would need something like
> >
> > keyname - name of the libvirt secret to use for authentication
> >
> > What do you think?
>
> Shall I take silence as general agreement and start putting together
> patches?
I would think so, yes :-)
> I don't think there's anything controversial here. I suspect the main
> concern is that these fields are named in a sufficiently general fashion
> so that they can be used for other backend storage types.
I guess the best is to start with the patches with the XML changes as
an RFC and then add the code to wire the backend as a second step. Even
if both are likely to be applied as a batch, the XML code review is
likely to be the most controversial :-)
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] specifying rbd images in libvirt
2011-09-16 10:28 ` Daniel Veillard
@ 2011-09-20 4:13 ` Sage Weil
0 siblings, 0 replies; 6+ messages in thread
From: Sage Weil @ 2011-09-20 4:13 UTC (permalink / raw)
To: Daniel Veillard; +Cc: Daniel P. Berrange, libvir-list, ceph-devel
On Fri, 16 Sep 2011, Daniel Veillard wrote:
> > I don't think there's anything controversial here. I suspect the main
> > concern is that these fields are named in a sufficiently general fashion
> > so that they can be used for other backend storage types.
>
> I guess the best is to start with the patches with the XML changes as
> an RFC and then add the code to wire the backend as a second step. Even
> if both are likely to be applied as a batch, the XML code review is
> likely to be the most controversial :-)
The first two patches of the series I posted last week add the
authentication bits to src/conf/. I just updated to include changes to
the schema files (docs/schemas) and am reposting.
Thanks!
sage
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-09-20 4:13 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-08-27 19:19 specifying rbd images in libvirt Sage Weil
2011-09-01 12:46 ` [libvirt] " Daniel P. Berrange
2011-09-01 21:07 ` Sage Weil
2011-09-07 23:54 ` Sage Weil
2011-09-16 10:28 ` Daniel Veillard
2011-09-20 4:13 ` Sage Weil
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.