Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

On 2016年06月01日 01:22, Eric Blake wrote:
> On 05/31/2016 10:30 AM, wexu@redhat.com wrote:
>> From: Wei Xu <wexu@redhat.com>
>>
>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>> create a unix socket for a chardev when managing VMs with libvirt,
>> because qemu don't have sufficient permissions in this case, and
>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>> passing it to qemu.
>
> Any reason this wasn't sent to the list?
Sorry, just forgot the list, also add Michal in the loop.

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>> From: Wei Xu <wexu@redhat.com>
>>
>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>> create a unix socket for a chardev when managing VMs with libvirt,
>> because qemu don't have sufficient permissions in this case, and
>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>> passing it to qemu.
>
> This sounds like a bug in libvirt, or selinux, or a mistaken configuration
> of the guest. It is entirely possible for QEMU to create a unix socket - not
> least because that is exactly what QEMU uses for its QMP monitor backend.
> Looking at your example command line, I think the issue is simply that you
> should be putting the sockets in a different location. ie at
> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has permission to
> create sockets already.
ah.. adjusting permission or file location can solve this problem, i'm 
guessing maybe this is a more security concern, the socket is used as a 
network interface for a vm, similar as the qcow image file, thus should 
prevent it to be arbitrarily accessed.

Michael, do you have any comment on this?

>
> Alternatively you could enhance the SELinux policy to grant svirt_t the
> permission to create sockets under /var/run/openvswitch too.
>
>> I finished a RFC patch for Unix socket after a glance of the code,
>> and not sure if this is right or there maybe other side-effects,
>> please point me out.
>>
>> I tested it for both server and client mode 'PF_UNIX' socket with a VM
>> running vhost-user.
>>
>> Old command line:
>> -chardev socket,id=char0,path=/var/run/openvswitch/vhost-user1,server
>>
>> New command line:
>> -chardev socket,id=char0,path=/var/run/openvswitch/vhost-user1,server,sockfd=$n
>>
>> because unix socket is bundled with a path, so it should be kept even with the
>> 'fd' is indicated, this looks odd, any comments?
>
> Yes, this syntax doesn't really make sense. The passed in sockfd may not
> even be a UNIX socket - it could be a TCP socket. As such, the 'sockfd'
> option should be mutually exclusive with the 'path' and 'host' options.
> ie you can only supply one out of 'sockfd', 'path', or 'host'.
Currently i just add it for unix socket, and the connect/listen syscall 
must have a path name, an inet socket doesn't need this param at all, 
should it be supported also?

>
>
> FWIW, I think the ability to pass a pre-opened socket FD with the
> -chardev socket backend is a useful enhancement, even though I don't
> think you need this in order to fix the problem you have.
OK, thanks for you quick feedback, i just wonder if 'add-fd' and 
qemu_open() can be a more general solution.
>
> Regards,
> Daniel
>

Re: [Qemu-devel] [RFC Patch 2/3] chardev: save the passed in 'fd' parameter during parsing

[Wei Xu]

On 2016年06月01日 01:26, Eric Blake wrote:
> On 05/31/2016 10:30 AM, wexu@redhat.com wrote:
>> From: Wei Xu <wexu@redhat.com>
>>
>> Save the 'fd' paramter as unix socket 'sockfd' member.
>>
>> Signed-off-by: Wei Xu <wexu@redhat.com>
>> ---
>>   qemu-char.c | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/qemu-char.c b/qemu-char.c
>> index ea9c02e..8d20494 100644
>> --- a/qemu-char.c
>> +++ b/qemu-char.c
>> @@ -3664,6 +3664,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
>>       bool is_telnet      = qemu_opt_get_bool(opts, "telnet", false);
>>       bool do_nodelay     = !qemu_opt_get_bool(opts, "delay", true);
>>       int64_t reconnect   = qemu_opt_get_number(opts, "reconnect", 0);
>> +    const int32_t fd = (int32_t)qemu_opt_get_number(opts, "sockfd", 0);
>>       const char *path = qemu_opt_get(opts, "path");
>>       const char *host = qemu_opt_get(opts, "host");
>>       const char *port = qemu_opt_get(opts, "port");
>> @@ -3708,6 +3709,12 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
>>           addr->type = SOCKET_ADDRESS_KIND_UNIX;
>>           q_unix = addr->u.q_unix.data = g_new0(UnixSocketAddress, 1);
>>           q_unix->path = g_strdup(path);
>> +
>> +        if (fd) {
>> +            q_unix->sockfd = fd;
>> +        } else {
>> +            q_unix->sockfd = 0;
>
> 0 is a valid fd number; this risks accidentally closing stdin later on.
>   Please use -1 for unset, if you must store an fd number.  But given my
> comments on patch 1, I'm not sure that you need this addition.
Thanks for your comment, i just wonder what's the motivation of 
qemu_open(), seems it's more like an regular file consideration, is it?
can it be easily expended to socket file?
>

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Michal Privoznik]

On 01.06.2016 18:16, Wei Xu wrote:
> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>> From: Wei Xu <wexu@redhat.com>
>>>
>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>> create a unix socket for a chardev when managing VMs with libvirt,
>>> because qemu don't have sufficient permissions in this case, and
>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>> passing it to qemu.
>>
>> This sounds like a bug in libvirt, or selinux, or a mistaken
>> configuration
>> of the guest. It is entirely possible for QEMU to create a unix socket
>> - not
>> least because that is exactly what QEMU uses for its QMP monitor backend.
>> Looking at your example command line, I think the issue is simply that
>> you
>> should be putting the sockets in a different location. ie at
>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>> permission to
>> create sockets already.
> ah.. adjusting permission or file location can solve this problem, i'm
> guessing maybe this is a more security concern, the socket is used as a
> network interface for a vm, similar as the qcow image file, thus should
> prevent it to be arbitrarily accessed.
> 
> Michael, do you have any comment on this?

I haven't seen the patches. But in libvirt we allow users to create a
vhostuser interface and even specify where the socket should be placed:

    <interface type='vhostuser'>
      <mac address='52:54:00:ee:96:6c'/>
      <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
      <model type='virtio'/>
    </interface>

The following cmd line is generated by libvirt then:

-chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
-netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
-device
virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\

Now, if we accept only /var/run/openvwitch path in
/interface/source/@path (or whatever path to OVS is), we don't need this
and have users manually label the dir (unless already labeled). But
since we accept just any path in there, we should make sure that qemu is
then able to create the socket. One possible fix would be to allow qemu
create sockets just anywhere in the system. This, however, brings huge
security risks and it's not acceptable IMO. The other option would be
that libvirt would create the socket, and pass its FD to qemu (since
libvirt already is allowed to create sockets anywhere).

> 
>>
>> Alternatively you could enhance the SELinux policy to grant svirt_t the
>> permission to create sockets under /var/run/openvswitch too.

Nah, the point of using libvirt is that you don't have to configure
anything (or just bare minimum) and libvirt makes sure your domains have
all the permissions needed - that's why we relabel domain's disks when
starting it up.

BTW: Is there any reason why the patches have not been sent to the list?

>>
>>> I finished a RFC patch for Unix socket after a glance of the code,
>>> and not sure if this is right or there maybe other side-effects,
>>> please point me out.
>>>
>>> I tested it for both server and client mode 'PF_UNIX' socket with a VM
>>> running vhost-user.
>>>
>>> Old command line:
>>> -chardev socket,id=char0,path=/var/run/openvswitch/vhost-user1,server
>>>
>>> New command line:
>>> -chardev
>>> socket,id=char0,path=/var/run/openvswitch/vhost-user1,server,sockfd=$n
>>>
>>> because unix socket is bundled with a path, so it should be kept even
>>> with the
>>> 'fd' is indicated, this looks odd, any comments?
>>
>> Yes, this syntax doesn't really make sense. The passed in sockfd may not
>> even be a UNIX socket - it could be a TCP socket. As such, the 'sockfd'
>> option should be mutually exclusive with the 'path' and 'host' options.
>> ie you can only supply one out of 'sockfd', 'path', or 'host'.
> Currently i just add it for unix socket, and the connect/listen syscall
> must have a path name, an inet socket doesn't need this param at all,
> should it be supported also?
> 
>>
>>
>> FWIW, I think the ability to pass a pre-opened socket FD with the
>> -chardev socket backend is a useful enhancement, even though I don't
>> think you need this in order to fix the problem you have.

Isn't this the ideal goal? I mean, in theory (and far far future)
libvirt would open all the files for qemu and then pass all the FDs
rather than what we have now?

Michal

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Daniel P. Berrange]

On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> On 01.06.2016 18:16, Wei Xu wrote:
> > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> >>> From: Wei Xu <wexu@redhat.com>
> >>>
> >>> Recently I'm working on a fd passing issue, selinux forbids qemu to
> >>> create a unix socket for a chardev when managing VMs with libvirt,
> >>> because qemu don't have sufficient permissions in this case, and
> >>> proposal from libvirt team is opening the 'fd' in libvirt and merely
> >>> passing it to qemu.
> >>
> >> This sounds like a bug in libvirt, or selinux, or a mistaken
> >> configuration
> >> of the guest. It is entirely possible for QEMU to create a unix socket
> >> - not
> >> least because that is exactly what QEMU uses for its QMP monitor backend.
> >> Looking at your example command line, I think the issue is simply that
> >> you
> >> should be putting the sockets in a different location. ie at
> >> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >> permission to
> >> create sockets already.
> > ah.. adjusting permission or file location can solve this problem, i'm
> > guessing maybe this is a more security concern, the socket is used as a
> > network interface for a vm, similar as the qcow image file, thus should
> > prevent it to be arbitrarily accessed.
> > 
> > Michael, do you have any comment on this?
> 
> I haven't seen the patches. But in libvirt we allow users to create a
> vhostuser interface and even specify where the socket should be placed:
> 
>     <interface type='vhostuser'>
>       <mac address='52:54:00:ee:96:6c'/>
>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>       <model type='virtio'/>
>     </interface>
> 
> The following cmd line is generated by libvirt then:
> 
> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> -device
> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> 
> Now, if we accept only /var/run/openvwitch path in
> /interface/source/@path (or whatever path to OVS is), we don't need this
> and have users manually label the dir (unless already labeled). But
> since we accept just any path in there, we should make sure that qemu is
> then able to create the socket. One possible fix would be to allow qemu
> create sockets just anywhere in the system. This, however, brings huge
> security risks and it's not acceptable IMO. The other option would be
> that libvirt would create the socket, and pass its FD to qemu (since
> libvirt already is allowed to create sockets anywhere).

There are plenty of other places where we allow arbitrary paths in the
XML, but which have restrictions imposed by the security drivers. Not
least the <channel> devices which have the exact same scenario as this
network device, and require use of /var/lib/libvirt/qemu as the directory
for the sockets. We certainly do not want to allow QEMU to create sockets
anywhere.

I don't think we want to grant QEMU svirtt permission to create sockets
in the /var/run/openvswitch directory either really.IMHO, users of vhost
user should really be using /var/lib/libvirt/qemu, as is used for all
other UNIX sockets we create wrt other devices.


> >> Alternatively you could enhance the SELinux policy to grant svirt_t the
> >> permission to create sockets under /var/run/openvswitch too.
> 
> Nah, the point of using libvirt is that you don't have to configure
> anything (or just bare minimum) and libvirt makes sure your domains have
> all the permissions needed - that's why we relabel domain's disks when
> starting it up.

That's semi-true - we don't actually support arbitrary locations for
disks - there are plenty of locations that will not work, even if
libvirt labels the disk file, due to restrictions accessing the parent
directories by SELinux.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Michal Privoznik]

On 02.06.2016 10:29, Daniel P. Berrange wrote:
> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>> On 01.06.2016 18:16, Wei Xu wrote:
>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>
>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>>>> create a unix socket for a chardev when managing VMs with libvirt,
>>>>> because qemu don't have sufficient permissions in this case, and
>>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>>>> passing it to qemu.
>>>>
>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>> configuration
>>>> of the guest. It is entirely possible for QEMU to create a unix socket
>>>> - not
>>>> least because that is exactly what QEMU uses for its QMP monitor backend.
>>>> Looking at your example command line, I think the issue is simply that
>>>> you
>>>> should be putting the sockets in a different location. ie at
>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>> permission to
>>>> create sockets already.
>>> ah.. adjusting permission or file location can solve this problem, i'm
>>> guessing maybe this is a more security concern, the socket is used as a
>>> network interface for a vm, similar as the qcow image file, thus should
>>> prevent it to be arbitrarily accessed.
>>>
>>> Michael, do you have any comment on this?
>>
>> I haven't seen the patches. But in libvirt we allow users to create a
>> vhostuser interface and even specify where the socket should be placed:
>>
>>     <interface type='vhostuser'>
>>       <mac address='52:54:00:ee:96:6c'/>
>>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>       <model type='virtio'/>
>>     </interface>
>>
>> The following cmd line is generated by libvirt then:
>>
>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>> -device
>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>
>> Now, if we accept only /var/run/openvwitch path in
>> /interface/source/@path (or whatever path to OVS is), we don't need this
>> and have users manually label the dir (unless already labeled). But
>> since we accept just any path in there, we should make sure that qemu is
>> then able to create the socket. One possible fix would be to allow qemu
>> create sockets just anywhere in the system. This, however, brings huge
>> security risks and it's not acceptable IMO. The other option would be
>> that libvirt would create the socket, and pass its FD to qemu (since
>> libvirt already is allowed to create sockets anywhere).
> 
> There are plenty of other places where we allow arbitrary paths in the
> XML, but which have restrictions imposed by the security drivers. Not
> least the <channel> devices which have the exact same scenario as this
> network device, and require use of /var/lib/libvirt/qemu as the directory
> for the sockets. We certainly do not want to allow QEMU to create sockets
> anywhere.
> 
> I don't think we want to grant QEMU svirtt permission to create sockets
> in the /var/run/openvswitch directory either really.IMHO, users of vhost
> user should really be using /var/lib/libvirt/qemu, as is used for all
> other UNIX sockets we create wrt other devices.

Okay. I can live with that; but in that case we should document it
somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
accessible and for the rest we do our best but maybe require sys admin
intervention (e.g. to label the whole tree for a non-standard location).

Michal

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

On 2016年06月02日 19:38, Michal Privoznik wrote:
> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>
>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
>>>>>> because qemu don't have sufficient permissions in this case, and
>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>>>>> passing it to qemu.
>>>>>
>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>> configuration
>>>>> of the guest. It is entirely possible for QEMU to create a unix socket
>>>>> - not
>>>>> least because that is exactly what QEMU uses for its QMP monitor backend.
>>>>> Looking at your example command line, I think the issue is simply that
>>>>> you
>>>>> should be putting the sockets in a different location. ie at
>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>> permission to
>>>>> create sockets already.
>>>> ah.. adjusting permission or file location can solve this problem, i'm
>>>> guessing maybe this is a more security concern, the socket is used as a
>>>> network interface for a vm, similar as the qcow image file, thus should
>>>> prevent it to be arbitrarily accessed.
>>>>
>>>> Michael, do you have any comment on this?
>>>
>>> I haven't seen the patches. But in libvirt we allow users to create a
>>> vhostuser interface and even specify where the socket should be placed:
>>>
>>>      <interface type='vhostuser'>
>>>        <mac address='52:54:00:ee:96:6c'/>
>>>        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>        <model type='virtio'/>
>>>      </interface>
>>>
>>> The following cmd line is generated by libvirt then:
>>>
>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>> -device
>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>
>>> Now, if we accept only /var/run/openvwitch path in
>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>> and have users manually label the dir (unless already labeled). But
>>> since we accept just any path in there, we should make sure that qemu is
>>> then able to create the socket. One possible fix would be to allow qemu
>>> create sockets just anywhere in the system. This, however, brings huge
>>> security risks and it's not acceptable IMO. The other option would be
>>> that libvirt would create the socket, and pass its FD to qemu (since
>>> libvirt already is allowed to create sockets anywhere).
>>
>> There are plenty of other places where we allow arbitrary paths in the
>> XML, but which have restrictions imposed by the security drivers. Not
>> least the <channel> devices which have the exact same scenario as this
>> network device, and require use of /var/lib/libvirt/qemu as the directory
>> for the sockets. We certainly do not want to allow QEMU to create sockets
>> anywhere.
AFAIK, Vhost user is an interface for third party implementations, and 
ovs/dpdk is one of the most popular choices, if it limits the socket 
location of a fixed and unprivileged directory to qemu, actually this 
should be the default and only one option, this maybe also a security 
consideration, so we'll have no other choice but ask sys admin to 
manipulate the permission, looks accepting a safe passed in 'fd' from 
libvirt is more rigorous and convenient, i'm not sure if this is a 
typical or a corner scenario.

Daniel,
How do you think about this with a general purpose? does qemu need such 
a feature?
>>
>> I don't think we want to grant QEMU svirtt permission to create sockets
>> in the /var/run/openvswitch directory either really.IMHO, users of vhost
>> user should really be using /var/lib/libvirt/qemu, as is used for all
>> other UNIX sockets we create wrt other devices.
>
> Okay. I can live with that; but in that case we should document it
> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> accessible and for the rest we do our best but maybe require sys admin
> intervention (e.g. to label the whole tree for a non-standard location).
>
> Michal
>

[Qemu-devel] Channel paths (was: Re: [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket)

[Sascha Silbe]

Dear Daniel,

"Daniel P. Berrange" <berrange@redhat.com> writes:

> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>> On 01.06.2016 18:16, Wei Xu wrote:
>> > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> There are plenty of other places where we allow arbitrary paths in the
> XML, but which have restrictions imposed by the security drivers. Not
> least the <channel> devices which have the exact same scenario as this
> network device, and require use of /var/lib/libvirt/qemu as the directory
> for the sockets. We certainly do not want to allow QEMU to create sockets
> anywhere.

Umm, how exactly is an application supposed to use (i.e. open) the
channel devices defined in XML?

Previously I deliberately left out the path in the XML to let libvirt
choose the path and extracted it from the XML after defining the
domain. This ensured qemu could access the path, plus it was the
responsibility of libvirt to clean it up once the domain was
undefined. Easy and simple.

Since 71408079 [qemu: Don't bother user with libvirt-internal paths],
the path chosen by libvirt isn't included in the domain XML anymore. So
now I need to choose the path inside the application. The only safe way
to do that is by using something in an application-managed namespace
(e.g. /var/lib/myapp/foo or /tmp/myapp/foo); I certainly wouldn't want
to second guess what paths would be safe inside the libvirt namespace
(/var/lib/libvirt/qemu). Except now I hear that anything outside
/var/lib/libvirt/qemu is not guaranteed to work due to e.g. the SELinux
policy configured by libvirt...

Sascha
-- 
Softwareentwicklung Sascha Silbe, Niederhofenstraße 5/1, 71229 Leonberg
https://se-silbe.de/
USt-IdNr. DE281696641

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Daniel P. Berrange]

On Fri, Jun 03, 2016 at 02:07:00AM +0800, Wei Xu wrote:
> On 2016年06月02日 19:38, Michal Privoznik wrote:
> > On 02.06.2016 10:29, Daniel P. Berrange wrote:
> > > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> > > > On 01.06.2016 18:16, Wei Xu wrote:
> > > > > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> > > > > > On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> > > > > > > From: Wei Xu <wexu@redhat.com>
> > > > > > > 
> > > > > > > Recently I'm working on a fd passing issue, selinux forbids qemu to
> > > > > > > create a unix socket for a chardev when managing VMs with libvirt,
> > > > > > > because qemu don't have sufficient permissions in this case, and
> > > > > > > proposal from libvirt team is opening the 'fd' in libvirt and merely
> > > > > > > passing it to qemu.
> > > > > > 
> > > > > > This sounds like a bug in libvirt, or selinux, or a mistaken
> > > > > > configuration
> > > > > > of the guest. It is entirely possible for QEMU to create a unix socket
> > > > > > - not
> > > > > > least because that is exactly what QEMU uses for its QMP monitor backend.
> > > > > > Looking at your example command line, I think the issue is simply that
> > > > > > you
> > > > > > should be putting the sockets in a different location. ie at
> > > > > > /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> > > > > > permission to
> > > > > > create sockets already.
> > > > > ah.. adjusting permission or file location can solve this problem, i'm
> > > > > guessing maybe this is a more security concern, the socket is used as a
> > > > > network interface for a vm, similar as the qcow image file, thus should
> > > > > prevent it to be arbitrarily accessed.
> > > > > 
> > > > > Michael, do you have any comment on this?
> > > > 
> > > > I haven't seen the patches. But in libvirt we allow users to create a
> > > > vhostuser interface and even specify where the socket should be placed:
> > > > 
> > > >      <interface type='vhostuser'>
> > > >        <mac address='52:54:00:ee:96:6c'/>
> > > >        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> > > >        <model type='virtio'/>
> > > >      </interface>
> > > > 
> > > > The following cmd line is generated by libvirt then:
> > > > 
> > > > -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> > > > -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> > > > -device
> > > > virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> > > > 
> > > > Now, if we accept only /var/run/openvwitch path in
> > > > /interface/source/@path (or whatever path to OVS is), we don't need this
> > > > and have users manually label the dir (unless already labeled). But
> > > > since we accept just any path in there, we should make sure that qemu is
> > > > then able to create the socket. One possible fix would be to allow qemu
> > > > create sockets just anywhere in the system. This, however, brings huge
> > > > security risks and it's not acceptable IMO. The other option would be
> > > > that libvirt would create the socket, and pass its FD to qemu (since
> > > > libvirt already is allowed to create sockets anywhere).
> > > 
> > > There are plenty of other places where we allow arbitrary paths in the
> > > XML, but which have restrictions imposed by the security drivers. Not
> > > least the <channel> devices which have the exact same scenario as this
> > > network device, and require use of /var/lib/libvirt/qemu as the directory
> > > for the sockets. We certainly do not want to allow QEMU to create sockets
> > > anywhere.
> AFAIK, Vhost user is an interface for third party implementations, and
> ovs/dpdk is one of the most popular choices, if it limits the socket
> location of a fixed and unprivileged directory to qemu, actually this should
> be the default and only one option, this maybe also a security
> consideration, so we'll have no other choice but ask sys admin to manipulate
> the permission, looks accepting a safe passed in 'fd' from libvirt is more
> rigorous and convenient, i'm not sure if this is a typical or a corner
> scenario.

Libvirt doesn't limit the socket locations because the restriction I describe
is related to the SELinux policy as it currently exists. It is possible to
further extend the SELinux policy to allow other locations to be used if
it makes sense for particular use cases, hence it would be wrong for libvirt
to enforce the location. There should rarely be any need to do this though,
hence why we suggest that apps use /var/lib/libvirt/qemu by default, unless
they have specific requirements and wish to extend the SELinux policy

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Channel paths (was: Re: [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket)

[Daniel P. Berrange]

On Thu, Jun 02, 2016 at 09:27:45PM +0200, Sascha Silbe wrote:
> Dear Daniel,
> 
> "Daniel P. Berrange" <berrange@redhat.com> writes:
> 
> > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >> On 01.06.2016 18:16, Wei Xu wrote:
> >> > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> > There are plenty of other places where we allow arbitrary paths in the
> > XML, but which have restrictions imposed by the security drivers. Not
> > least the <channel> devices which have the exact same scenario as this
> > network device, and require use of /var/lib/libvirt/qemu as the directory
> > for the sockets. We certainly do not want to allow QEMU to create sockets
> > anywhere.
> 
> Umm, how exactly is an application supposed to use (i.e. open) the
> channel devices defined in XML?
> 
> Previously I deliberately left out the path in the XML to let libvirt
> choose the path and extracted it from the XML after defining the
> domain. This ensured qemu could access the path, plus it was the
> responsibility of libvirt to clean it up once the domain was
> undefined. Easy and simple.
> 
> Since 71408079 [qemu: Don't bother user with libvirt-internal paths],
> the path chosen by libvirt isn't included in the domain XML anymore. So
> now I need to choose the path inside the application. The only safe way
> to do that is by using something in an application-managed namespace
> (e.g. /var/lib/myapp/foo or /tmp/myapp/foo); I certainly wouldn't want
> to second guess what paths would be safe inside the libvirt namespace
> (/var/lib/libvirt/qemu). Except now I hear that anything outside
> /var/lib/libvirt/qemu is not guaranteed to work due to e.g. the SELinux
> policy configured by libvirt...

IIUC that change 71408079 ought to only apply to the persistent XML
configuration on disk. When the guest is running the live XML ought
to still report the path libvirt chose, so you should still be able
to see it while running. If that isn't the case then please raise a
bug, because we certainly expect apps to be able to discover the
path libvirt picked for exactly the reason you describe


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] Channel paths

[Sascha Silbe]

Dear Daniel,

[CC += libvir-list, didn't notice before this thread is on qemu-devel
 only]

"Daniel P. Berrange" <berrange@redhat.com> writes:

> On Thu, Jun 02, 2016 at 09:27:45PM +0200, Sascha Silbe wrote:
>> Since 71408079 [qemu: Don't bother user with libvirt-internal paths],
>> the path chosen by libvirt isn't included in the domain XML anymore. So
>> now I need to choose the path inside the application. The only safe way
>> to do that is by using something in an application-managed namespace
>> (e.g. /var/lib/myapp/foo or /tmp/myapp/foo); I certainly wouldn't want
>> to second guess what paths would be safe inside the libvirt namespace
>> (/var/lib/libvirt/qemu). Except now I hear that anything outside
>> /var/lib/libvirt/qemu is not guaranteed to work due to e.g. the SELinux
>> policy configured by libvirt...
>
> IIUC that change 71408079 ought to only apply to the persistent XML
> configuration on disk. When the guest is running the live XML ought
> to still report the path libvirt chose, so you should still be able
> to see it while running. If that isn't the case then please raise a
> bug, because we certainly expect apps to be able to discover the
> path libvirt picked for exactly the reason you describe

OK, this makes a lot more sense now (persistent vs. live). Just tried it
with current libvirt master and it works, thanks.

It even worked with 71408079 itself. I'm pretty sure things were broken
some time in between, but a few quick probes didn't turn up a faulty
version and I'm not curious enough to do a linear search. Cannot
completely rule out other effects having played a role, either.

Sascha
-- 
Softwareentwicklung Sascha Silbe, Niederhofenstraße 5/1, 71229 Leonberg
https://se-silbe.de/
USt-IdNr. DE281696641

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Amnon Ilan]

----- Original Message -----
> From: "Michal Privoznik" <mprivozn@redhat.com>
> To: "Daniel P. Berrange" <berrange@redhat.com>
> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>, jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
> armbru@redhat.com
> Sent: Thursday, June 2, 2016 2:38:53 PM
> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket
> 
> On 02.06.2016 10:29, Daniel P. Berrange wrote:
> > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >> On 01.06.2016 18:16, Wei Xu wrote:
> >>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> >>>>> From: Wei Xu <wexu@redhat.com>
> >>>>>
> >>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
> >>>>> create a unix socket for a chardev when managing VMs with libvirt,
> >>>>> because qemu don't have sufficient permissions in this case, and
> >>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
> >>>>> passing it to qemu.
> >>>>
> >>>> This sounds like a bug in libvirt, or selinux, or a mistaken
> >>>> configuration
> >>>> of the guest. It is entirely possible for QEMU to create a unix socket
> >>>> - not
> >>>> least because that is exactly what QEMU uses for its QMP monitor
> >>>> backend.
> >>>> Looking at your example command line, I think the issue is simply that
> >>>> you
> >>>> should be putting the sockets in a different location. ie at
> >>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >>>> permission to
> >>>> create sockets already.
> >>> ah.. adjusting permission or file location can solve this problem, i'm
> >>> guessing maybe this is a more security concern, the socket is used as a
> >>> network interface for a vm, similar as the qcow image file, thus should
> >>> prevent it to be arbitrarily accessed.
> >>>
> >>> Michael, do you have any comment on this?
> >>
> >> I haven't seen the patches. But in libvirt we allow users to create a
> >> vhostuser interface and even specify where the socket should be placed:
> >>
> >>     <interface type='vhostuser'>
> >>       <mac address='52:54:00:ee:96:6c'/>
> >>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> >>       <model type='virtio'/>
> >>     </interface>
> >>
> >> The following cmd line is generated by libvirt then:
> >>
> >> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> >> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> >> -device
> >> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> >>
> >> Now, if we accept only /var/run/openvwitch path in
> >> /interface/source/@path (or whatever path to OVS is), we don't need this
> >> and have users manually label the dir (unless already labeled). But
> >> since we accept just any path in there, we should make sure that qemu is
> >> then able to create the socket. One possible fix would be to allow qemu
> >> create sockets just anywhere in the system. This, however, brings huge
> >> security risks and it's not acceptable IMO. The other option would be
> >> that libvirt would create the socket, and pass its FD to qemu (since
> >> libvirt already is allowed to create sockets anywhere).
> > 
> > There are plenty of other places where we allow arbitrary paths in the
> > XML, but which have restrictions imposed by the security drivers. Not
> > least the <channel> devices which have the exact same scenario as this
> > network device, and require use of /var/lib/libvirt/qemu as the directory
> > for the sockets. We certainly do not want to allow QEMU to create sockets
> > anywhere.
> > 
> > I don't think we want to grant QEMU svirtt permission to create sockets
> > in the /var/run/openvswitch directory either really.IMHO, users of vhost
> > user should really be using /var/lib/libvirt/qemu, as is used for all
> > other UNIX sockets we create wrt other devices.
> 
> Okay. I can live with that; but in that case we should document it
> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> accessible and for the rest we do our best but maybe require sys admin
> intervention (e.g. to label the whole tree for a non-standard location).

Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
Flavio, do you know?
If not, we are good as it is today with /var/lib/libvirt/qemu, right?

Thanks,
Amnon

> 
> Michal
> 
> 

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Amnon Ilan]

----- Original Message -----
> From: "Aaron Conole" <aconole@redhat.com>
> To: "Flavio Leitner" <fbl@redhat.com>
> Cc: "Amnon Ilan" <ailan@redhat.com>, "Michal Privoznik" <mprivozn@redhat.com>, "Daniel P. Berrange"
> <berrange@redhat.com>, qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>, jasowang@redhat.com, "Wei Xu"
> <wexu@redhat.com>, armbru@redhat.com
> Sent: Thursday, June 9, 2016 12:48:57 AM
> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket
> 
> Flavio Leitner <fbl@redhat.com> writes:
> 
> > Adding Aaron who is fixing exactly that on the OVS side.
> >
> > Aaron, please see the last question in the bottom of this email.
> >
> > On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
> >> 
> >> 
> >> ----- Original Message -----
> >> > From: "Michal Privoznik" <mprivozn@redhat.com>
> >> > To: "Daniel P. Berrange" <berrange@redhat.com>
> >> > Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
> >> > jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
> >> > armbru@redhat.com
> >> > Sent: Thursday, June 2, 2016 2:38:53 PM
> >> > Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
> >> > 'fd' open from outside for unix socket
> >> > 
> >> > On 02.06.2016 10:29, Daniel P. Berrange wrote:
> >> > > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >> > >> On 01.06.2016 18:16, Wei Xu wrote:
> >> > >>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >> > >>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> >> > >>>>> From: Wei Xu <wexu@redhat.com>
> >> > >>>>>
> >> > >>>>> Recently I'm working on a fd passing issue, selinux forbids qemu
> >> > >>>>> to
> >> > >>>>> create a unix socket for a chardev when managing VMs with libvirt,
> >> > >>>>> because qemu don't have sufficient permissions in this case, and
> >> > >>>>> proposal from libvirt team is opening the 'fd' in libvirt and
> >> > >>>>> merely
> >> > >>>>> passing it to qemu.
> >> > >>>>
> >> > >>>> This sounds like a bug in libvirt, or selinux, or a mistaken
> >> > >>>> configuration
> >> > >>>> of the guest. It is entirely possible for QEMU to create a unix
> >> > >>>> socket
> >> > >>>> - not
> >> > >>>> least because that is exactly what QEMU uses for its QMP monitor
> >> > >>>> backend.
> >> > >>>> Looking at your example command line, I think the issue is simply
> >> > >>>> that
> >> > >>>> you
> >> > >>>> should be putting the sockets in a different location. ie at
> >> > >>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >> > >>>> permission to
> >> > >>>> create sockets already.
> >> > >>> ah.. adjusting permission or file location can solve this problem,
> >> > >>> i'm
> >> > >>> guessing maybe this is a more security concern, the socket is used
> >> > >>> as a
> >> > >>> network interface for a vm, similar as the qcow image file, thus
> >> > >>> should
> >> > >>> prevent it to be arbitrarily accessed.
> >> > >>>
> >> > >>> Michael, do you have any comment on this?
> >> > >>
> >> > >> I haven't seen the patches. But in libvirt we allow users to create a
> >> > >> vhostuser interface and even specify where the socket should be
> >> > >> placed:
> >> > >>
> >> > >>     <interface type='vhostuser'>
> >> > >>       <mac address='52:54:00:ee:96:6c'/>
> >> > >>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> >> > >>       <model type='virtio'/>
> >> > >>     </interface>
> >> > >>
> >> > >> The following cmd line is generated by libvirt then:
> >> > >>
> >> > >> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> >> > >> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> >> > >> -device
> >> > >> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> >> > >>
> >> > >> Now, if we accept only /var/run/openvwitch path in
> >> > >> /interface/source/@path (or whatever path to OVS is), we don't need
> >> > >> this
> >> > >> and have users manually label the dir (unless already labeled). But
> >> > >> since we accept just any path in there, we should make sure that qemu
> >> > >> is
> >> > >> then able to create the socket. One possible fix would be to allow
> >> > >> qemu
> >> > >> create sockets just anywhere in the system. This, however, brings
> >> > >> huge
> >> > >> security risks and it's not acceptable IMO. The other option would be
> >> > >> that libvirt would create the socket, and pass its FD to qemu (since
> >> > >> libvirt already is allowed to create sockets anywhere).
> >> > > 
> >> > > There are plenty of other places where we allow arbitrary paths in the
> >> > > XML, but which have restrictions imposed by the security drivers. Not
> >> > > least the <channel> devices which have the exact same scenario as this
> >> > > network device, and require use of /var/lib/libvirt/qemu as the
> >> > > directory
> >> > > for the sockets. We certainly do not want to allow QEMU to create
> >> > > sockets
> >> > > anywhere.
> >> > > 
> >> > > I don't think we want to grant QEMU svirtt permission to create
> >> > > sockets
> >> > > in the /var/run/openvswitch directory either really.IMHO, users of
> >> > > vhost
> >> > > user should really be using /var/lib/libvirt/qemu, as is used for all
> >> > > other UNIX sockets we create wrt other devices.
> >> > 
> >> > Okay. I can live with that; but in that case we should document it
> >> > somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> >> > accessible and for the rest we do our best but maybe require sys admin
> >> > intervention (e.g. to label the whole tree for a non-standard location).
> >> 
> >> Does OVS has some limit for it's sockets to be only in
> >> /var/run/openvswitch ?
> 
> As of a recent commit, it can only be in /var/run/openvswitch or a
> subdirectory therein (found in the openvswitch database).
> 
> >> Flavio, do you know?
> >> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
> 
> Probably not.  There are other issues as well.  From a DAC perspective
> (so forgetting selinux at the moment), qemu and ovs run as different
> users.  This will mean that when ovs creates the unix domain socket
> file, qemu will not be allowed to actually open it properly.  I have a
> commit I'm trying to get upstream for that particular issue (see
> bz:1281911 and upstream list discussion:
>   http://openvswitch.org/pipermail/dev/2016-May/071453.html
>   and
>   http://openvswitch.org/pipermail/dev/2016-June/071978.html)
> 
> Ansis is (I think) attacking this from the selinux/MAC side.  It would
> be great to hear from users, libvirt folks, or anyone else in that
> thread to help push it to a conclusion one way or another (even if the
> approach that I've proposed is crap and wrong - then say it there :).

Based on Aaron's comments, I think we should let libvirt open the socket and pass 
the file descriptor to Qemu.

Thanks,
Amnon

> 
> Hope this helps,
> -Aaron
> 

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Daniel P. Berrange]

On Wed, Jun 08, 2016 at 05:48:57PM -0400, Aaron Conole wrote:
> Flavio Leitner <fbl@redhat.com> writes:
> 
> > Adding Aaron who is fixing exactly that on the OVS side.
> >
> > Aaron, please see the last question in the bottom of this email.
> >
> > On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
> >> 
> >> 
> >> ----- Original Message -----
> >> > From: "Michal Privoznik" <mprivozn@redhat.com>
> >> > To: "Daniel P. Berrange" <berrange@redhat.com>
> >> > Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
> >> > jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
> >> > armbru@redhat.com
> >> > Sent: Thursday, June 2, 2016 2:38:53 PM
> >> > Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
> >> > 'fd' open from outside for unix socket
> >> > 
> >> > On 02.06.2016 10:29, Daniel P. Berrange wrote:
> >> > > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >> > >> On 01.06.2016 18:16, Wei Xu wrote:
> >> > >>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >> > >>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> >> > >>>>> From: Wei Xu <wexu@redhat.com>
> >> > >>>>>
> >> > >>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
> >> > >>>>> create a unix socket for a chardev when managing VMs with libvirt,
> >> > >>>>> because qemu don't have sufficient permissions in this case, and
> >> > >>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
> >> > >>>>> passing it to qemu.
> >> > >>>>
> >> > >>>> This sounds like a bug in libvirt, or selinux, or a mistaken
> >> > >>>> configuration
> >> > >>>> of the guest. It is entirely possible for QEMU to create a unix socket
> >> > >>>> - not
> >> > >>>> least because that is exactly what QEMU uses for its QMP monitor
> >> > >>>> backend.
> >> > >>>> Looking at your example command line, I think the issue is simply that
> >> > >>>> you
> >> > >>>> should be putting the sockets in a different location. ie at
> >> > >>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >> > >>>> permission to
> >> > >>>> create sockets already.
> >> > >>> ah.. adjusting permission or file location can solve this problem, i'm
> >> > >>> guessing maybe this is a more security concern, the socket is used as a
> >> > >>> network interface for a vm, similar as the qcow image file, thus should
> >> > >>> prevent it to be arbitrarily accessed.
> >> > >>>
> >> > >>> Michael, do you have any comment on this?
> >> > >>
> >> > >> I haven't seen the patches. But in libvirt we allow users to create a
> >> > >> vhostuser interface and even specify where the socket should be placed:
> >> > >>
> >> > >>     <interface type='vhostuser'>
> >> > >>       <mac address='52:54:00:ee:96:6c'/>
> >> > >>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> >> > >>       <model type='virtio'/>
> >> > >>     </interface>
> >> > >>
> >> > >> The following cmd line is generated by libvirt then:
> >> > >>
> >> > >> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> >> > >> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> >> > >> -device
> >> > >> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> >> > >>
> >> > >> Now, if we accept only /var/run/openvwitch path in
> >> > >> /interface/source/@path (or whatever path to OVS is), we don't need this
> >> > >> and have users manually label the dir (unless already labeled). But
> >> > >> since we accept just any path in there, we should make sure that qemu is
> >> > >> then able to create the socket. One possible fix would be to allow qemu
> >> > >> create sockets just anywhere in the system. This, however, brings huge
> >> > >> security risks and it's not acceptable IMO. The other option would be
> >> > >> that libvirt would create the socket, and pass its FD to qemu (since
> >> > >> libvirt already is allowed to create sockets anywhere).
> >> > > 
> >> > > There are plenty of other places where we allow arbitrary paths in the
> >> > > XML, but which have restrictions imposed by the security drivers. Not
> >> > > least the <channel> devices which have the exact same scenario as this
> >> > > network device, and require use of /var/lib/libvirt/qemu as the directory
> >> > > for the sockets. We certainly do not want to allow QEMU to create sockets
> >> > > anywhere.
> >> > > 
> >> > > I don't think we want to grant QEMU svirtt permission to create sockets
> >> > > in the /var/run/openvswitch directory either really.IMHO, users of vhost
> >> > > user should really be using /var/lib/libvirt/qemu, as is used for all
> >> > > other UNIX sockets we create wrt other devices.
> >> > 
> >> > Okay. I can live with that; but in that case we should document it
> >> > somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> >> > accessible and for the rest we do our best but maybe require sys admin
> >> > intervention (e.g. to label the whole tree for a non-standard location).
> >> 
> >> Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
> 
> As of a recent commit, it can only be in /var/run/openvswitch or a
> subdirectory therein (found in the openvswitch database).
> 
> >> Flavio, do you know?
> >> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
> 
> Probably not.  There are other issues as well.  From a DAC perspective
> (so forgetting selinux at the moment), qemu and ovs run as different
> users.  This will mean that when ovs creates the unix domain socket
> file, qemu will not be allowed to actually open it properly.  I have a
> commit I'm trying to get upstream for that particular issue (see
> bz:1281911 and upstream list discussion:
>   http://openvswitch.org/pipermail/dev/2016-May/071453.html
>   and
>   http://openvswitch.org/pipermail/dev/2016-June/071978.html)
> 
> Ansis is (I think) attacking this from the selinux/MAC side.  It would
> be great to hear from users, libvirt folks, or anyone else in that
> thread to help push it to a conclusion one way or another (even if the
> approach that I've proposed is crap and wrong - then say it there :).

NB, that libvirt is designed from a general POV, rather than tailored
to any single application. For all resources that are file based, libvirt
should be changing ownership and setting SELinux labels to grant QEMU
access. This is something we need to fix in libvirts vhostuser handling
regardless of what openvswitch is doing.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Michal Privoznik]

On 09.06.2016 11:16, Daniel P. Berrange wrote:
> On Wed, Jun 08, 2016 at 05:48:57PM -0400, Aaron Conole wrote:
>> Flavio Leitner <fbl@redhat.com> writes:
>>
>>> Adding Aaron who is fixing exactly that on the OVS side.
>>>
>>> Aaron, please see the last question in the bottom of this email.
>>>
>>> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Michal Privoznik" <mprivozn@redhat.com>
>>>>> To: "Daniel P. Berrange" <berrange@redhat.com>
>>>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>>>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>>>>> armbru@redhat.com
>>>>> Sent: Thursday, June 2, 2016 2:38:53 PM
>>>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>>>>> 'fd' open from outside for unix socket
>>>>>
>>>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>>>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>>>>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>>>>>
>>>>>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>>>>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
>>>>>>>>>> because qemu don't have sufficient permissions in this case, and
>>>>>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>>>>>>>>> passing it to qemu.
>>>>>>>>>
>>>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>>>>>> configuration
>>>>>>>>> of the guest. It is entirely possible for QEMU to create a unix socket
>>>>>>>>> - not
>>>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
>>>>>>>>> backend.
>>>>>>>>> Looking at your example command line, I think the issue is simply that
>>>>>>>>> you
>>>>>>>>> should be putting the sockets in a different location. ie at
>>>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>>>>>> permission to
>>>>>>>>> create sockets already.
>>>>>>>> ah.. adjusting permission or file location can solve this problem, i'm
>>>>>>>> guessing maybe this is a more security concern, the socket is used as a
>>>>>>>> network interface for a vm, similar as the qcow image file, thus should
>>>>>>>> prevent it to be arbitrarily accessed.
>>>>>>>>
>>>>>>>> Michael, do you have any comment on this?
>>>>>>>
>>>>>>> I haven't seen the patches. But in libvirt we allow users to create a
>>>>>>> vhostuser interface and even specify where the socket should be placed:
>>>>>>>
>>>>>>>     <interface type='vhostuser'>
>>>>>>>       <mac address='52:54:00:ee:96:6c'/>
>>>>>>>       <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>>>>>       <model type='virtio'/>
>>>>>>>     </interface>
>>>>>>>
>>>>>>> The following cmd line is generated by libvirt then:
>>>>>>>
>>>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>>>>>> -device
>>>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>>>>>
>>>>>>> Now, if we accept only /var/run/openvwitch path in
>>>>>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>>>>>> and have users manually label the dir (unless already labeled). But
>>>>>>> since we accept just any path in there, we should make sure that qemu is
>>>>>>> then able to create the socket. One possible fix would be to allow qemu
>>>>>>> create sockets just anywhere in the system. This, however, brings huge
>>>>>>> security risks and it's not acceptable IMO. The other option would be
>>>>>>> that libvirt would create the socket, and pass its FD to qemu (since
>>>>>>> libvirt already is allowed to create sockets anywhere).
>>>>>>
>>>>>> There are plenty of other places where we allow arbitrary paths in the
>>>>>> XML, but which have restrictions imposed by the security drivers. Not
>>>>>> least the <channel> devices which have the exact same scenario as this
>>>>>> network device, and require use of /var/lib/libvirt/qemu as the directory
>>>>>> for the sockets. We certainly do not want to allow QEMU to create sockets
>>>>>> anywhere.
>>>>>>
>>>>>> I don't think we want to grant QEMU svirtt permission to create sockets
>>>>>> in the /var/run/openvswitch directory either really.IMHO, users of vhost
>>>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
>>>>>> other UNIX sockets we create wrt other devices.
>>>>>
>>>>> Okay. I can live with that; but in that case we should document it
>>>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>>>>> accessible and for the rest we do our best but maybe require sys admin
>>>>> intervention (e.g. to label the whole tree for a non-standard location).
>>>>
>>>> Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
>>
>> As of a recent commit, it can only be in /var/run/openvswitch or a
>> subdirectory therein (found in the openvswitch database).

Well, this changes game rules for libvirt. The documentation I've
suggested above won't any good. Therefore I think we need to be able to
have libvirt opening the socket and then just pass the FD to qemu.

Michal

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

On 2016年06月09日 05:48, Aaron Conole wrote:
> Flavio Leitner <fbl@redhat.com> writes:
>
>> Adding Aaron who is fixing exactly that on the OVS side.
>>
>> Aaron, please see the last question in the bottom of this email.
>>
>> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Michal Privoznik" <mprivozn@redhat.com>
>>>> To: "Daniel P. Berrange" <berrange@redhat.com>
>>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>>>> armbru@redhat.com
>>>> Sent: Thursday, June 2, 2016 2:38:53 PM
>>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>>>> 'fd' open from outside for unix socket
>>>>
>>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>>>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>>>>
>>>>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>>>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
>>>>>>>>> because qemu don't have sufficient permissions in this case, and
>>>>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>>>>>>>> passing it to qemu.
>>>>>>>>
>>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>>>>> configuration
>>>>>>>> of the guest. It is entirely possible for QEMU to create a unix socket
>>>>>>>> - not
>>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
>>>>>>>> backend.
>>>>>>>> Looking at your example command line, I think the issue is simply that
>>>>>>>> you
>>>>>>>> should be putting the sockets in a different location. ie at
>>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>>>>> permission to
>>>>>>>> create sockets already.
>>>>>>> ah.. adjusting permission or file location can solve this problem, i'm
>>>>>>> guessing maybe this is a more security concern, the socket is used as a
>>>>>>> network interface for a vm, similar as the qcow image file, thus should
>>>>>>> prevent it to be arbitrarily accessed.
>>>>>>>
>>>>>>> Michael, do you have any comment on this?
>>>>>>
>>>>>> I haven't seen the patches. But in libvirt we allow users to create a
>>>>>> vhostuser interface and even specify where the socket should be placed:
>>>>>>
>>>>>>      <interface type='vhostuser'>
>>>>>>        <mac address='52:54:00:ee:96:6c'/>
>>>>>>        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>>>>        <model type='virtio'/>
>>>>>>      </interface>
>>>>>>
>>>>>> The following cmd line is generated by libvirt then:
>>>>>>
>>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>>>>> -device
>>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>>>>
>>>>>> Now, if we accept only /var/run/openvwitch path in
>>>>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>>>>> and have users manually label the dir (unless already labeled). But
>>>>>> since we accept just any path in there, we should make sure that qemu is
>>>>>> then able to create the socket. One possible fix would be to allow qemu
>>>>>> create sockets just anywhere in the system. This, however, brings huge
>>>>>> security risks and it's not acceptable IMO. The other option would be
>>>>>> that libvirt would create the socket, and pass its FD to qemu (since
>>>>>> libvirt already is allowed to create sockets anywhere).
>>>>>
>>>>> There are plenty of other places where we allow arbitrary paths in the
>>>>> XML, but which have restrictions imposed by the security drivers. Not
>>>>> least the <channel> devices which have the exact same scenario as this
>>>>> network device, and require use of /var/lib/libvirt/qemu as the directory
>>>>> for the sockets. We certainly do not want to allow QEMU to create sockets
>>>>> anywhere.
>>>>>
>>>>> I don't think we want to grant QEMU svirtt permission to create sockets
>>>>> in the /var/run/openvswitch directory either really.IMHO, users of vhost
>>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
>>>>> other UNIX sockets we create wrt other devices.
>>>>
>>>> Okay. I can live with that; but in that case we should document it
>>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>>>> accessible and for the rest we do our best but maybe require sys admin
>>>> intervention (e.g. to label the whole tree for a non-standard location).
>>>
>>> Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
>
> As of a recent commit, it can only be in /var/run/openvswitch or a
> subdirectory therein (found in the openvswitch database).
Aaron, thanks for your reply.

Just a question about the usage of openvswitch, in this user case when 
launching a vhostuser/dpdk via libvirt, qemu works as server mode for 
socket under /var/run/openvswitch, but per my previous test, ovs/dpdk 
always works as server mode, which means ovs will creates the socket and 
listening for connection, thus qemu works as client mode, does ovs/dpdk 
support working in client mode? which means it's qemu's duty to create 
the socket? and ovs will connect to it on demanding?

>
>>> Flavio, do you know?
>>> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
>
> Probably not.  There are other issues as well.  From a DAC perspective
> (so forgetting selinux at the moment), qemu and ovs run as different
> users.  This will mean that when ovs creates the unix domain socket
> file, qemu will not be allowed to actually open it properly.  I have a
> commit I'm trying to get upstream for that particular issue (see
> bz:1281911 and upstream list discussion:
>    http://openvswitch.org/pipermail/dev/2016-May/071453.html
>    and
>    http://openvswitch.org/pipermail/dev/2016-June/071978.html)
>
> Ansis is (I think) attacking this from the selinux/MAC side.  It would
> be great to hear from users, libvirt folks, or anyone else in that
> thread to help push it to a conclusion one way or another (even if the
> approach that I've proposed is crap and wrong - then say it there :).
>
> Hope this helps,
> -Aaron
>

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Daniel P. Berrange]

On Tue, Jun 14, 2016 at 04:03:43PM +0800, Wei Xu wrote:
> On 2016年06月09日 05:48, Aaron Conole wrote:
> > Flavio Leitner <fbl@redhat.com> writes:
> > 
> > > Adding Aaron who is fixing exactly that on the OVS side.
> > > 
> > > Aaron, please see the last question in the bottom of this email.
> > > 
> > > On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Michal Privoznik" <mprivozn@redhat.com>
> > > > > To: "Daniel P. Berrange" <berrange@redhat.com>
> > > > > Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
> > > > > jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
> > > > > armbru@redhat.com
> > > > > Sent: Thursday, June 2, 2016 2:38:53 PM
> > > > > Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
> > > > > 'fd' open from outside for unix socket
> > > > > 
> > > > > On 02.06.2016 10:29, Daniel P. Berrange wrote:
> > > > > > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> > > > > > > On 01.06.2016 18:16, Wei Xu wrote:
> > > > > > > > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> > > > > > > > > On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> > > > > > > > > > From: Wei Xu <wexu@redhat.com>
> > > > > > > > > > 
> > > > > > > > > > Recently I'm working on a fd passing issue, selinux forbids qemu to
> > > > > > > > > > create a unix socket for a chardev when managing VMs with libvirt,
> > > > > > > > > > because qemu don't have sufficient permissions in this case, and
> > > > > > > > > > proposal from libvirt team is opening the 'fd' in libvirt and merely
> > > > > > > > > > passing it to qemu.
> > > > > > > > > 
> > > > > > > > > This sounds like a bug in libvirt, or selinux, or a mistaken
> > > > > > > > > configuration
> > > > > > > > > of the guest. It is entirely possible for QEMU to create a unix socket
> > > > > > > > > - not
> > > > > > > > > least because that is exactly what QEMU uses for its QMP monitor
> > > > > > > > > backend.
> > > > > > > > > Looking at your example command line, I think the issue is simply that
> > > > > > > > > you
> > > > > > > > > should be putting the sockets in a different location. ie at
> > > > > > > > > /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> > > > > > > > > permission to
> > > > > > > > > create sockets already.
> > > > > > > > ah.. adjusting permission or file location can solve this problem, i'm
> > > > > > > > guessing maybe this is a more security concern, the socket is used as a
> > > > > > > > network interface for a vm, similar as the qcow image file, thus should
> > > > > > > > prevent it to be arbitrarily accessed.
> > > > > > > > 
> > > > > > > > Michael, do you have any comment on this?
> > > > > > > 
> > > > > > > I haven't seen the patches. But in libvirt we allow users to create a
> > > > > > > vhostuser interface and even specify where the socket should be placed:
> > > > > > > 
> > > > > > >      <interface type='vhostuser'>
> > > > > > >        <mac address='52:54:00:ee:96:6c'/>
> > > > > > >        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> > > > > > >        <model type='virtio'/>
> > > > > > >      </interface>
> > > > > > > 
> > > > > > > The following cmd line is generated by libvirt then:
> > > > > > > 
> > > > > > > -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> > > > > > > -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> > > > > > > -device
> > > > > > > virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> > > > > > > 
> > > > > > > Now, if we accept only /var/run/openvwitch path in
> > > > > > > /interface/source/@path (or whatever path to OVS is), we don't need this
> > > > > > > and have users manually label the dir (unless already labeled). But
> > > > > > > since we accept just any path in there, we should make sure that qemu is
> > > > > > > then able to create the socket. One possible fix would be to allow qemu
> > > > > > > create sockets just anywhere in the system. This, however, brings huge
> > > > > > > security risks and it's not acceptable IMO. The other option would be
> > > > > > > that libvirt would create the socket, and pass its FD to qemu (since
> > > > > > > libvirt already is allowed to create sockets anywhere).
> > > > > > 
> > > > > > There are plenty of other places where we allow arbitrary paths in the
> > > > > > XML, but which have restrictions imposed by the security drivers. Not
> > > > > > least the <channel> devices which have the exact same scenario as this
> > > > > > network device, and require use of /var/lib/libvirt/qemu as the directory
> > > > > > for the sockets. We certainly do not want to allow QEMU to create sockets
> > > > > > anywhere.
> > > > > > 
> > > > > > I don't think we want to grant QEMU svirtt permission to create sockets
> > > > > > in the /var/run/openvswitch directory either really.IMHO, users of vhost
> > > > > > user should really be using /var/lib/libvirt/qemu, as is used for all
> > > > > > other UNIX sockets we create wrt other devices.
> > > > > 
> > > > > Okay. I can live with that; but in that case we should document it
> > > > > somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> > > > > accessible and for the rest we do our best but maybe require sys admin
> > > > > intervention (e.g. to label the whole tree for a non-standard location).
> > > > 
> > > > Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
> > 
> > As of a recent commit, it can only be in /var/run/openvswitch or a
> > subdirectory therein (found in the openvswitch database).
> Aaron, thanks for your reply.
> 
> Just a question about the usage of openvswitch, in this user case when
> launching a vhostuser/dpdk via libvirt, qemu works as server mode for socket
> under /var/run/openvswitch, but per my previous test, ovs/dpdk always works
> as server mode, which means ovs will creates the socket and listening for
> connection, thus qemu works as client mode, does ovs/dpdk support working in
> client mode? which means it's qemu's duty to create the socket? and ovs will
> connect to it on demanding?

Oh, I was assuming that QEMU would be working in server mode - no wonder
we have somewhat different views :-)

If OVS is running the UNIX socket server, and QEMU is purely the client,
then that means the solution would be slightly different. In particular
libvirt would *not* do any SELinux relabelling. Instead you would have
to get an addition to the SELinux policy, to allow svirt_t type to connect
to the SELinux type associated with the openvswitch socket.

For file permissions, if the OVS socket is owned by a particular UNIX
group, you could potentially add the 'qemu' user account to that group
to grant access.


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Amnon Ilan]

----- Original Message -----
> From: "Wei Xu" <wexu@redhat.com>
> To: "Aaron Conole" <aconole@redhat.com>, "Flavio Leitner" <fbl@redhat.com>
> Cc: "Michal Privoznik" <mprivozn@redhat.com>, jasowang@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, "amit
> shah" <amit.shah@redhat.com>, "Amnon Ilan" <ailan@redhat.com>
> Sent: Tuesday, June 14, 2016 11:03:43 AM
> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket
> 
> On 2016年06月09日 05:48, Aaron Conole wrote:
> > Flavio Leitner <fbl@redhat.com> writes:
> >
> >> Adding Aaron who is fixing exactly that on the OVS side.
> >>
> >> Aaron, please see the last question in the bottom of this email.
> >>
> >> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Michal Privoznik" <mprivozn@redhat.com>
> >>>> To: "Daniel P. Berrange" <berrange@redhat.com>
> >>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
> >>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
> >>>> armbru@redhat.com
> >>>> Sent: Thursday, June 2, 2016 2:38:53 PM
> >>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
> >>>> 'fd' open from outside for unix socket
> >>>>
> >>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
> >>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
> >>>>>> On 01.06.2016 18:16, Wei Xu wrote:
> >>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
> >>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
> >>>>>>>>> From: Wei Xu <wexu@redhat.com>
> >>>>>>>>>
> >>>>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
> >>>>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
> >>>>>>>>> because qemu don't have sufficient permissions in this case, and
> >>>>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and
> >>>>>>>>> merely
> >>>>>>>>> passing it to qemu.
> >>>>>>>>
> >>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
> >>>>>>>> configuration
> >>>>>>>> of the guest. It is entirely possible for QEMU to create a unix
> >>>>>>>> socket
> >>>>>>>> - not
> >>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
> >>>>>>>> backend.
> >>>>>>>> Looking at your example command line, I think the issue is simply
> >>>>>>>> that
> >>>>>>>> you
> >>>>>>>> should be putting the sockets in a different location. ie at
> >>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
> >>>>>>>> permission to
> >>>>>>>> create sockets already.
> >>>>>>> ah.. adjusting permission or file location can solve this problem,
> >>>>>>> i'm
> >>>>>>> guessing maybe this is a more security concern, the socket is used as
> >>>>>>> a
> >>>>>>> network interface for a vm, similar as the qcow image file, thus
> >>>>>>> should
> >>>>>>> prevent it to be arbitrarily accessed.
> >>>>>>>
> >>>>>>> Michael, do you have any comment on this?
> >>>>>>
> >>>>>> I haven't seen the patches. But in libvirt we allow users to create a
> >>>>>> vhostuser interface and even specify where the socket should be
> >>>>>> placed:
> >>>>>>
> >>>>>>      <interface type='vhostuser'>
> >>>>>>        <mac address='52:54:00:ee:96:6c'/>
> >>>>>>        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
> >>>>>>        <model type='virtio'/>
> >>>>>>      </interface>
> >>>>>>
> >>>>>> The following cmd line is generated by libvirt then:
> >>>>>>
> >>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
> >>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
> >>>>>> -device
> >>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
> >>>>>>
> >>>>>> Now, if we accept only /var/run/openvwitch path in
> >>>>>> /interface/source/@path (or whatever path to OVS is), we don't need
> >>>>>> this
> >>>>>> and have users manually label the dir (unless already labeled). But
> >>>>>> since we accept just any path in there, we should make sure that qemu
> >>>>>> is
> >>>>>> then able to create the socket. One possible fix would be to allow
> >>>>>> qemu
> >>>>>> create sockets just anywhere in the system. This, however, brings huge
> >>>>>> security risks and it's not acceptable IMO. The other option would be
> >>>>>> that libvirt would create the socket, and pass its FD to qemu (since
> >>>>>> libvirt already is allowed to create sockets anywhere).
> >>>>>
> >>>>> There are plenty of other places where we allow arbitrary paths in the
> >>>>> XML, but which have restrictions imposed by the security drivers. Not
> >>>>> least the <channel> devices which have the exact same scenario as this
> >>>>> network device, and require use of /var/lib/libvirt/qemu as the
> >>>>> directory
> >>>>> for the sockets. We certainly do not want to allow QEMU to create
> >>>>> sockets
> >>>>> anywhere.
> >>>>>
> >>>>> I don't think we want to grant QEMU svirtt permission to create sockets
> >>>>> in the /var/run/openvswitch directory either really.IMHO, users of
> >>>>> vhost
> >>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
> >>>>> other UNIX sockets we create wrt other devices.
> >>>>
> >>>> Okay. I can live with that; but in that case we should document it
> >>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
> >>>> accessible and for the rest we do our best but maybe require sys admin
> >>>> intervention (e.g. to label the whole tree for a non-standard location).
> >>>
> >>> Does OVS has some limit for it's sockets to be only in
> >>> /var/run/openvswitch ?
> >
> > As of a recent commit, it can only be in /var/run/openvswitch or a
> > subdirectory therein (found in the openvswitch database).
> Aaron, thanks for your reply.
> 
> Just a question about the usage of openvswitch, in this user case when
> launching a vhostuser/dpdk via libvirt, qemu works as server mode for
> socket under /var/run/openvswitch, but per my previous test, ovs/dpdk
> always works as server mode, which means ovs will creates the socket and
> listening for connection, thus qemu works as client mode, does ovs/dpdk
> support working in client mode? which means it's qemu's duty to create
> the socket? and ovs will connect to it on demanding?

AFAIK qemu supports both client mode and server mode (configurable), so 
the solution should support both cases.

Thanks,
Amnon

> 
> >
> >>> Flavio, do you know?
> >>> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
> >
> > Probably not.  There are other issues as well.  From a DAC perspective
> > (so forgetting selinux at the moment), qemu and ovs run as different
> > users.  This will mean that when ovs creates the unix domain socket
> > file, qemu will not be allowed to actually open it properly.  I have a
> > commit I'm trying to get upstream for that particular issue (see
> > bz:1281911 and upstream list discussion:
> >    http://openvswitch.org/pipermail/dev/2016-May/071453.html
> >    and
> >    http://openvswitch.org/pipermail/dev/2016-June/071978.html)
> >
> > Ansis is (I think) attacking this from the selinux/MAC side.  It would
> > be great to hear from users, libvirt folks, or anyone else in that
> > thread to help push it to a conclusion one way or another (even if the
> > approach that I've proposed is crap and wrong - then say it there :).
> >
> > Hope this helps,
> > -Aaron
> >
> 
> 

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Aaron Conole]

Wei Xu <wexu@redhat.com> writes:

> On 2016年06月09日 05:48, Aaron Conole wrote:
>> Flavio Leitner <fbl@redhat.com> writes:
>>
>>> Adding Aaron who is fixing exactly that on the OVS side.
>>>
>>> Aaron, please see the last question in the bottom of this email.
>>>
>>> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Michal Privoznik" <mprivozn@redhat.com>
>>>>> To: "Daniel P. Berrange" <berrange@redhat.com>
>>>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>>>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>>>>> armbru@redhat.com
>>>>> Sent: Thursday, June 2, 2016 2:38:53 PM
>>>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>>>>> 'fd' open from outside for unix socket
>>>>>
>>>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>>>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>>>>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>>>>>
>>>>>>>>>> Recently I'm working on a fd passing issue, selinux forbids qemu to
>>>>>>>>>> create a unix socket for a chardev when managing VMs with libvirt,
>>>>>>>>>> because qemu don't have sufficient permissions in this case, and
>>>>>>>>>> proposal from libvirt team is opening the 'fd' in libvirt and merely
>>>>>>>>>> passing it to qemu.
>>>>>>>>>
>>>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>>>>>> configuration
>>>>>>>>> of the guest. It is entirely possible for QEMU to create a unix socket
>>>>>>>>> - not
>>>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
>>>>>>>>> backend.
>>>>>>>>> Looking at your example command line, I think the issue is simply that
>>>>>>>>> you
>>>>>>>>> should be putting the sockets in a different location. ie at
>>>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>>>>>> permission to
>>>>>>>>> create sockets already.
>>>>>>>> ah.. adjusting permission or file location can solve this problem, i'm
>>>>>>>> guessing maybe this is a more security concern, the socket is used as a
>>>>>>>> network interface for a vm, similar as the qcow image file, thus should
>>>>>>>> prevent it to be arbitrarily accessed.
>>>>>>>>
>>>>>>>> Michael, do you have any comment on this?
>>>>>>>
>>>>>>> I haven't seen the patches. But in libvirt we allow users to create a
>>>>>>> vhostuser interface and even specify where the socket should be placed:
>>>>>>>
>>>>>>>      <interface type='vhostuser'>
>>>>>>>        <mac address='52:54:00:ee:96:6c'/>
>>>>>>>        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>>>>>        <model type='virtio'/>
>>>>>>>      </interface>
>>>>>>>
>>>>>>> The following cmd line is generated by libvirt then:
>>>>>>>
>>>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>>>>>> -device
>>>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>>>>>
>>>>>>> Now, if we accept only /var/run/openvwitch path in
>>>>>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>>>>>> and have users manually label the dir (unless already labeled). But
>>>>>>> since we accept just any path in there, we should make sure that qemu is
>>>>>>> then able to create the socket. One possible fix would be to allow qemu
>>>>>>> create sockets just anywhere in the system. This, however, brings huge
>>>>>>> security risks and it's not acceptable IMO. The other option would be
>>>>>>> that libvirt would create the socket, and pass its FD to qemu (since
>>>>>>> libvirt already is allowed to create sockets anywhere).
>>>>>>
>>>>>> There are plenty of other places where we allow arbitrary paths in the
>>>>>> XML, but which have restrictions imposed by the security drivers. Not
>>>>>> least the <channel> devices which have the exact same scenario as this
>>>>>> network device, and require use of /var/lib/libvirt/qemu as the directory
>>>>>> for the sockets. We certainly do not want to allow QEMU to create sockets
>>>>>> anywhere.
>>>>>>
>>>>>> I don't think we want to grant QEMU svirtt permission to create sockets
>>>>>> in the /var/run/openvswitch directory either really.IMHO, users of vhost
>>>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
>>>>>> other UNIX sockets we create wrt other devices.
>>>>>
>>>>> Okay. I can live with that; but in that case we should document it
>>>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>>>>> accessible and for the rest we do our best but maybe require sys admin
>>>>> intervention (e.g. to label the whole tree for a non-standard location).
>>>>
>>>> Does OVS has some limit for it's sockets to be only in /var/run/openvswitch ?
>>
>> As of a recent commit, it can only be in /var/run/openvswitch or a
>> subdirectory therein (found in the openvswitch database).
> Aaron, thanks for your reply.
>
> Just a question about the usage of openvswitch, in this user case when
> launching a vhostuser/dpdk via libvirt, qemu works as server mode for
> socket under /var/run/openvswitch, but per my previous test, ovs/dpdk
> always works as server mode, which means ovs will creates the socket
> and listening for connection, thus qemu works as client mode, does
> ovs/dpdk support working in client mode? which means it's qemu's duty
> to create the socket? and ovs will connect to it on demanding?

OvS does not currently support client mode.  It will be some work to add
support, as dpdk upstream does not currently support vhostuser in client
mode, and that would need to be added before ovs would have support.

>>
>>>> Flavio, do you know?
>>>> If not, we are good as it is today with /var/lib/libvirt/qemu, right?
>>
>> Probably not.  There are other issues as well.  From a DAC perspective
>> (so forgetting selinux at the moment), qemu and ovs run as different
>> users.  This will mean that when ovs creates the unix domain socket
>> file, qemu will not be allowed to actually open it properly.  I have a
>> commit I'm trying to get upstream for that particular issue (see
>> bz:1281911 and upstream list discussion:
>>    http://openvswitch.org/pipermail/dev/2016-May/071453.html
>>    and
>>    http://openvswitch.org/pipermail/dev/2016-June/071978.html)
>>
>> Ansis is (I think) attacking this from the selinux/MAC side.  It would
>> be great to hear from users, libvirt folks, or anyone else in that
>> thread to help push it to a conclusion one way or another (even if the
>> approach that I've proposed is crap and wrong - then say it there :).
>>
>> Hope this helps,
>> -Aaron
>>

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Aaron Conole]

"Daniel P. Berrange" <berrange@redhat.com> writes:

> On Tue, Jun 14, 2016 at 04:03:43PM +0800, Wei Xu wrote:
>> On 2016年06月09日 05:48, Aaron Conole wrote:
>> > Flavio Leitner <fbl@redhat.com> writes:
>> > 
>> > > Adding Aaron who is fixing exactly that on the OVS side.
>> > > 
>> > > Aaron, please see the last question in the bottom of this email.
>> > > 
>> > > On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>> > > > 
>> > > > 
>> > > > ----- Original Message -----
>> > > > > From: "Michal Privoznik" <mprivozn@redhat.com>
>> > > > > To: "Daniel P. Berrange" <berrange@redhat.com>
>> > > > > Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>> > > > > jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>> > > > > armbru@redhat.com
>> > > > > Sent: Thursday, June 2, 2016 2:38:53 PM
>> > > > > Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>> > > > > 'fd' open from outside for unix socket
>> > > > > 
>> > > > > On 02.06.2016 10:29, Daniel P. Berrange wrote:
>> > > > > > On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>> > > > > > > On 01.06.2016 18:16, Wei Xu wrote:
>> > > > > > > > On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>> > > > > > > > > On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>> > > > > > > > > > From: Wei Xu <wexu@redhat.com>
>> > > > > > > > > > 
>> > > > > > > > > > Recently I'm working on a fd passing issue,
>> > > > > > > > > > selinux forbids qemu to
>> > > > > > > > > > create a unix socket for a chardev when managing
>> > > > > > > > > > VMs with libvirt,
>> > > > > > > > > > because qemu don't have sufficient permissions in
>> > > > > > > > > > this case, and
>> > > > > > > > > > proposal from libvirt team is opening the 'fd' in
>> > > > > > > > > > libvirt and merely
>> > > > > > > > > > passing it to qemu.
>> > > > > > > > > 
>> > > > > > > > > This sounds like a bug in libvirt, or selinux, or a mistaken
>> > > > > > > > > configuration
>> > > > > > > > > of the guest. It is entirely possible for QEMU to
>> > > > > > > > > create a unix socket
>> > > > > > > > > - not
>> > > > > > > > > least because that is exactly what QEMU uses for its QMP monitor
>> > > > > > > > > backend.
>> > > > > > > > > Looking at your example command line, I think the
>> > > > > > > > > issue is simply that
>> > > > > > > > > you
>> > > > > > > > > should be putting the sockets in a different location. ie at
>> > > > > > > > > /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>> > > > > > > > > permission to
>> > > > > > > > > create sockets already.
>> > > > > > > > ah.. adjusting permission or file location can solve
>> > > > > > > > this problem, i'm
>> > > > > > > > guessing maybe this is a more security concern, the
>> > > > > > > > socket is used as a
>> > > > > > > > network interface for a vm, similar as the qcow image
>> > > > > > > > file, thus should
>> > > > > > > > prevent it to be arbitrarily accessed.
>> > > > > > > > 
>> > > > > > > > Michael, do you have any comment on this?
>> > > > > > > 
>> > > > > > > I haven't seen the patches. But in libvirt we allow users to create a
>> > > > > > > vhostuser interface and even specify where the socket should be placed:
>> > > > > > > 
>> > > > > > >      <interface type='vhostuser'>
>> > > > > > >        <mac address='52:54:00:ee:96:6c'/>
>> > > > > > >        <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>> > > > > > >        <model type='virtio'/>
>> > > > > > >      </interface>
>> > > > > > > 
>> > > > > > > The following cmd line is generated by libvirt then:
>> > > > > > > 
>> > > > > > > -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>> > > > > > > -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>> > > > > > > -device
>> > > > > > > virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>> > > > > > > 
>> > > > > > > Now, if we accept only /var/run/openvwitch path in
>> > > > > > > /interface/source/@path (or whatever path to OVS is), we don't need this
>> > > > > > > and have users manually label the dir (unless already labeled). But
>> > > > > > > since we accept just any path in there, we should make sure that qemu is
>> > > > > > > then able to create the socket. One possible fix would be to allow qemu
>> > > > > > > create sockets just anywhere in the system. This, however, brings huge
>> > > > > > > security risks and it's not acceptable IMO. The other option would be
>> > > > > > > that libvirt would create the socket, and pass its FD to qemu (since
>> > > > > > > libvirt already is allowed to create sockets anywhere).
>> > > > > > 
>> > > > > > There are plenty of other places where we allow arbitrary paths in the
>> > > > > > XML, but which have restrictions imposed by the security drivers. Not
>> > > > > > least the <channel> devices which have the exact same scenario as this
>> > > > > > network device, and require use of /var/lib/libvirt/qemu
>> > > > > > as the directory
>> > > > > > for the sockets. We certainly do not want to allow QEMU to
>> > > > > > create sockets
>> > > > > > anywhere.
>> > > > > > 
>> > > > > > I don't think we want to grant QEMU svirtt permission to create sockets
>> > > > > > in the /var/run/openvswitch directory either really.IMHO,
>> > > > > > users of vhost
>> > > > > > user should really be using /var/lib/libvirt/qemu, as is used for all
>> > > > > > other UNIX sockets we create wrt other devices.
>> > > > > 
>> > > > > Okay. I can live with that; but in that case we should document it
>> > > > > somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>> > > > > accessible and for the rest we do our best but maybe require sys admin
>> > > > > intervention (e.g. to label the whole tree for a non-standard location).
>> > > > 
>> > > > Does OVS has some limit for it's sockets to be only in
>> > > > /var/run/openvswitch ?
>> > 
>> > As of a recent commit, it can only be in /var/run/openvswitch or a
>> > subdirectory therein (found in the openvswitch database).
>> Aaron, thanks for your reply.
>> 
>> Just a question about the usage of openvswitch, in this user case when
>> launching a vhostuser/dpdk via libvirt, qemu works as server mode for socket
>> under /var/run/openvswitch, but per my previous test, ovs/dpdk always works
>> as server mode, which means ovs will creates the socket and listening for
>> connection, thus qemu works as client mode, does ovs/dpdk support working in
>> client mode? which means it's qemu's duty to create the socket? and ovs will
>> connect to it on demanding?
>
> Oh, I was assuming that QEMU would be working in server mode - no wonder
> we have somewhat different views :-)
>
> If OVS is running the UNIX socket server, and QEMU is purely the client,
> then that means the solution would be slightly different. In particular
> libvirt would *not* do any SELinux relabelling. Instead you would have
> to get an addition to the SELinux policy, to allow svirt_t type to connect
> to the SELinux type associated with the openvswitch socket.

I agree, this is a good MAC solution.

> For file permissions, if the OVS socket is owned by a particular UNIX
> group, you could potentially add the 'qemu' user account to that group
> to grant access.

I actually would propose making a new vhost group, and adding the qemu
user account and openvswitch user accounts to that group.  That way reduces
pollution into other aspects of each process' function.

-Aaron

>
> Regards,
> Daniel

Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

On 2016年06月14日 22:23, Aaron Conole wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> On Tue, Jun 14, 2016 at 04:03:43PM +0800, Wei Xu wrote:
>>> On 2016年06月09日 05:48, Aaron Conole wrote:
>>>> Flavio Leitner <fbl@redhat.com> writes:
>>>>
>>>>> Adding Aaron who is fixing exactly that on the OVS side.
>>>>>
>>>>> Aaron, please see the last question in the bottom of this email.
>>>>>
>>>>> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Michal Privoznik" <mprivozn@redhat.com>
>>>>>>> To: "Daniel P. Berrange" <berrange@redhat.com>
>>>>>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>>>>>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>>>>>>> armbru@redhat.com
>>>>>>> Sent: Thursday, June 2, 2016 2:38:53 PM
>>>>>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>>>>>>> 'fd' open from outside for unix socket
>>>>>>>
>>>>>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>>>>>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>>>>>>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>>>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>>>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>>>>>>>
>>>>>>>>>>>> Recently I'm working on a fd passing issue,
>>>>>>>>>>>> selinux forbids qemu to
>>>>>>>>>>>> create a unix socket for a chardev when managing
>>>>>>>>>>>> VMs with libvirt,
>>>>>>>>>>>> because qemu don't have sufficient permissions in
>>>>>>>>>>>> this case, and
>>>>>>>>>>>> proposal from libvirt team is opening the 'fd' in
>>>>>>>>>>>> libvirt and merely
>>>>>>>>>>>> passing it to qemu.
>>>>>>>>>>>
>>>>>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>>>>>>>> configuration
>>>>>>>>>>> of the guest. It is entirely possible for QEMU to
>>>>>>>>>>> create a unix socket
>>>>>>>>>>> - not
>>>>>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
>>>>>>>>>>> backend.
>>>>>>>>>>> Looking at your example command line, I think the
>>>>>>>>>>> issue is simply that
>>>>>>>>>>> you
>>>>>>>>>>> should be putting the sockets in a different location. ie at
>>>>>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>>>>>>>> permission to
>>>>>>>>>>> create sockets already.
>>>>>>>>>> ah.. adjusting permission or file location can solve
>>>>>>>>>> this problem, i'm
>>>>>>>>>> guessing maybe this is a more security concern, the
>>>>>>>>>> socket is used as a
>>>>>>>>>> network interface for a vm, similar as the qcow image
>>>>>>>>>> file, thus should
>>>>>>>>>> prevent it to be arbitrarily accessed.
>>>>>>>>>>
>>>>>>>>>> Michael, do you have any comment on this?
>>>>>>>>>
>>>>>>>>> I haven't seen the patches. But in libvirt we allow users to create a
>>>>>>>>> vhostuser interface and even specify where the socket should be placed:
>>>>>>>>>
>>>>>>>>>       <interface type='vhostuser'>
>>>>>>>>>         <mac address='52:54:00:ee:96:6c'/>
>>>>>>>>>         <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>>>>>>>         <model type='virtio'/>
>>>>>>>>>       </interface>
>>>>>>>>>
>>>>>>>>> The following cmd line is generated by libvirt then:
>>>>>>>>>
>>>>>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>>>>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>>>>>>>> -device
>>>>>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>>>>>>>
>>>>>>>>> Now, if we accept only /var/run/openvwitch path in
>>>>>>>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>>>>>>>> and have users manually label the dir (unless already labeled). But
>>>>>>>>> since we accept just any path in there, we should make sure that qemu is
>>>>>>>>> then able to create the socket. One possible fix would be to allow qemu
>>>>>>>>> create sockets just anywhere in the system. This, however, brings huge
>>>>>>>>> security risks and it's not acceptable IMO. The other option would be
>>>>>>>>> that libvirt would create the socket, and pass its FD to qemu (since
>>>>>>>>> libvirt already is allowed to create sockets anywhere).
>>>>>>>>
>>>>>>>> There are plenty of other places where we allow arbitrary paths in the
>>>>>>>> XML, but which have restrictions imposed by the security drivers. Not
>>>>>>>> least the <channel> devices which have the exact same scenario as this
>>>>>>>> network device, and require use of /var/lib/libvirt/qemu
>>>>>>>> as the directory
>>>>>>>> for the sockets. We certainly do not want to allow QEMU to
>>>>>>>> create sockets
>>>>>>>> anywhere.
>>>>>>>>
>>>>>>>> I don't think we want to grant QEMU svirtt permission to create sockets
>>>>>>>> in the /var/run/openvswitch directory either really.IMHO,
>>>>>>>> users of vhost
>>>>>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
>>>>>>>> other UNIX sockets we create wrt other devices.
>>>>>>>
>>>>>>> Okay. I can live with that; but in that case we should document it
>>>>>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>>>>>>> accessible and for the rest we do our best but maybe require sys admin
>>>>>>> intervention (e.g. to label the whole tree for a non-standard location).
>>>>>>
>>>>>> Does OVS has some limit for it's sockets to be only in
>>>>>> /var/run/openvswitch ?
>>>>
>>>> As of a recent commit, it can only be in /var/run/openvswitch or a
>>>> subdirectory therein (found in the openvswitch database).
>>> Aaron, thanks for your reply.
>>>
>>> Just a question about the usage of openvswitch, in this user case when
>>> launching a vhostuser/dpdk via libvirt, qemu works as server mode for socket
>>> under /var/run/openvswitch, but per my previous test, ovs/dpdk always works
>>> as server mode, which means ovs will creates the socket and listening for
>>> connection, thus qemu works as client mode, does ovs/dpdk support working in
>>> client mode? which means it's qemu's duty to create the socket? and ovs will
>>> connect to it on demanding?
>>
>> Oh, I was assuming that QEMU would be working in server mode - no wonder
>> we have somewhat different views :-)
>>
>> If OVS is running the UNIX socket server, and QEMU is purely the client,
>> then that means the solution would be slightly different. In particular
>> libvirt would *not* do any SELinux relabelling. Instead you would have
>> to get an addition to the SELinux policy, to allow svirt_t type to connect
>> to the SELinux type associated with the openvswitch socket.
>
> I agree, this is a good MAC solution.
OK.
>
>> For file permissions, if the OVS socket is owned by a particular UNIX
>> group, you could potentially add the 'qemu' user account to that group
>> to grant access.
>
> I actually would propose making a new vhost group, and adding the qemu
> user account and openvswitch user accounts to that group.  That way reduces
> pollution into other aspects of each process' function.
Michal,
Can we fix it with this as a replacement?

One of my concerns about the patch is looks there are a few other places 
in system which need 'fd' like sockets, therefore each needs a new 'fd' 
parameter in command line, this maybe not a clean and concise way, i 
think qemu_open() proposed by Eric is better to generally solve these 
kind of issues, maybe we can consider enhancing it along that direction 
in the future, what's your opinion?
>
> -Aaron
>
>>
>> Regards,
>> Daniel
>

[Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu]

There has been comments on this patch, but i forgot adding this patch to 
the list, just forward it again.

Recently I'm working on a fd passing issue, selinux forbids qemu to
create a unix socket for a chardev when managing VMs with libvirt,
because qemu don't have sufficient permissions in this case, and
proposal from libvirt team is opening the 'fd' in libvirt and merely
passing it to qemu.

I finished a RFC patch for Unix socket after a glance of the code,
and not sure if this is right or there maybe other side-effects,
please point me out.

I tested it for both server and client mode 'PF_UNIX' socket with a VM
running vhost-user.

Old command line:
-chardev socket,id=char0,path=/var/run/openvswitch/vhost-user1,server

New command line:
-chardev 
socket,id=char0,path=/var/run/openvswitch/vhost-user1,server,sockfd=$n

because unix socket is bundled with a path, so it should be kept even 
with the
'fd' is indicated, this looks odd, any comments?

Wei Xu (3):
   chardev: add new socket fd parameter for unix socket
   chardev: save the passed in 'fd' parameter during parsing
   sockets: replace creating a new socket with the record one

  qapi-schema.json    |  3 ++-
  qemu-char.c         | 10 ++++++++++
  util/qemu-sockets.c | 25 +++++++++++++++++--------
  3 files changed, 29 insertions(+), 9 deletions(-)

-- 
2.7.1

[Qemu-devel] [RFC Patch 2/3] chardev: save the passed in 'fd' parameter during parsing

[Wei Xu]

There has been comments on this patch, but i forgot adding this patch to 
the list, just forward it again.

Save the 'fd' paramter as unix socket 'sockfd' member.

Signed-off-by: Wei Xu <wexu@redhat.com>
---
  qemu-char.c | 7 +++++++
  1 file changed, 7 insertions(+)

diff --git a/qemu-char.c b/qemu-char.c
index ea9c02e..8d20494 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -3664,6 +3664,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, 
ChardevBackend *backend,
      bool is_telnet      = qemu_opt_get_bool(opts, "telnet", false);
      bool do_nodelay     = !qemu_opt_get_bool(opts, "delay", true);
      int64_t reconnect   = qemu_opt_get_number(opts, "reconnect", 0);
+    const int32_t fd = (int32_t)qemu_opt_get_number(opts, "sockfd", 0);
      const char *path = qemu_opt_get(opts, "path");
      const char *host = qemu_opt_get(opts, "host");
      const char *port = qemu_opt_get(opts, "port");
@@ -3708,6 +3709,12 @@ static void qemu_chr_parse_socket(QemuOpts *opts, 
ChardevBackend *backend,
          addr->type = SOCKET_ADDRESS_KIND_UNIX;
          q_unix = addr->u.q_unix.data = g_new0(UnixSocketAddress, 1);
          q_unix->path = g_strdup(path);
+
+        if (fd) {
+            q_unix->sockfd = fd;
+        } else {
+            q_unix->sockfd = 0;
+        }
      } else {
          addr->type = SOCKET_ADDRESS_KIND_INET;
          addr->u.inet.data = g_new(InetSocketAddress, 1);
-- 
2.7.1