[Virtio-fs] restorcon/SELinux virtiofs question

[Virtio-fs] restorcon/SELinux virtiofs question

[Harry G. Coin]

Hello virtiofs team.  I need clarification about a 'restorecon' selinux
guest giving an 'operation not supported' response.

If the host fs is btrfs (with xattr enabled in virtiofsd) but not
running SELinux, and the guest has virtiofs root with selinux active,
what version [if any] for virtiofs is necessary before I can expect the
restorecon command to operate properly?  (Or, maybe I've missed a config
setting somewhere?) 

Packages such as freeipa fail to install because they issue dozens of
'restorecon' calls which fail using virtiofs.

Thanks,

Harry Coin

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
> guest giving an 'operation not supported' response.
> 
> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
> running SELinux,

I suspect that on host setxattr(security.selinux) is failing with 
"operation not supported". 

What do you mean by host "not running SELinux". SElinux is not compiled
in? Or it is disabled or in passive mode?

Is it working with filesystems other than btrfs, say ext4 or xfs.

Now qemu supports xattr remapping. You might want to run virtiofsd
to remap security.selinux. I think that might get you going till
the root cause of the issue is found.

Vivek

> and the guest has virtiofs root with selinux active,
> what version [if any] for virtiofs is necessary before I can expect the
> restorecon command to operate properly?  (Or, maybe I've missed a config
> setting somewhere?) 
> 
> Packages such as freeipa fail to install because they issue dozens of
> 'restorecon' calls which fail using virtiofs.
> 
> Thanks,
> 
> Harry Coin
> 
> 
> 
> 
> _______________________________________________
> Virtio-fs mailing list
> Virtio-fs@redhat.com
> https://www.redhat.com/mailman/listinfo/virtio-fs

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Harry G. Coin]

On 11/19/20 12:16 PM, Vivek Goyal wrote:
> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
>> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
>> guest giving an 'operation not supported' response.
>>
>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
>> running SELinux,
> I suspect that on host setxattr(security.selinux) is failing with 
> "operation not supported". 
>
> What do you mean by host "not running SELinux". SElinux is not compiled
> in? Or it is disabled or in passive mode?
>
> Is it working with filesystems other than btrfs, say ext4 or xfs.
>
> Now qemu supports xattr remapping. You might want to run virtiofsd
> to remap security.selinux. I think that might get you going till
> the root cause of the issue is found.
>
> Vivek

Thank you for the focus.   The host os in this instance is not from the
fedora/rhel/centos world with selinux running.  My case is a debian
sourced distro (ubuntu).  That world uses 'apparmor' by default, not
selinux.   I think it's reasonable to suppose there are a lot of servers
out there not running selinux that have lots of vms running on them, not
all using virtiofs.  There should be a documented way to allow the
'restorcon' command on one of many guests on such hosts to work.  I
suppose to wrap this up:

For the future readers who got here by searching,  could you give the
first kernel version that supports a non-selinux host supporting an
selinux enabled guest and the virtiofsd command line necessary to get
the restorecon command to work normally?

Thanks in advance!!  (And thanks for the work -- can't wait for dax to
make it into standard kernels!!)

Harry Coin




>
>> and the guest has virtiofs root with selinux active,
>> what version [if any] for virtiofs is necessary before I can expect the
>> restorecon command to operate properly?  (Or, maybe I've missed a config
>> setting somewhere?) 
>>
>> Packages such as freeipa fail to install because they issue dozens of
>> 'restorecon' calls which fail using virtiofs.
>>
>> Thanks,
>>
>> Harry Coin
>>
>>
>>
>>
>> _______________________________________________
>> Virtio-fs mailing list
>> Virtio-fs@redhat.com
>> https://www.redhat.com/mailman/listinfo/virtio-fs

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
> 
> On 11/19/20 12:16 PM, Vivek Goyal wrote:
> > On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
> >> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
> >> guest giving an 'operation not supported' response.
> >>
> >> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
> >> running SELinux,
> > I suspect that on host setxattr(security.selinux) is failing with 
> > "operation not supported". 
> >
> > What do you mean by host "not running SELinux". SElinux is not compiled
> > in? Or it is disabled or in passive mode?
> >
> > Is it working with filesystems other than btrfs, say ext4 or xfs.
> >
> > Now qemu supports xattr remapping. You might want to run virtiofsd
> > to remap security.selinux. I think that might get you going till
> > the root cause of the issue is found.
> >
> > Vivek
> 
> Thank you for the focus.   The host os in this instance is not from the
> fedora/rhel/centos world with selinux running.  My case is a debian
> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
> selinux.   I think it's reasonable to suppose there are a lot of servers
> out there not running selinux that have lots of vms running on them, not
> all using virtiofs.  There should be a documented way to allow the
> 'restorcon' command on one of many guests on such hosts to work.  I
> suppose to wrap this up:
> 
> For the future readers who got here by searching,  could you give the
> first kernel version that supports a non-selinux host supporting an
> selinux enabled guest and the virtiofsd command line necessary to get
> the restorecon command to work normally?

I don't know yet. Because I don't know what's the root cause of the
issue.

The way you are explaining it, looks like host kernel somehow is
blocking setxattr(security.selinux). And I have no idea why. Is it
apparmor or something else.

If no selinux module is loaded on host, then as long as virtiofsd
process has CAP_SYS_ADMIN, it should be able to set security.selinux.

"Operation not supported" means error "EOPNOTSUP". I am assuming
you are running virtiofsd with "-o xattr" to make sure virtiofsd
supports xattr. If that's the case somehow kernel is returning
"EOPNOTSUP".

Can you run virtiofsd with debug option -d and try to install that
package in guest and capture outout of virtiofsd and post here. It
might confirm that host kernel is returning error.

Thanks
Vivek

> 
> Thanks in advance!!  (And thanks for the work -- can't wait for dax to
> make it into standard kernels!!)
> 
> Harry Coin
> 
> 
> 
> 
> >
> >> and the guest has virtiofs root with selinux active,
> >> what version [if any] for virtiofs is necessary before I can expect the
> >> restorecon command to operate properly?  (Or, maybe I've missed a config
> >> setting somewhere?) 
> >>
> >> Packages such as freeipa fail to install because they issue dozens of
> >> 'restorecon' calls which fail using virtiofs.
> >>
> >> Thanks,
> >>
> >> Harry Coin
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Virtio-fs mailing list
> >> Virtio-fs@redhat.com
> >> https://www.redhat.com/mailman/listinfo/virtio-fs
> 

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
> > 
> > On 11/19/20 12:16 PM, Vivek Goyal wrote:
> > > On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
> > >> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
> > >> guest giving an 'operation not supported' response.
> > >>
> > >> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
> > >> running SELinux,
> > > I suspect that on host setxattr(security.selinux) is failing with 
> > > "operation not supported". 
> > >
> > > What do you mean by host "not running SELinux". SElinux is not compiled
> > > in? Or it is disabled or in passive mode?
> > >
> > > Is it working with filesystems other than btrfs, say ext4 or xfs.
> > >
> > > Now qemu supports xattr remapping. You might want to run virtiofsd
> > > to remap security.selinux. I think that might get you going till
> > > the root cause of the issue is found.
> > >
> > > Vivek
> > 
> > Thank you for the focus.   The host os in this instance is not from the
> > fedora/rhel/centos world with selinux running.  My case is a debian
> > sourced distro (ubuntu).  That world uses 'apparmor' by default, not
> > selinux.   I think it's reasonable to suppose there are a lot of servers
> > out there not running selinux that have lots of vms running on them, not
> > all using virtiofs.  There should be a documented way to allow the
> > 'restorcon' command on one of many guests on such hosts to work.  I
> > suppose to wrap this up:
> > 
> > For the future readers who got here by searching,  could you give the
> > first kernel version that supports a non-selinux host supporting an
> > selinux enabled guest and the virtiofsd command line necessary to get
> > the restorecon command to work normally?
> 
> I don't know yet. Because I don't know what's the root cause of the
> issue.
> 
> The way you are explaining it, looks like host kernel somehow is
> blocking setxattr(security.selinux). And I have no idea why. Is it
> apparmor or something else.
> 
> If no selinux module is loaded on host, then as long as virtiofsd
> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
> 
> "Operation not supported" means error "EOPNOTSUP". I am assuming
> you are running virtiofsd with "-o xattr" to make sure virtiofsd
> supports xattr. If that's the case somehow kernel is returning
> "EOPNOTSUP".
> 
> Can you run virtiofsd with debug option -d and try to install that
> package in guest and capture outout of virtiofsd and post here. It
> might confirm that host kernel is returning error.

I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt" 
on a file in virtiofs and got "Operation not supported". I think
guest kernel failed this. Will need to debug further.

Vivek

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
> > On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
> > > 
> > > On 11/19/20 12:16 PM, Vivek Goyal wrote:
> > > > On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
> > > >> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
> > > >> guest giving an 'operation not supported' response.
> > > >>
> > > >> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
> > > >> running SELinux,
> > > > I suspect that on host setxattr(security.selinux) is failing with 
> > > > "operation not supported". 
> > > >
> > > > What do you mean by host "not running SELinux". SElinux is not compiled
> > > > in? Or it is disabled or in passive mode?
> > > >
> > > > Is it working with filesystems other than btrfs, say ext4 or xfs.
> > > >
> > > > Now qemu supports xattr remapping. You might want to run virtiofsd
> > > > to remap security.selinux. I think that might get you going till
> > > > the root cause of the issue is found.
> > > >
> > > > Vivek
> > > 
> > > Thank you for the focus.   The host os in this instance is not from the
> > > fedora/rhel/centos world with selinux running.  My case is a debian
> > > sourced distro (ubuntu).  That world uses 'apparmor' by default, not
> > > selinux.   I think it's reasonable to suppose there are a lot of servers
> > > out there not running selinux that have lots of vms running on them, not
> > > all using virtiofs.  There should be a documented way to allow the
> > > 'restorcon' command on one of many guests on such hosts to work.  I
> > > suppose to wrap this up:
> > > 
> > > For the future readers who got here by searching,  could you give the
> > > first kernel version that supports a non-selinux host supporting an
> > > selinux enabled guest and the virtiofsd command line necessary to get
> > > the restorecon command to work normally?
> > 
> > I don't know yet. Because I don't know what's the root cause of the
> > issue.
> > 
> > The way you are explaining it, looks like host kernel somehow is
> > blocking setxattr(security.selinux). And I have no idea why. Is it
> > apparmor or something else.
> > 
> > If no selinux module is loaded on host, then as long as virtiofsd
> > process has CAP_SYS_ADMIN, it should be able to set security.selinux.
> > 
> > "Operation not supported" means error "EOPNOTSUP". I am assuming
> > you are running virtiofsd with "-o xattr" to make sure virtiofsd
> > supports xattr. If that's the case somehow kernel is returning
> > "EOPNOTSUP".
> > 
> > Can you run virtiofsd with debug option -d and try to install that
> > package in guest and capture outout of virtiofsd and post here. It
> > might confirm that host kernel is returning error.
> 
> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt" 
> on a file in virtiofs and got "Operation not supported". I think
> guest kernel failed this. Will need to debug further.

Ok, Dan Walsh says that it probably is due to the fact that selinux
policy in guest is not aware of virtiofs. He has opened a PR to
add that.

https://github.com/fedora-selinux/selinux-policy/pull/478

I am not sure what distribution you are running as guest but it
probably will require similar changes. Once this package is built
I will give it a try.

Thanks
Vivek

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Daniel Walsh]

On 11/19/20 14:44, Vivek Goyal wrote:
> On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
>> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
>>> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
>>>> On 11/19/20 12:16 PM, Vivek Goyal wrote:
>>>>> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
>>>>>> Hello virtiofs team.  I need clarification about a 'restorecon' selinux
>>>>>> guest giving an 'operation not supported' response.
>>>>>>
>>>>>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
>>>>>> running SELinux,
>>>>> I suspect that on host setxattr(security.selinux) is failing with
>>>>> "operation not supported".
>>>>>
>>>>> What do you mean by host "not running SELinux". SElinux is not compiled
>>>>> in? Or it is disabled or in passive mode?
>>>>>
>>>>> Is it working with filesystems other than btrfs, say ext4 or xfs.
>>>>>
>>>>> Now qemu supports xattr remapping. You might want to run virtiofsd
>>>>> to remap security.selinux. I think that might get you going till
>>>>> the root cause of the issue is found.
>>>>>
>>>>> Vivek
>>>> Thank you for the focus.   The host os in this instance is not from the
>>>> fedora/rhel/centos world with selinux running.  My case is a debian
>>>> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
>>>> selinux.   I think it's reasonable to suppose there are a lot of servers
>>>> out there not running selinux that have lots of vms running on them, not
>>>> all using virtiofs.  There should be a documented way to allow the
>>>> 'restorcon' command on one of many guests on such hosts to work.  I
>>>> suppose to wrap this up:
>>>>
>>>> For the future readers who got here by searching,  could you give the
>>>> first kernel version that supports a non-selinux host supporting an
>>>> selinux enabled guest and the virtiofsd command line necessary to get
>>>> the restorecon command to work normally?
>>> I don't know yet. Because I don't know what's the root cause of the
>>> issue.
>>>
>>> The way you are explaining it, looks like host kernel somehow is
>>> blocking setxattr(security.selinux). And I have no idea why. Is it
>>> apparmor or something else.
>>>
>>> If no selinux module is loaded on host, then as long as virtiofsd
>>> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
>>>
>>> "Operation not supported" means error "EOPNOTSUP". I am assuming
>>> you are running virtiofsd with "-o xattr" to make sure virtiofsd
>>> supports xattr. If that's the case somehow kernel is returning
>>> "EOPNOTSUP".
>>>
>>> Can you run virtiofsd with debug option -d and try to install that
>>> package in guest and capture outout of virtiofsd and post here. It
>>> might confirm that host kernel is returning error.
>> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt"
>> on a file in virtiofs and got "Operation not supported". I think
>> guest kernel failed this. Will need to debug further.
> Ok, Dan Walsh says that it probably is due to the fact that selinux
> policy in guest is not aware of virtiofs. He has opened a PR to
> add that.
>
> https://github.com/fedora-selinux/selinux-policy/pull/478
>
> I am not sure what distribution you are running as guest but it
> probably will require similar changes. Once this package is built
> I will give it a try.
>
> Thanks
> Vivek

Correct. The Guest OS Has to have SELinux enabled and the virtiofs file 
system within the VM

needs to have SELinux policy that says it support labeling on Xattrs.  
Otherwise when you attempt

to set labels on the file system.  SELinux in side of the kernel will 
say that virtiofs does not support

SELinux labels, which is what you are seeing.

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Harry G. Coin]

On 11/20/20 9:01 AM, Daniel Walsh wrote:
> On 11/19/20 14:44, Vivek Goyal wrote:
>> On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
>>> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
>>>> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
>>>>> On 11/19/20 12:16 PM, Vivek Goyal wrote:
>>>>>> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
>>>>>>> Hello virtiofs team.  I need clarification about a 'restorecon'
>>>>>>> selinux
>>>>>>> guest giving an 'operation not supported' response.
>>>>>>>
>>>>>>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
>>>>>>> running SELinux,
>>>>>> I suspect that on host setxattr(security.selinux) is failing with
>>>>>> "operation not supported".
>>>>>>
>>>>>> What do you mean by host "not running SELinux". SElinux is not
>>>>>> compiled
>>>>>> in? Or it is disabled or in passive mode?
>>>>>>
>>>>>> Is it working with filesystems other than btrfs, say ext4 or xfs.
>>>>>>
>>>>>> Now qemu supports xattr remapping. You might want to run virtiofsd
>>>>>> to remap security.selinux. I think that might get you going till
>>>>>> the root cause of the issue is found.
>>>>>>
>>>>>> Vivek
>>>>> Thank you for the focus.   The host os in this instance is not
>>>>> from the
>>>>> fedora/rhel/centos world with selinux running.  My case is a debian
>>>>> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
>>>>> selinux.   I think it's reasonable to suppose there are a lot of
>>>>> servers
>>>>> out there not running selinux that have lots of vms running on
>>>>> them, not
>>>>> all using virtiofs.  There should be a documented way to allow the
>>>>> 'restorcon' command on one of many guests on such hosts to work.  I
>>>>> suppose to wrap this up:
>>>>>
>>>>> For the future readers who got here by searching,  could you give the
>>>>> first kernel version that supports a non-selinux host supporting an
>>>>> selinux enabled guest and the virtiofsd command line necessary to get
>>>>> the restorecon command to work normally?
>>>> I don't know yet. Because I don't know what's the root cause of the
>>>> issue.
>>>>
>>>> The way you are explaining it, looks like host kernel somehow is
>>>> blocking setxattr(security.selinux). And I have no idea why. Is it
>>>> apparmor or something else.
>>>>
>>>> If no selinux module is loaded on host, then as long as virtiofsd
>>>> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
>>>>
>>>> "Operation not supported" means error "EOPNOTSUP". I am assuming
>>>> you are running virtiofsd with "-o xattr" to make sure virtiofsd
>>>> supports xattr. If that's the case somehow kernel is returning
>>>> "EOPNOTSUP".
>>>>
>>>> Can you run virtiofsd with debug option -d and try to install that
>>>> package in guest and capture outout of virtiofsd and post here. It
>>>> might confirm that host kernel is returning error.
>>> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt"
>>> on a file in virtiofs and got "Operation not supported". I think
>>> guest kernel failed this. Will need to debug further.
>> Ok, Dan Walsh says that it probably is due to the fact that selinux
>> policy in guest is not aware of virtiofs. He has opened a PR to
>> add that.
>>
>> https://github.com/fedora-selinux/selinux-policy/pull/478
>>
>> I am not sure what distribution you are running as guest but it
>> probably will require similar changes. Once this package is built
>> I will give it a try.
>>
>> Thanks
>> Vivek
>
> Correct. The Guest OS Has to have SELinux enabled and the virtiofs
> file system within the VM
>
> needs to have SELinux policy that says it support labeling on Xattrs. 
> Otherwise when you attempt
>
> to set labels on the file system.  SELinux in side of the kernel will
> say that virtiofs does not support
>
> SELinux labels, which is what you are seeing.
>
It is the advertising and presumption of those using 'virtual machines'
that they are 'runnable' on any host.  If I read the above correctly,
because there's no telling which of the hundreds of packages in the
fedora/centos/rhel world will fail on built-in restorecon calls,
virtiofs is now excluded for general use except on SELinux enabled hosts
.    There are, (cough) a fair few hosts out there which are not running
SElinux, whose operators hope/need to provide vm guest services to the
fedora/rhel/centos package users.  So, I ask the virtiofs folks to
consider creating or defining an option allowing fedora/rhel/centos
guests a way to succeed.  Or, in the alternative, a clear warning that
virtiofs is not a good choice for  rhel/centos/fedora guests on other
than rhel/centos/fedora bare-metal requiring selinux enabled.

HC

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Fri, Nov 20, 2020 at 11:11:28AM -0600, Harry G. Coin wrote:
> 
> On 11/20/20 9:01 AM, Daniel Walsh wrote:
> > On 11/19/20 14:44, Vivek Goyal wrote:
> >> On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
> >>> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
> >>>> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
> >>>>> On 11/19/20 12:16 PM, Vivek Goyal wrote:
> >>>>>> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
> >>>>>>> Hello virtiofs team.  I need clarification about a 'restorecon'
> >>>>>>> selinux
> >>>>>>> guest giving an 'operation not supported' response.
> >>>>>>>
> >>>>>>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
> >>>>>>> running SELinux,
> >>>>>> I suspect that on host setxattr(security.selinux) is failing with
> >>>>>> "operation not supported".
> >>>>>>
> >>>>>> What do you mean by host "not running SELinux". SElinux is not
> >>>>>> compiled
> >>>>>> in? Or it is disabled or in passive mode?
> >>>>>>
> >>>>>> Is it working with filesystems other than btrfs, say ext4 or xfs.
> >>>>>>
> >>>>>> Now qemu supports xattr remapping. You might want to run virtiofsd
> >>>>>> to remap security.selinux. I think that might get you going till
> >>>>>> the root cause of the issue is found.
> >>>>>>
> >>>>>> Vivek
> >>>>> Thank you for the focus.   The host os in this instance is not
> >>>>> from the
> >>>>> fedora/rhel/centos world with selinux running.  My case is a debian
> >>>>> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
> >>>>> selinux.   I think it's reasonable to suppose there are a lot of
> >>>>> servers
> >>>>> out there not running selinux that have lots of vms running on
> >>>>> them, not
> >>>>> all using virtiofs.  There should be a documented way to allow the
> >>>>> 'restorcon' command on one of many guests on such hosts to work.  I
> >>>>> suppose to wrap this up:
> >>>>>
> >>>>> For the future readers who got here by searching,  could you give the
> >>>>> first kernel version that supports a non-selinux host supporting an
> >>>>> selinux enabled guest and the virtiofsd command line necessary to get
> >>>>> the restorecon command to work normally?
> >>>> I don't know yet. Because I don't know what's the root cause of the
> >>>> issue.
> >>>>
> >>>> The way you are explaining it, looks like host kernel somehow is
> >>>> blocking setxattr(security.selinux). And I have no idea why. Is it
> >>>> apparmor or something else.
> >>>>
> >>>> If no selinux module is loaded on host, then as long as virtiofsd
> >>>> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
> >>>>
> >>>> "Operation not supported" means error "EOPNOTSUP". I am assuming
> >>>> you are running virtiofsd with "-o xattr" to make sure virtiofsd
> >>>> supports xattr. If that's the case somehow kernel is returning
> >>>> "EOPNOTSUP".
> >>>>
> >>>> Can you run virtiofsd with debug option -d and try to install that
> >>>> package in guest and capture outout of virtiofsd and post here. It
> >>>> might confirm that host kernel is returning error.
> >>> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt"
> >>> on a file in virtiofs and got "Operation not supported". I think
> >>> guest kernel failed this. Will need to debug further.
> >> Ok, Dan Walsh says that it probably is due to the fact that selinux
> >> policy in guest is not aware of virtiofs. He has opened a PR to
> >> add that.
> >>
> >> https://github.com/fedora-selinux/selinux-policy/pull/478
> >>
> >> I am not sure what distribution you are running as guest but it
> >> probably will require similar changes. Once this package is built
> >> I will give it a try.
> >>
> >> Thanks
> >> Vivek
> >
> > Correct. The Guest OS Has to have SELinux enabled and the virtiofs
> > file system within the VM
> >
> > needs to have SELinux policy that says it support labeling on Xattrs. 
> > Otherwise when you attempt
> >
> > to set labels on the file system.  SELinux in side of the kernel will
> > say that virtiofs does not support
> >
> > SELinux labels, which is what you are seeing.
> >
> It is the advertising and presumption of those using 'virtual machines'
> that they are 'runnable' on any host.  If I read the above correctly,
> because there's no telling which of the hundreds of packages in the
> fedora/centos/rhel world will fail on built-in restorecon calls,
> virtiofs is now excluded for general use except on SELinux enabled hosts
> .

Hi,

This is SELinux policy change required in guest (and not host). So after
this change in selinux policy in guest it should work in your setup
(where you are not running SELinux on host). Can you please give it
a try. selinux developers provided simple instructions to test it.

https://github.com/fedora-selinux/selinux-policy/pull/478#issuecomment-731290656

*********************
FWIW, you can apply the fix locally without rebuilding the selinux-policy RPM as follows:

# echo '(fsuse xattr virtiofs (system_u object_r fs_t ((s0) (s0))))' >virtiofs.cil
# semodule -i virtiofs.cil
And to check that the change is applied:

# seinfo --fs_use | grep virtiofs
    fs_use_xattr virtiofs system_u:object_r:fs_t:s0;
To revert the local workaround:

# semodule -r virtiofs
***********************************

So please load above policy module in guest (and not host) and then try
installing the package which was failing for you. Please let us know
if this fixes the issue you are seeing or not. 

I tested it and it fixed the chcon issue I was seeing.

> There are, (cough) a fair few hosts out there which are not running
> SElinux, whose operators hope/need to provide vm guest services to the
> fedora/rhel/centos package users.  So, I ask the virtiofs folks to
> consider creating or defining an option allowing fedora/rhel/centos
> guests a way to succeed.  Or, in the alternative, a clear warning that
> virtiofs is not a good choice for  rhel/centos/fedora guests on other
> than rhel/centos/fedora bare-metal requiring selinux enabled.

To enable selinux in guest, we don't need selinux to be enabled 
on host.

In fact selinux policy on on host can potentially interfere with guest
policy. So I think we should run virtiofsd with remapped
"security.capability" xattr in qemu. That way both guest and host can
have their own selinux policy.

Thanks
Vivek

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Harry G. Coin]

On 11/20/20 12:55 PM, Vivek Goyal wrote:
> On Fri, Nov 20, 2020 at 11:11:28AM -0600, Harry G. Coin wrote:
>> On 11/20/20 9:01 AM, Daniel Walsh wrote:
>>> On 11/19/20 14:44, Vivek Goyal wrote:
>>>> On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
>>>>> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
>>>>>> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
>>>>>>> On 11/19/20 12:16 PM, Vivek Goyal wrote:
>>>>>>>> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
>>>>>>>>> Hello virtiofs team.  I need clarification about a 'restorecon'
>>>>>>>>> selinux
>>>>>>>>> guest giving an 'operation not supported' response.
>>>>>>>>>
>>>>>>>>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
>>>>>>>>> running SELinux,
>>>>>>>> I suspect that on host setxattr(security.selinux) is failing with
>>>>>>>> "operation not supported".
>>>>>>>>
>>>>>>>> What do you mean by host "not running SELinux". SElinux is not
>>>>>>>> compiled
>>>>>>>> in? Or it is disabled or in passive mode?
>>>>>>>>
>>>>>>>> Is it working with filesystems other than btrfs, say ext4 or xfs.
>>>>>>>>
>>>>>>>> Now qemu supports xattr remapping. You might want to run virtiofsd
>>>>>>>> to remap security.selinux. I think that might get you going till
>>>>>>>> the root cause of the issue is found.
>>>>>>>>
>>>>>>>> Vivek
>>>>>>> Thank you for the focus.   The host os in this instance is not
>>>>>>> from the
>>>>>>> fedora/rhel/centos world with selinux running.  My case is a debian
>>>>>>> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
>>>>>>> selinux.   I think it's reasonable to suppose there are a lot of
>>>>>>> servers
>>>>>>> out there not running selinux that have lots of vms running on
>>>>>>> them, not
>>>>>>> all using virtiofs.  There should be a documented way to allow the
>>>>>>> 'restorcon' command on one of many guests on such hosts to work.  I
>>>>>>> suppose to wrap this up:
>>>>>>>
>>>>>>> For the future readers who got here by searching,  could you give the
>>>>>>> first kernel version that supports a non-selinux host supporting an
>>>>>>> selinux enabled guest and the virtiofsd command line necessary to get
>>>>>>> the restorecon command to work normally?
>>>>>> I don't know yet. Because I don't know what's the root cause of the
>>>>>> issue.
>>>>>>
>>>>>> The way you are explaining it, looks like host kernel somehow is
>>>>>> blocking setxattr(security.selinux). And I have no idea why. Is it
>>>>>> apparmor or something else.
>>>>>>
>>>>>> If no selinux module is loaded on host, then as long as virtiofsd
>>>>>> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
>>>>>>
>>>>>> "Operation not supported" means error "EOPNOTSUP". I am assuming
>>>>>> you are running virtiofsd with "-o xattr" to make sure virtiofsd
>>>>>> supports xattr. If that's the case somehow kernel is returning
>>>>>> "EOPNOTSUP".
>>>>>>
>>>>>> Can you run virtiofsd with debug option -d and try to install that
>>>>>> package in guest and capture outout of virtiofsd and post here. It
>>>>>> might confirm that host kernel is returning error.
>>>>> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt"
>>>>> on a file in virtiofs and got "Operation not supported". I think
>>>>> guest kernel failed this. Will need to debug further.
>>>> Ok, Dan Walsh says that it probably is due to the fact that selinux
>>>> policy in guest is not aware of virtiofs. He has opened a PR to
>>>> add that.
>>>>
>>>> https://github.com/fedora-selinux/selinux-policy/pull/478
>>>>
>>>> I am not sure what distribution you are running as guest but it
>>>> probably will require similar changes. Once this package is built
>>>> I will give it a try.
>>>>
>>>> Thanks
>>>> Vivek
>>> Correct. The Guest OS Has to have SELinux enabled and the virtiofs
>>> file system within the VM
>>>
>>> needs to have SELinux policy that says it support labeling on Xattrs. 
>>> Otherwise when you attempt
>>>
>>> to set labels on the file system.  SELinux in side of the kernel will
>>> say that virtiofs does not support
>>>
>>> SELinux labels, which is what you are seeing.
>>>
>> It is the advertising and presumption of those using 'virtual machines'
>> that they are 'runnable' on any host.  If I read the above correctly,
>> because there's no telling which of the hundreds of packages in the
>> fedora/centos/rhel world will fail on built-in restorecon calls,
>> virtiofs is now excluded for general use except on SELinux enabled hosts
>> .
> Hi,
>
> This is SELinux policy change required in guest (and not host). So after
> this change in selinux policy in guest it should work in your setup
> (where you are not running SELinux on host). Can you please give it
> a try. selinux developers provided simple instructions to test it.
>
> https://github.com/fedora-selinux/selinux-policy/pull/478#issuecomment-731290656
>
> *********************
> FWIW, you can apply the fix locally without rebuilding the selinux-policy RPM as follows:
>
> # echo '(fsuse xattr virtiofs (system_u object_r fs_t ((s0) (s0))))' >virtiofs.cil
> # semodule -i virtiofs.cil
> And to check that the change is applied:
>
> # seinfo --fs_use | grep virtiofs
>     fs_use_xattr virtiofs system_u:object_r:fs_t:s0;
> To revert the local workaround:
>
> # semodule -r virtiofs
> ***********************************
>
> So please load above policy module in guest (and not host) and then try
> installing the package which was failing for you. Please let us know
> if this fixes the issue you are seeing or not. 
>
> I tested it and it fixed the chcon issue I was seeing.
>
>>  There are, (cough) a fair few hosts out there which are not running
>> SElinux, whose operators hope/need to provide vm guest services to the
>> fedora/rhel/centos package users.  So, I ask the virtiofs folks to
>> consider creating or defining an option allowing fedora/rhel/centos
>> guests a way to succeed.  Or, in the alternative, a clear warning that
>> virtiofs is not a good choice for  rhel/centos/fedora guests on other
>> than rhel/centos/fedora bare-metal requiring selinux enabled.
> To enable selinux in guest, we don't need selinux to be enabled 
> on host.
>
> In fact selinux policy on on host can potentially interfere with guest
> policy. So I think we should run virtiofsd with remapped
> "security.capability" xattr in qemu. That way both guest and host can
> have their own selinux policy.
>
> Thanks
> Vivek
>
Testing results follow.  Short version:  Commands above applied without
error,  failure remains until vm is rebooted, then success.  Good enough
for today!

Thanks

Harry Coin

---

  Detail:

In this case, the VM host is running a debian/ubuntu os, not running
selinux, the underlying filesystem is btrfs. 

root@noc1:~# uname -a
Linux noc1.1.quietfountain.com 5.8.0-29-generic #31-Ubuntu SMP Fri Nov 6
12:37:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

ps axu

...

root       84306  0.0  0.0  80216   992 ?        Sl   14:45   0:00
/usr/lib/qemu/virtiofsd --fd=44 -o
source=/vmsystems/fedora_generic,xattr,flock,no_posix_lock -o writeback

root       84356  4.1  0.0 4165836 30088 ?       Sl   14:45   1:23
/usr/lib/qemu/virtiofsd --fd=44 -o
source=/vmsystems/fedora_generic,xattr,flock,no_posix_lock -o writeback

...

On the otherwise default fedora workstation guest we have:

[root@fedora ~]# uname -a
Linux fedora.1.quietfountain.com 5.9.10-200.fc33.x86_64 #1 SMP Mon Nov
23 18:12:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@fedora ~]# mount

...

myfs on / type virtiofs (rw,relatime)

...

[root@fedora ~]# touch foo
[root@fedora ~]# restorecon foo
restorecon: Could not set context for /root/foo:  Operation not supported
[root@fedora ~]# echo '(fsuse xattr virtiofs (system_u object_r fs_t
((s0) (s0))))' >virtiofs.cil
[root@fedora ~]# semodule -i virtiofs.cil
[root@fedora ~]# seinfo --fs_use | grep virtiofs

fs_use_xattr virtiofs system_u:object_r:fs_t:s0;

[root@fedora ~]# restorecon foo
restorecon: Could not set context for /root/foo:  Operation not supported
[root@fedora ~]# touch foo2
[root@fedora ~]# restorecon foo2
restorecon: Could not set context for /root/foo2:  Operation not supported
[root@fedora ~]# reboot

...

[root@fedora ~]# restorecon foo2
[root@fedora ~]# touch foo3
[root@fedora ~]# restorecon foo3
[root@fedora ~]#

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Sun, Nov 29, 2020 at 03:41:00PM -0600, Harry G. Coin wrote:

[..]
> Testing results follow.  Short version:  Commands above applied without
> error,  failure remains until vm is rebooted, then success.  Good enough
> for today!

During my testing, I too noticed that I had to unmount virtiofs and mount
again and then it worked. So not sure why this SELinux policy change does
not work for a already mounted filesystem. May be some caching happens
somewhere.

Thanks
Vivek

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Harry G. Coin]

During further testing, and with the selinux patch above in place,
during an ordinary dnf package install, (virtiofs root), I see this:

...

  Running scriptlet:
bind-dyndb-ldap-11.3-5.fc33.x86_64                   13/14

  Installing       :
freeipa-server-dns-4.8.10-6.fc33.noarch              14/14
  Running scriptlet:
freeipa-server-dns-4.8.10-6.fc33.noarch              14/14
/usr/lib/tmpfiles.d/krb5-krb5kdc.conf:1: Line references path below
legacy directory /var/run/, updating /var/run/krb5kdc → /run/krb5kdc;
please update the tmpfiles.d/ drop-in file accordingly.
Cannot set file attribute for '/var/log/journal', value=0x00800000,
mask=0x00800000, ignoring: Function not implemented
Cannot set file attribute for
'/var/log/journal/659f950795794c76814a499623c1ddf1', value=0x00800000,
mask=0x00800000, ignoring: Function not implemented

  Verifying        :
bind-32:9.11.24-2.fc33.x86_64                         1/14
  Verifying        :
bind-dnssec-doc-32:9.11.24-2.fc33.noarch              2/14

...

Any ideas?

Harry

Re: [Virtio-fs] restorcon/SELinux virtiofs question

[Vivek Goyal]

On Thu, Dec 03, 2020 at 12:24:04PM -0600, Harry G. Coin wrote:
> During further testing, and with the selinux patch above in place,
> during an ordinary dnf package install, (virtiofs root), I see this:
> 
> ...
> 
>   Running scriptlet:
> bind-dyndb-ldap-11.3-5.fc33.x86_64                   13/14
> 
>   Installing       :
> freeipa-server-dns-4.8.10-6.fc33.noarch              14/14
>   Running scriptlet:
> freeipa-server-dns-4.8.10-6.fc33.noarch              14/14
> /usr/lib/tmpfiles.d/krb5-krb5kdc.conf:1: Line references path below
> legacy directory /var/run/, updating /var/run/krb5kdc → /run/krb5kdc;
> please update the tmpfiles.d/ drop-in file accordingly.
> Cannot set file attribute for '/var/log/journal', value=0x00800000,
> mask=0x00800000, ignoring: Function not implemented
> Cannot set file attribute for
> '/var/log/journal/659f950795794c76814a499623c1ddf1', value=0x00800000,
> mask=0x00800000, ignoring: Function not implemented

virtiofsd/fuse is returning error "-ENOSYS" for some operation. You
can try running virtiofsd with option "-d" which enables debug. 
(And remove --daemonize option). That will log all the messages coming
to virtiofsd and it will give clue what operation returned -ENOSYS

strace inside guest might help too. (With the option of tracing all
children).

Thanks
Vivek

> 
>   Verifying        :
> bind-32:9.11.24-2.fc33.x86_64                         1/14
>   Verifying        :
> bind-dnssec-doc-32:9.11.24-2.fc33.noarch              2/14
> 
> ...
> 
> Any ideas?
> 
> Harry
> 
>