libsepol releases

libsepol releases

[Russell Coker]

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955154

We have the above Debian bug report requesting a GIT patch for libsepol for 
GCC-10 support.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955286

We also have the above for a policy constraint error.

Could we have a new upstream release soon to cover all GCC-10 issues as well 
as any other important things?  Maybe version 3.0.1?

-- 
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

Re: libsepol releases

[Petr Lautrbach]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

On Sun, Apr 05, 2020 at 07:46:37PM +1000, Russell Coker wrote:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955154
> 
> We have the above Debian bug report requesting a GIT patch for libsepol for 
> GCC-10 support.
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955286
> 
> We also have the above for a policy constraint error.
> 
> Could we have a new upstream release soon to cover all GCC-10 issues as well 
> as any other important things?  Maybe version 3.0.1?
> 


It's more than 4 months since the latest release so I think it's worth it to release 3.1.

If there's no objection, I'll send a heads up that we'll release 3.1-rc1 next week.


Petr


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Fedora VM On Travis for Testing

[William Roberts]

I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
on Travis with execution happening in the Fedora VM.
The Fedora VM contains the selinux source code for that particular
travis build, so it has the PR patches, etc.
The VM has networking.

The build logs for the Travis run are here:
  - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489

Which comes from my git tree here:
  - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing

Note that it's super messy, I need to go through and cleanup both the
patch and the git
history. I also should verify the downloaded image is signed for the VM.
Also note that I may rebase/squash my git branch at any time.

Petr started a release document here:
  - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md

I'd like to gather feedback, and perhaps more comments on:
1. Is this CI approach worth continuing?
2. Comments on the CI scripts (I have no idea what I am doing, common issue)
3. Comments, patches, suggestions on what tests to run in this CI environment.
4. More information in in the RELEASE_PROCESS.md file. I made some
comments there
    as well on ideas.

My goal here is that a release can occur if CI is passing without
worry, and that
we automate manual steps as much as possible. This way, if we get hit by a bus
releases can occur without much effort.

Thanks,
Bill

Re: Fedora VM On Travis for Testing

[Nicolas Iooss]

On Thu, May 14, 2020 at 11:00 PM William Roberts
<bill.c.roberts@gmail.com> wrote:
>
> I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
> on Travis with execution happening in the Fedora VM.
> The Fedora VM contains the selinux source code for that particular
> travis build, so it has the PR patches, etc.
> The VM has networking.
>
> The build logs for the Travis run are here:
>   - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489
>
> Which comes from my git tree here:
>   - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing
>
> Note that it's super messy, I need to go through and cleanup both the
> patch and the git
> history. I also should verify the downloaded image is signed for the VM.
> Also note that I may rebase/squash my git branch at any time.
>
> Petr started a release document here:
>   - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md
>
> I'd like to gather feedback, and perhaps more comments on:
> 1. Is this CI approach worth continuing?
> 2. Comments on the CI scripts (I have no idea what I am doing, common issue)
> 3. Comments, patches, suggestions on what tests to run in this CI environment.
> 4. More information in in the RELEASE_PROCESS.md file. I made some
> comments there
>     as well on ideas.
>
> My goal here is that a release can occur if CI is passing without
> worry, and that
> we automate manual steps as much as possible. This way, if we get hit by a bus
> releases can occur without much effort.

Hi,
This sounds like a nice idea :) Also it appears it only takes 5
minutes to run the whole Travis CI job of setting the Fedora VM up and
booting it, which is more than acceptable in term of delays (and is
very quicker than building and testing the project itself). This could
allow to run more tests that require to be able to manage the SELinux
policy, automatically at every pull requests.

By the way I wasn't expecting Travis-CI to easily allow nested
virtualization ("nested" because jobs are already run inside VMs,
IIRC). If it works, it is great!

Some quick thoughts/suggestions/comments that I have in my head after
reading very quickly your changes:
- The networking setup of the VM seems to be more complex than needed
because you are relying on getting the IP address from DHCP. Would it
be possible to configure libvirt with port forwarding, so that "ssh -p
2222 127.0.0.1" always work? (with raw QEMU, the command-line
parameters would be something like -net
nic,model=virtio,macaddr=52:54:00:12:34:56 -net
user,hostfwd=tcp:127.0.0.1:2222-:22)
- Usually the helper scripts are located in directory scripts/, which
make them more easily discovered by people reading the code (hidden
files and directories seems more appropriate to store the
configuration for different CI platforms that require them ; some
projects choose to hide everything that is not useful for the build
process but here it does seem to be so). Are there advantages of using
.ci/ directory?
- Moreover having a "fedora-test.run" (or "fedora-test.sh", I am more
used to .sh extension for shell scripts) enables to easily reproduce
tests on a local development machine using "vagrant init
fedora/32-cloud-base && vagrant ssh" + "sudo
/vagrant/.../fedora-test.run". Such a command could be documented in
the header of the shell script.
- Right now, there are many things in .travis.yml. This file could be
split in shell script(s) if that reduces duplication.
- To verify the download of Fedora cloud image, instead of hard-coding
its SHA256 digest, you could download the checksum file which is
signed using GnuPG and check its signature, as described on
https://alt.fedoraproject.org/cloud/

Cheers,
Nicolas

Re: Fedora VM On Travis for Testing

[William Roberts]

On Thu, May 14, 2020 at 5:03 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>
> On Thu, May 14, 2020 at 11:00 PM William Roberts
> <bill.c.roberts@gmail.com> wrote:
> >
> > I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
> > on Travis with execution happening in the Fedora VM.
> > The Fedora VM contains the selinux source code for that particular
> > travis build, so it has the PR patches, etc.
> > The VM has networking.
> >
> > The build logs for the Travis run are here:
> >   - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489
> >
> > Which comes from my git tree here:
> >   - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing
> >
> > Note that it's super messy, I need to go through and cleanup both the
> > patch and the git
> > history. I also should verify the downloaded image is signed for the VM.
> > Also note that I may rebase/squash my git branch at any time.
> >
> > Petr started a release document here:
> >   - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md
> >
> > I'd like to gather feedback, and perhaps more comments on:
> > 1. Is this CI approach worth continuing?
> > 2. Comments on the CI scripts (I have no idea what I am doing, common issue)
> > 3. Comments, patches, suggestions on what tests to run in this CI environment.
> > 4. More information in in the RELEASE_PROCESS.md file. I made some
> > comments there
> >     as well on ideas.
> >
> > My goal here is that a release can occur if CI is passing without
> > worry, and that
> > we automate manual steps as much as possible. This way, if we get hit by a bus
> > releases can occur without much effort.
>
> Hi,
> This sounds like a nice idea :) Also it appears it only takes 5
> minutes to run the whole Travis CI job of setting the Fedora VM up and
> booting it, which is more than acceptable in term of delays (and is
> very quicker than building and testing the project itself). This could

Yeah I was pretty happy with the timing it took to add this. But CI doesn't
necessarily need to be fast. We run a huge test matrix, Travis only
spins up a few at a time anyways.

> allow to run more tests that require to be able to manage the SELinux
> policy, automatically at every pull requests.
>
> By the way I wasn't expecting Travis-CI to easily allow nested
> virtualization ("nested" because jobs are already run inside VMs,
> IIRC). If it works, it is great!

I was surprised too, I was expecting that to be disabled, when I cat
cpuinfo, I was very happy.

>
> Some quick thoughts/suggestions/comments that I have in my head after
> reading very quickly your changes:
> - The networking setup of the VM seems to be more complex than needed
> because you are relying on getting the IP address from DHCP. Would it
> be possible to configure libvirt with port forwarding, so that "ssh -p
> 2222 127.0.0.1" always work? (with raw QEMU, the command-line
> parameters would be something like -net
> nic,model=virtio,macaddr=52:54:00:12:34:56 -net
> user,hostfwd=tcp:127.0.0.1:2222-:22)

I would love something like that (no idea what I am doing),
but it doesn't get rid of polling for network. We have to wait
for the sshd to be ready.


> - Usually the helper scripts are located in directory scripts/, which
> make them more easily discovered by people reading the code (hidden
> files and directories seems more appropriate to store the
> configuration for different CI platforms that require them ; some
> projects choose to hide everything that is not useful for the build
> process but here it does seem to be so). Are there advantages of using
> .ci/ directory?

No benefit, just did something. They can live wherever....

> - Moreover having a "fedora-test.run" (or "fedora-test.sh", I am more
> used to .sh extension for shell scripts) enables to easily reproduce

and be named whatever....

> tests on a local development machine using "vagrant init
> fedora/32-cloud-base && vagrant ssh" + "sudo
> /vagrant/.../fedora-test.run". Such a command could be documented in
> the header of the shell script.

That sounds good, I like being able to do that. I have no idea what vagrant
is. You could say im ignorant to vargrant!

> - Right now, there are many things in .travis.yml. This file could be
> split in shell script(s) if that reduces duplication.

Yep I agree. Thats how I run all my other projects, I dislike large
travis.yml files,
but I don't want to take on that workload right now, ideally get this
working and then port the other stuff.

> - To verify the download of Fedora cloud image, instead of hard-coding
> its SHA256 digest, you could download the checksum file which is
> signed using GnuPG and check its signature, as described on
> https://alt.fedoraproject.org/cloud/

Yeah I saw that. Thats why I had that verify link in the TODO
comment:
https://alt.fedoraproject.org/en/verify.html

>
> Cheers,
> Nicolas

Thank you, appreciate your feedback

Re: Fedora VM On Travis for Testing

[Stephen Smalley]

On Thu, May 14, 2020 at 5:03 PM William Roberts
<bill.c.roberts@gmail.com> wrote:
>
> I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
> on Travis with execution happening in the Fedora VM.
> The Fedora VM contains the selinux source code for that particular
> travis build, so it has the PR patches, etc.
> The VM has networking.
>
> The build logs for the Travis run are here:
>   - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489
>
> Which comes from my git tree here:
>   - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing
>
> Note that it's super messy, I need to go through and cleanup both the
> patch and the git
> history. I also should verify the downloaded image is signed for the VM.
> Also note that I may rebase/squash my git branch at any time.
>
> Petr started a release document here:
>   - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md
>
> I'd like to gather feedback, and perhaps more comments on:
> 1. Is this CI approach worth continuing?
> 2. Comments on the CI scripts (I have no idea what I am doing, common issue)
> 3. Comments, patches, suggestions on what tests to run in this CI environment.
> 4. More information in in the RELEASE_PROCESS.md file. I made some
> comments there
>     as well on ideas.
>
> My goal here is that a release can occur if CI is passing without
> worry, and that
> we automate manual steps as much as possible. This way, if we get hit by a bus
> releases can occur without much effort.

I'm amazed travis-ci supports all of that (especially for free).
Hopefully you aren't violating their terms of use.
The biggest thing I'd add is to build and install the selinux
userspace in place of the stock Fedora versions (sudo make
LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel) and
then build and run the selinux-testsuite to exercise the SELinux
userspace and kernel runtime functionality.  Other things to possibly
add would be rebuilding upstream refpolicy (similar to its
.travis.yml) with the latest userspace, rebuilding and running setools
(but until we decouple it from libsepol internals this will
periodically break).

Re: Fedora VM On Travis for Testing

[William Roberts]

On Thu, May 14, 2020 at 5:21 PM Stephen Smalley
<stephen.smalley.work@gmail.com> wrote:
>
> On Thu, May 14, 2020 at 5:03 PM William Roberts
> <bill.c.roberts@gmail.com> wrote:
> >
> > I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
> > on Travis with execution happening in the Fedora VM.
> > The Fedora VM contains the selinux source code for that particular
> > travis build, so it has the PR patches, etc.
> > The VM has networking.
> >
> > The build logs for the Travis run are here:
> >   - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489
> >
> > Which comes from my git tree here:
> >   - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing
> >
> > Note that it's super messy, I need to go through and cleanup both the
> > patch and the git
> > history. I also should verify the downloaded image is signed for the VM.
> > Also note that I may rebase/squash my git branch at any time.
> >
> > Petr started a release document here:
> >   - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md
> >
> > I'd like to gather feedback, and perhaps more comments on:
> > 1. Is this CI approach worth continuing?
> > 2. Comments on the CI scripts (I have no idea what I am doing, common issue)
> > 3. Comments, patches, suggestions on what tests to run in this CI environment.
> > 4. More information in in the RELEASE_PROCESS.md file. I made some
> > comments there
> >     as well on ideas.
> >
> > My goal here is that a release can occur if CI is passing without
> > worry, and that
> > we automate manual steps as much as possible. This way, if we get hit by a bus
> > releases can occur without much effort.
>
> I'm amazed travis-ci supports all of that (especially for free).
> Hopefully you aren't violating their terms of use.

Not that I am aware of and I read their terms of use.
https://docs.travis-ci.com/legal/terms-of-service/

They don't seem to limit what you can do, it seems to be centered
around availability, liability and support.

I stumbled into other projects on github that were running virsh commands

> The biggest thing I'd add is to build and install the selinux
> userspace in place of the stock Fedora versions (sudo make
> LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel) and
> then build and run the selinux-testsuite to exercise the SELinux
> userspace and kernel runtime functionality.  Other things to possibly

Perfect, that's the info I was looking for.

> add would be rebuilding upstream refpolicy (similar to its
> .travis.yml) with the latest userspace, rebuilding and running setools
> (but until we decouple it from libsepol internals this will
> periodically break).

If it breaks, it breaks, just means we need to merge something or actually
fix it properly IIUC.

Re: Fedora VM On Travis for Testing

[William Roberts]

On Thu, May 14, 2020 at 5:31 PM William Roberts
<bill.c.roberts@gmail.com> wrote:
>
> On Thu, May 14, 2020 at 5:21 PM Stephen Smalley
> <stephen.smalley.work@gmail.com> wrote:
> >
> > On Thu, May 14, 2020 at 5:03 PM William Roberts
> > <bill.c.roberts@gmail.com> wrote:
> > >
> > > I just wanted to lay out a demo of a Fedora 32 cloud VM image booting
> > > on Travis with execution happening in the Fedora VM.
> > > The Fedora VM contains the selinux source code for that particular
> > > travis build, so it has the PR patches, etc.
> > > The VM has networking.
> > >
> > > The build logs for the Travis run are here:
> > >   - https://travis-ci.org/github/williamcroberts/selinux/builds/687185489
> > >
> > > Which comes from my git tree here:
> > >   - https://github.com/williamcroberts/selinux/tree/kvm-fedora-testing
> > >
> > > Note that it's super messy, I need to go through and cleanup both the
> > > patch and the git
> > > history. I also should verify the downloaded image is signed for the VM.
> > > Also note that I may rebase/squash my git branch at any time.
> > >
> > > Petr started a release document here:
> > >   - https://github.com/bachradsusi/SELinuxProject-selinux/blob/RELEASE/RELEASE_PROCESS.md
> > >
> > > I'd like to gather feedback, and perhaps more comments on:
> > > 1. Is this CI approach worth continuing?
> > > 2. Comments on the CI scripts (I have no idea what I am doing, common issue)
> > > 3. Comments, patches, suggestions on what tests to run in this CI environment.
> > > 4. More information in in the RELEASE_PROCESS.md file. I made some
> > > comments there
> > >     as well on ideas.
> > >
> > > My goal here is that a release can occur if CI is passing without
> > > worry, and that
> > > we automate manual steps as much as possible. This way, if we get hit by a bus
> > > releases can occur without much effort.
> >
> > I'm amazed travis-ci supports all of that (especially for free).
> > Hopefully you aren't violating their terms of use.
>
> Not that I am aware of and I read their terms of use.
> https://docs.travis-ci.com/legal/terms-of-service/
>
> They don't seem to limit what you can do, it seems to be centered
> around availability, liability and support.
>
> I stumbled into other projects on github that were running virsh commands
>
> > The biggest thing I'd add is to build and install the selinux
> > userspace in place of the stock Fedora versions (sudo make
> > LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel) and
> > then build and run the selinux-testsuite to exercise the SELinux
> > userspace and kernel runtime functionality.  Other things to possibly
>
> Perfect, that's the info I was looking for.
>
> > add would be rebuilding upstream refpolicy (similar to its
> > .travis.yml) with the latest userspace, rebuilding and running setools
> > (but until we decouple it from libsepol internals this will
> > periodically break).
>
> If it breaks, it breaks, just means we need to merge something or actually
> fix it properly IIUC.

So I just wanted to share, that the test suite is running with a PASS on
a travis CI instance with a nested KVM of Fedora32.

https://travis-ci.org/github/williamcroberts/selinux/builds/687592735

Next week I'll clean this up and make sure that its working consistently.
I've had 2 hangs when building the libselinux src, not sure why..
I don't want a CI check that'd flaky, so I want to do some more testing.

An example is on this build:
https://travis-ci.org/github/williamcroberts/selinux/builds/687496735

I'm gonna assume I need to allocate more memory and im gonna bump
the vcpu count as well to match the host. As Travis gives us two cores.

Bill