[PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Nicolas Iooss]

Use the Vagrant configuration in scripts/ci in a workflow for GitHub
Actions.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
---
Example of run: https://github.com/fishilico/selinux/runs/1627586446

 .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 .github/workflows/vm_testsuite.yml

diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
new file mode 100644
index 000000000000..af2fad1e73f5
--- /dev/null
+++ b/.github/workflows/vm_testsuite.yml
@@ -0,0 +1,22 @@
+name: Run SELinux testsuite in a virtual machine
+
+on: [push, pull_request]
+
+jobs:
+  vm_testsuite:
+
+    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
+    runs-on: macos-latest
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Create Vagrant VM
+      run: |
+        cd scripts/ci
+        vagrant up
+
+    - name: Run SELinux testsuite in the VM
+      run: |
+        cd scripts/ci
+        vagrant ssh -- ./run-selinux-test.sh
-- 
2.29.2

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Nicolas Iooss]

Right after sending the patch, I realized there could be some
confusion about this patch: it is for SELinux userspace repository
(https://github.com/SELinuxProject/selinux).

On Wed, Dec 30, 2020 at 9:09 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>
> Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> Actions.
>
> Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> ---
> Example of run: https://github.com/fishilico/selinux/runs/1627586446
>
>  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>  create mode 100644 .github/workflows/vm_testsuite.yml
>
> diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> new file mode 100644
> index 000000000000..af2fad1e73f5
> --- /dev/null
> +++ b/.github/workflows/vm_testsuite.yml
> @@ -0,0 +1,22 @@
> +name: Run SELinux testsuite in a virtual machine
> +
> +on: [push, pull_request]
> +
> +jobs:
> +  vm_testsuite:
> +
> +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> +    runs-on: macos-latest
> +
> +    steps:
> +    - uses: actions/checkout@v2
> +
> +    - name: Create Vagrant VM
> +      run: |
> +        cd scripts/ci
> +        vagrant up
> +
> +    - name: Run SELinux testsuite in the VM
> +      run: |
> +        cd scripts/ci
> +        vagrant ssh -- ./run-selinux-test.sh
> --
> 2.29.2
>

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[William Roberts]

On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>
> Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> Actions.
>
> Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> ---
> Example of run: https://github.com/fishilico/selinux/runs/1627586446
>
>  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>  create mode 100644 .github/workflows/vm_testsuite.yml
>
> diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> new file mode 100644
> index 000000000000..af2fad1e73f5
> --- /dev/null
> +++ b/.github/workflows/vm_testsuite.yml
> @@ -0,0 +1,22 @@
> +name: Run SELinux testsuite in a virtual machine
> +
> +on: [push, pull_request]
> +
> +jobs:
> +  vm_testsuite:
> +
> +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> +    runs-on: macos-latest

Good find, they support nested virtualization.

> +
> +    steps:
> +    - uses: actions/checkout@v2
> +
> +    - name: Create Vagrant VM
> +      run: |
> +        cd scripts/ci
> +        vagrant up
> +
> +    - name: Run SELinux testsuite in the VM
> +      run: |
> +        cd scripts/ci
> +        vagrant ssh -- ./run-selinux-test.sh
> --
> 2.29.2
>

LGTM ack. Do you have any fork that has a build with this just to see
it running?

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Nicolas Iooss]

On Thu, Dec 31, 2020 at 3:43 PM William Roberts
<bill.c.roberts@gmail.com> wrote:
>
> On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> >
> > Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> > Actions.
> >
> > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> > ---
> > Example of run: https://github.com/fishilico/selinux/runs/1627586446
> >
> >  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
> >  1 file changed, 22 insertions(+)
> >  create mode 100644 .github/workflows/vm_testsuite.yml
> >
> > diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> > new file mode 100644
> > index 000000000000..af2fad1e73f5
> > --- /dev/null
> > +++ b/.github/workflows/vm_testsuite.yml
> > @@ -0,0 +1,22 @@
> > +name: Run SELinux testsuite in a virtual machine
> > +
> > +on: [push, pull_request]
> > +
> > +jobs:
> > +  vm_testsuite:
> > +
> > +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> > +    runs-on: macos-latest
>
> Good find, they support nested virtualization.
>
> > +
> > +    steps:
> > +    - uses: actions/checkout@v2
> > +
> > +    - name: Create Vagrant VM
> > +      run: |
> > +        cd scripts/ci
> > +        vagrant up
> > +
> > +    - name: Run SELinux testsuite in the VM
> > +      run: |
> > +        cd scripts/ci
> > +        vagrant ssh -- ./run-selinux-test.sh
> > --
> > 2.29.2
> >
>
> LGTM ack. Do you have any fork that has a build with this just to see
> it running?

Yes, I have been testing it on
https://github.com/fishilico/selinux/actions, with several successful
builds such as https://github.com/fishilico/selinux/runs/1627586446.

Thanks,
Nicolas

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Nicolas Iooss]

On Sat, Jan 2, 2021 at 11:35 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>
> On Thu, Dec 31, 2020 at 3:43 PM William Roberts
> <bill.c.roberts@gmail.com> wrote:
> >
> > On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > >
> > > Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> > > Actions.
> > >
> > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> > > ---
> > > Example of run: https://github.com/fishilico/selinux/runs/1627586446
> > >
> > >  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
> > >  1 file changed, 22 insertions(+)
> > >  create mode 100644 .github/workflows/vm_testsuite.yml
> > >
> > > diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> > > new file mode 100644
> > > index 000000000000..af2fad1e73f5
> > > --- /dev/null
> > > +++ b/.github/workflows/vm_testsuite.yml
> > > @@ -0,0 +1,22 @@
> > > +name: Run SELinux testsuite in a virtual machine
> > > +
> > > +on: [push, pull_request]
> > > +
> > > +jobs:
> > > +  vm_testsuite:
> > > +
> > > +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> > > +    runs-on: macos-latest
> >
> > Good find, they support nested virtualization.
> >
> > > +
> > > +    steps:
> > > +    - uses: actions/checkout@v2
> > > +
> > > +    - name: Create Vagrant VM
> > > +      run: |
> > > +        cd scripts/ci
> > > +        vagrant up
> > > +
> > > +    - name: Run SELinux testsuite in the VM
> > > +      run: |
> > > +        cd scripts/ci
> > > +        vagrant ssh -- ./run-selinux-test.sh
> > > --
> > > 2.29.2
> > >
> >
> > LGTM ack. Do you have any fork that has a build with this just to see
> > it running?
>
> Yes, I have been testing it on
> https://github.com/fishilico/selinux/actions, with several successful
> builds such as https://github.com/fishilico/selinux/runs/1627586446.
>
> Thanks,
> Nicolas

For information, I have been experiencing some failures due to timeout
issues while Github Actions creates the Vagrant environment, for
example in https://github.com/fishilico/selinux/runs/1652713066. I am
currently trying to run test jobs with "config.vm.boot_timeout = 600"
(the default value is 300 seconds) in scripts/ci/Vagrantfile and would
like to wait a little bit before merging this patch, to see whether
this fixes my issue. I do not want to introduce instability in our CI
pipeline.

Nicolas

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Ondrej Mosnacek]

On Wed, Jan 6, 2021 at 9:19 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>
> On Sat, Jan 2, 2021 at 11:35 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> >
> > On Thu, Dec 31, 2020 at 3:43 PM William Roberts
> > <bill.c.roberts@gmail.com> wrote:
> > >
> > > On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > > >
> > > > Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> > > > Actions.
> > > >
> > > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> > > > ---
> > > > Example of run: https://github.com/fishilico/selinux/runs/1627586446
> > > >
> > > >  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
> > > >  1 file changed, 22 insertions(+)
> > > >  create mode 100644 .github/workflows/vm_testsuite.yml
> > > >
> > > > diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> > > > new file mode 100644
> > > > index 000000000000..af2fad1e73f5
> > > > --- /dev/null
> > > > +++ b/.github/workflows/vm_testsuite.yml
> > > > @@ -0,0 +1,22 @@
> > > > +name: Run SELinux testsuite in a virtual machine
> > > > +
> > > > +on: [push, pull_request]
> > > > +
> > > > +jobs:
> > > > +  vm_testsuite:
> > > > +
> > > > +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> > > > +    runs-on: macos-latest
> > >
> > > Good find, they support nested virtualization.
> > >
> > > > +
> > > > +    steps:
> > > > +    - uses: actions/checkout@v2
> > > > +
> > > > +    - name: Create Vagrant VM
> > > > +      run: |
> > > > +        cd scripts/ci
> > > > +        vagrant up
> > > > +
> > > > +    - name: Run SELinux testsuite in the VM
> > > > +      run: |
> > > > +        cd scripts/ci
> > > > +        vagrant ssh -- ./run-selinux-test.sh
> > > > --
> > > > 2.29.2
> > > >
> > >
> > > LGTM ack. Do you have any fork that has a build with this just to see
> > > it running?
> >
> > Yes, I have been testing it on
> > https://github.com/fishilico/selinux/actions, with several successful
> > builds such as https://github.com/fishilico/selinux/runs/1627586446.
> >
> > Thanks,
> > Nicolas
>
> For information, I have been experiencing some failures due to timeout
> issues while Github Actions creates the Vagrant environment, for
> example in https://github.com/fishilico/selinux/runs/1652713066. I am
> currently trying to run test jobs with "config.vm.boot_timeout = 600"
> (the default value is 300 seconds) in scripts/ci/Vagrantfile and would
> like to wait a little bit before merging this patch, to see whether
> this fixes my issue. I do not want to introduce instability in our CI
> pipeline.

Hi Nicholas, any update here? I have drafted a similar patch for
selinux-testsuite's own CI, but have been waiting on your findings
regarding the timeouts to avoid needless followups.

Thanks a lot for finding a way to make this work with GH Actions!

--
Ondrej Mosnacek
Software Engineer, Platform Security - SELinux kernel
Red Hat, Inc.

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Nicolas Iooss]

On Tue, Jan 19, 2021 at 3:32 PM Ondrej Mosnacek <omosnace@redhat.com> wrote:
>
> On Wed, Jan 6, 2021 at 9:19 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> >
> > On Sat, Jan 2, 2021 at 11:35 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > >
> > > On Thu, Dec 31, 2020 at 3:43 PM William Roberts
> > > <bill.c.roberts@gmail.com> wrote:
> > > >
> > > > On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > > > >
> > > > > Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> > > > > Actions.
> > > > >
> > > > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> > > > > ---
> > > > > Example of run: https://github.com/fishilico/selinux/runs/1627586446
> > > > >
> > > > >  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
> > > > >  1 file changed, 22 insertions(+)
> > > > >  create mode 100644 .github/workflows/vm_testsuite.yml
> > > > >
> > > > > diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> > > > > new file mode 100644
> > > > > index 000000000000..af2fad1e73f5
> > > > > --- /dev/null
> > > > > +++ b/.github/workflows/vm_testsuite.yml
> > > > > @@ -0,0 +1,22 @@
> > > > > +name: Run SELinux testsuite in a virtual machine
> > > > > +
> > > > > +on: [push, pull_request]
> > > > > +
> > > > > +jobs:
> > > > > +  vm_testsuite:
> > > > > +
> > > > > +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> > > > > +    runs-on: macos-latest
> > > >
> > > > Good find, they support nested virtualization.
> > > >
> > > > > +
> > > > > +    steps:
> > > > > +    - uses: actions/checkout@v2
> > > > > +
> > > > > +    - name: Create Vagrant VM
> > > > > +      run: |
> > > > > +        cd scripts/ci
> > > > > +        vagrant up
> > > > > +
> > > > > +    - name: Run SELinux testsuite in the VM
> > > > > +      run: |
> > > > > +        cd scripts/ci
> > > > > +        vagrant ssh -- ./run-selinux-test.sh
> > > > > --
> > > > > 2.29.2
> > > > >
> > > >
> > > > LGTM ack. Do you have any fork that has a build with this just to see
> > > > it running?
> > >
> > > Yes, I have been testing it on
> > > https://github.com/fishilico/selinux/actions, with several successful
> > > builds such as https://github.com/fishilico/selinux/runs/1627586446.
> > >
> > > Thanks,
> > > Nicolas
> >
> > For information, I have been experiencing some failures due to timeout
> > issues while Github Actions creates the Vagrant environment, for
> > example in https://github.com/fishilico/selinux/runs/1652713066. I am
> > currently trying to run test jobs with "config.vm.boot_timeout = 600"
> > (the default value is 300 seconds) in scripts/ci/Vagrantfile and would
> > like to wait a little bit before merging this patch, to see whether
> > this fixes my issue. I do not want to introduce instability in our CI
> > pipeline.
>
> Hi Nicholas, any update here? I have drafted a similar patch for
> selinux-testsuite's own CI, but have been waiting on your findings
> regarding the timeouts to avoid needless followups.
>
> Thanks a lot for finding a way to make this work with GH Actions!

Hi,
Sorry for the delay. I have been quite busy in the past two weeks, but
I am catching up on the latest patches and messages.

I am still experiencing occasional boot timeouts (for example in
https://github.com/fishilico/selinux/runs/1667663667), despite using
"config.vm.boot_timeout = 600" in Vagrantfile. But when this occurs,
restarting the GitHub Actions job works. So this is quite annoying but
I believe we can live with it (this means that someone will need to
click on "Re-run all jobs" on GitHub when this happens). So I merged
my patch, and if it becomes too annoying, this workflow can be
disabled in the future.

Nicolas

Re: [PATCH 1/1] GitHub Actions: run SELinux testsuite in Fedora virtual machine

[Ondrej Mosnacek]

On Wed, Jan 20, 2021 at 5:07 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> On Tue, Jan 19, 2021 at 3:32 PM Ondrej Mosnacek <omosnace@redhat.com> wrote:
> >
> > On Wed, Jan 6, 2021 at 9:19 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > >
> > > On Sat, Jan 2, 2021 at 11:35 AM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > > >
> > > > On Thu, Dec 31, 2020 at 3:43 PM William Roberts
> > > > <bill.c.roberts@gmail.com> wrote:
> > > > >
> > > > > On Wed, Dec 30, 2020 at 2:12 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
> > > > > >
> > > > > > Use the Vagrant configuration in scripts/ci in a workflow for GitHub
> > > > > > Actions.
> > > > > >
> > > > > > Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
> > > > > > ---
> > > > > > Example of run: https://github.com/fishilico/selinux/runs/1627586446
> > > > > >
> > > > > >  .github/workflows/vm_testsuite.yml | 22 ++++++++++++++++++++++
> > > > > >  1 file changed, 22 insertions(+)
> > > > > >  create mode 100644 .github/workflows/vm_testsuite.yml
> > > > > >
> > > > > > diff --git a/.github/workflows/vm_testsuite.yml b/.github/workflows/vm_testsuite.yml
> > > > > > new file mode 100644
> > > > > > index 000000000000..af2fad1e73f5
> > > > > > --- /dev/null
> > > > > > +++ b/.github/workflows/vm_testsuite.yml
> > > > > > @@ -0,0 +1,22 @@
> > > > > > +name: Run SELinux testsuite in a virtual machine
> > > > > > +
> > > > > > +on: [push, pull_request]
> > > > > > +
> > > > > > +jobs:
> > > > > > +  vm_testsuite:
> > > > > > +
> > > > > > +    # Use VirtualBox+vagrant on macOS, as described in https://github.com/actions/virtual-environments/issues/433
> > > > > > +    runs-on: macos-latest
> > > > >
> > > > > Good find, they support nested virtualization.
> > > > >
> > > > > > +
> > > > > > +    steps:
> > > > > > +    - uses: actions/checkout@v2
> > > > > > +
> > > > > > +    - name: Create Vagrant VM
> > > > > > +      run: |
> > > > > > +        cd scripts/ci
> > > > > > +        vagrant up
> > > > > > +
> > > > > > +    - name: Run SELinux testsuite in the VM
> > > > > > +      run: |
> > > > > > +        cd scripts/ci
> > > > > > +        vagrant ssh -- ./run-selinux-test.sh
> > > > > > --
> > > > > > 2.29.2
> > > > > >
> > > > >
> > > > > LGTM ack. Do you have any fork that has a build with this just to see
> > > > > it running?
> > > >
> > > > Yes, I have been testing it on
> > > > https://github.com/fishilico/selinux/actions, with several successful
> > > > builds such as https://github.com/fishilico/selinux/runs/1627586446.
> > > >
> > > > Thanks,
> > > > Nicolas
> > >
> > > For information, I have been experiencing some failures due to timeout
> > > issues while Github Actions creates the Vagrant environment, for
> > > example in https://github.com/fishilico/selinux/runs/1652713066. I am
> > > currently trying to run test jobs with "config.vm.boot_timeout = 600"
> > > (the default value is 300 seconds) in scripts/ci/Vagrantfile and would
> > > like to wait a little bit before merging this patch, to see whether
> > > this fixes my issue. I do not want to introduce instability in our CI
> > > pipeline.
> >
> > Hi Nicholas, any update here? I have drafted a similar patch for
> > selinux-testsuite's own CI, but have been waiting on your findings
> > regarding the timeouts to avoid needless followups.
> >
> > Thanks a lot for finding a way to make this work with GH Actions!
>
> Hi,
> Sorry for the delay. I have been quite busy in the past two weeks, but
> I am catching up on the latest patches and messages.
>
> I am still experiencing occasional boot timeouts (for example in
> https://github.com/fishilico/selinux/runs/1667663667), despite using
> "config.vm.boot_timeout = 600" in Vagrantfile. But when this occurs,
> restarting the GitHub Actions job works. So this is quite annoying but
> I believe we can live with it (this means that someone will need to
> click on "Re-run all jobs" on GitHub when this happens). So I merged
> my patch, and if it becomes too annoying, this workflow can be
> disabled in the future.

Ok, thank you for the update. I'll post my testsuite CI patch then.

--
Ondrej Mosnacek
Software Engineer, Platform Security - SELinux kernel
Red Hat, Inc.