[PATCH 0/2] codeql: add support for Xen and tools

[PATCH 0/2] codeql: add support for Xen and tools

[Roger Pau Monne]

Hello,

The following series add support for Xen and tools to be analyzed with
CodeQL using a github workflow. The result of such analysis ends up in
the "Security" github tab.

Currently we perform 3 different analyses for C, Python and Go code.

Roger Pau Monne (2):
  codeql: add support for analyzing C, Python and Go
  codeql: ignore Kconfig

 .github/codeql/codeql-config.yml |  2 ++
 .github/workflows/codeql.yml     | 60 ++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
 create mode 100644 .github/codeql/codeql-config.yml
 create mode 100644 .github/workflows/codeql.yml

-- 
2.34.1

[PATCH 1/2] codeql: add support for analyzing C, Python and Go

[Roger Pau Monne]

Introduce CodeQL support for Xen and analyze the C, Python and Go
files.

Note than when analyzing Python or Go we avoid building the hypervisor
and only build the tools.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
TBD: there's no limit in the number of scans here unlike Coverity, but
each takes github minutes and we are limited to 2000 per month IIRC.
We might want to not perform a scan for each push.

TBD: should we also disable the shim build? I'm not sure there's much
value in analyzing it.
---
 .github/workflows/codeql.yml | 59 ++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 .github/workflows/codeql.yml

diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
new file mode 100644
index 0000000000..5bfe478983
--- /dev/null
+++ b/.github/workflows/codeql.yml
@@ -0,0 +1,59 @@
+name: CodeQL
+
+on:
+  workflow_dispatch:
+  push:
+    branches: [staging]
+  schedule:
+    - cron: '18 10 * * WED,SUN' # Bi-weekly at 10:18 UTC
+
+jobs:
+  analyse:
+
+    strategy:
+      matrix:
+        language: [ 'cpp', 'python', 'go' ]
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Install build dependencies
+      run: |
+        sudo apt-get install -y wget git \
+          libbz2-dev build-essential \
+          zlib1g-dev libncurses5-dev iasl \
+          libbz2-dev e2fslibs-dev uuid-dev libyajl-dev \
+          autoconf libtool liblzma-dev \
+          python3-dev golang python-dev libsystemd-dev
+
+    - uses: actions/checkout@v2
+      with:
+        ref: staging
+
+    - name: Configure Xen
+      run: |
+        ./configure --with-system-qemu=/bin/true \
+                    --with-system-seabios=/bin/true \
+                    --with-system-ovmf=/bin/true
+
+    - name: Pre build stuff
+      run: |
+        make -j`nproc` mini-os-dir
+
+    - uses: github/codeql-action/init@v1
+      with:
+        languages: ${{matrix.language}}
+        queries: security-and-quality
+
+    - if: matrix.language == 'cpp'
+      name: Full Build
+      run: |
+        make -j`nproc` build-xen build-tools
+        make -j`nproc` -C extras/mini-os/
+
+    - if: matrix.language == 'python' || matrix.language == 'go'
+      name: Tools Build
+      run: |
+        make -j`nproc` build-tools
+
+    - uses: github/codeql-action/analyze@v1
-- 
2.34.1

[PATCH 2/2] codeql: ignore Kconfig

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 .github/codeql/codeql-config.yml | 2 ++
 .github/workflows/codeql.yml     | 1 +
 2 files changed, 3 insertions(+)
 create mode 100644 .github/codeql/codeql-config.yml

diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml
new file mode 100644
index 0000000000..3648e8e39b
--- /dev/null
+++ b/.github/codeql/codeql-config.yml
@@ -0,0 +1,2 @@
+paths-ignore:
+  - xen/tools/kconfig
diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
index 5bfe478983..2e2dfeaee1 100644
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@@ -42,6 +42,7 @@ jobs:
 
     - uses: github/codeql-action/init@v1
       with:
+        config-file: ./.github/codeql/codeql-config.yml
         languages: ${{matrix.language}}
         queries: security-and-quality
 
-- 
2.34.1

Re: [PATCH 1/2] codeql: add support for analyzing C, Python and Go

[Andrew Cooper]

On 25/02/2022 15:19, Roger Pau Monne wrote:
> Introduce CodeQL support for Xen and analyze the C, Python and Go
> files.
>
> Note than when analyzing Python or Go we avoid building the hypervisor
> and only build the tools.
>
> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> TBD: there's no limit in the number of scans here unlike Coverity, but
> each takes github minutes and we are limited to 2000 per month IIRC.
> We might want to not perform a scan for each push.

We don't push very often, and github is slower at noticing anyway, so I
think we ought to be fine.

We can always revisit the decision if we do end up hitting limits.

> TBD: should we also disable the shim build? I'm not sure there's much
> value in analyzing it.

Shim's logic is quite different in areas.  I'd say it's worth keeping.

> ---
>  .github/workflows/codeql.yml | 59 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 59 insertions(+)
>  create mode 100644 .github/workflows/codeql.yml
>
> diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
> new file mode 100644
> index 0000000000..5bfe478983
> --- /dev/null
> +++ b/.github/workflows/codeql.yml
> @@ -0,0 +1,59 @@
> +name: CodeQL

As a thought... As we're considering doing cross-arm checks, should we
use an x86 suffix here ?

> +
> +on:
> +  workflow_dispatch:
> +  push:
> +    branches: [staging]
> +  schedule:
> +    - cron: '18 10 * * WED,SUN' # Bi-weekly at 10:18 UTC
> +
> +jobs:
> +  analyse:
> +
> +    strategy:
> +      matrix:
> +        language: [ 'cpp', 'python', 'go' ]
> +
> +    runs-on: ubuntu-latest
> +
> +    steps:
> +    - name: Install build dependencies
> +      run: |
> +        sudo apt-get install -y wget git \
> +          libbz2-dev build-essential \
> +          zlib1g-dev libncurses5-dev iasl \
> +          libbz2-dev e2fslibs-dev uuid-dev libyajl-dev \
> +          autoconf libtool liblzma-dev \
> +          python3-dev golang python-dev libsystemd-dev
> +
> +    - uses: actions/checkout@v2
> +      with:
> +        ref: staging

This one doesn't want to be forced to staging.  The github integration
does work properly with different branches.

Importantly, it will allow anyone with a fork of Xen to use the
workflow_dispatch UI to do a one-off run on any branch.

~Andrew

Re: [PATCH 2/2] codeql: ignore Kconfig

[Andrew Cooper]

On 25/02/2022 15:19, Roger Pau Monne wrote:
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

I agree with this, but it looks like it wants to be folded into the
previous patch.

~Andrew

Re: [PATCH 1/2] codeql: add support for analyzing C, Python and Go

[Roger Pau Monné]

On Fri, Feb 25, 2022 at 03:38:42PM +0000, Andrew Cooper wrote:
> On 25/02/2022 15:19, Roger Pau Monne wrote:
> > Introduce CodeQL support for Xen and analyze the C, Python and Go
> > files.
> >
> > Note than when analyzing Python or Go we avoid building the hypervisor
> > and only build the tools.
> >
> > Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> > TBD: there's no limit in the number of scans here unlike Coverity, but
> > each takes github minutes and we are limited to 2000 per month IIRC.
> > We might want to not perform a scan for each push.
> 
> We don't push very often, and github is slower at noticing anyway, so I
> think we ought to be fine.
> 
> We can always revisit the decision if we do end up hitting limits.
> 
> > TBD: should we also disable the shim build? I'm not sure there's much
> > value in analyzing it.
> 
> Shim's logic is quite different in areas.  I'd say it's worth keeping.

Ack to both.

> > ---
> >  .github/workflows/codeql.yml | 59 ++++++++++++++++++++++++++++++++++++
> >  1 file changed, 59 insertions(+)
> >  create mode 100644 .github/workflows/codeql.yml
> >
> > diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
> > new file mode 100644
> > index 0000000000..5bfe478983
> > --- /dev/null
> > +++ b/.github/workflows/codeql.yml
> > @@ -0,0 +1,59 @@
> > +name: CodeQL
> 
> As a thought... As we're considering doing cross-arm checks, should we
> use an x86 suffix here ?

It would be nice to handle all arches in the same file, but the
divergences could be bigger than the shared parts, since setting up
all the Arm cross-build env is likely not trivial.

I also wonder how to tag x86 vs Arm databases, but I guess we will
find a way. Maybe using the 'category' field for the analyze action?

> > +
> > +on:
> > +  workflow_dispatch:
> > +  push:
> > +    branches: [staging]
> > +  schedule:
> > +    - cron: '18 10 * * WED,SUN' # Bi-weekly at 10:18 UTC
> > +
> > +jobs:
> > +  analyse:
> > +
> > +    strategy:
> > +      matrix:
> > +        language: [ 'cpp', 'python', 'go' ]
> > +
> > +    runs-on: ubuntu-latest
> > +
> > +    steps:
> > +    - name: Install build dependencies
> > +      run: |
> > +        sudo apt-get install -y wget git \
> > +          libbz2-dev build-essential \
> > +          zlib1g-dev libncurses5-dev iasl \
> > +          libbz2-dev e2fslibs-dev uuid-dev libyajl-dev \
> > +          autoconf libtool liblzma-dev \
> > +          python3-dev golang python-dev libsystemd-dev
> > +
> > +    - uses: actions/checkout@v2
> > +      with:
> > +        ref: staging
> 
> This one doesn't want to be forced to staging.  The github integration
> does work properly with different branches.

Don't we need this so the `schedule` triggered run picks the staging
branch instead of master?

Thanks, Roger.

Re: [PATCH 2/2] codeql: ignore Kconfig

[Roger Pau Monné]

On Fri, Feb 25, 2022 at 03:39:22PM +0000, Andrew Cooper wrote:
> On 25/02/2022 15:19, Roger Pau Monne wrote:
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> I agree with this, but it looks like it wants to be folded into the
> previous patch.

Can do. We also need to add
tools/firmware/xen-dir/xen-root/xen/tools/kconfig/ in order to prevent
Kconfig from the shim also showing up.

Thanks, Roger.