* [PATCH 2/3] Add a weekly coverity flight
2016-02-03 9:46 [PATCH 1/3] Move collectversions from ts-xen-build into Osstest::BuildSupport Ian Campbell
@ 2016-02-03 9:46 ` Ian Campbell
2016-02-03 9:46 ` [PATCH 3/3] make-coverity-flight: set coverity_upload=true Ian Campbell
` (2 more replies)
2016-02-03 9:59 ` [PATCH 1/3] Move collectversions from ts-xen-build into Osstest::BuildSupport Ian Campbell
2016-02-03 12:02 ` Ian Jackson
2 siblings, 3 replies; 11+ messages in thread
From: Ian Campbell @ 2016-02-03 9:46 UTC (permalink / raw)
To: ian.jackson, xen-devel; +Cc: Andrew Cooper, Ian Campbell
This primarily consists of ts-coverity-scan and make-coverity-flight
which constructs the sole job.
The most recently scanned revision is pushed to a new coverity-scanned
branch in the usual xen.git, tests are run on the master branch.
For the cr-* integration we treat branch=coverity as a special case of
tree=xen. I didn't think tree=coverity made much sense, and would
probably reach tendrils into lots of other places (such as the
invocations of check_tested).
I initially thoughts that $c{CoverityEmail} would need to be an actual
account registered with scan, however a manual experiment using
email=security@xen.org was accepted by the service. An "analysis
complete" message was sent to security@ while individual results mails
were sent to each member of the coverity project who was configured to
receive them. I think this is what we want. The "analysis complete"
mail contained no sensitive data, but also no real information other
than "success" (or presumably "failure" if that were to be the case).
I think going to security@ is probably OK.
I have run this in non-uploading mode on the production infra and then
run the curl manually, adjusting the CLI until it works and updated
the script to match. I have not yet run in uploading mode but will do
so once another upload is allowed by the service,
In my experiments the curl command took ~35 minutes to complete (rate
in the 100-200k range). Not sure if this is a problem. Note that curl
is run on the controller (via system_checked) and consequently has no
timeout etc.
Note that the token must be supplied with </path/to/token and not
@/path/to/token. The latter appears to the server as a file upload
rather than a text field in a form which doesn't work. In early
attempts I thought that the trailing \n in /path/to/token might be an
issue and hence wrote a big comment. However having discovered < vs @
I am no longer 100% sure that is the case, but I left the comment
anyway since I can observe on the wire that the \n is included in the
upload (but each test takes ~35 mins and there is a ratelimit on the
server side too).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
v2:
- Split move of collect_xen_built_versions() into separate patch
- Implemented support for coverity_upload = true (but don't yet set
it)
- Add host_hostflags to the job so it can actually run somewhere.
- Call tsreadconfig() before referencing $r{coverity_upload} so that
$r is actually populated.
- use token=</path/to/token not token=@/path/to/token in curl
arguments.
- CoverityEmail == security@xen.org seems to work fine.
- Add $flight to description.
Deployment notes:
- Put cov-analysis-linux64-7.7.0.4.tar.gz in the Images
directory. DONE in COLO
- Populate $HOME/.xen-osstest/coverity-secret with the token
DONE in COLO
- Populate xen.git#coverity-scanned with an initial baseline, update
ap-fetch-version-old to refer to it instead of master.
---
ap-fetch-version | 3 ++
ap-fetch-version-old | 4 ++
ap-print-url | 2 +-
ap-push | 4 ++
cr-daily-branch | 18 ++++++++-
cri-common | 1 +
crontab | 1 +
make-coverity-flight | 52 ++++++++++++++++++++++++
production-config | 4 ++
sg-run-job | 5 +++
ts-coverity-scan | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++
ts-xen-build-prep | 2 +-
12 files changed, 203 insertions(+), 3 deletions(-)
create mode 100755 make-coverity-flight
create mode 100755 ts-coverity-scan
diff --git a/ap-fetch-version b/ap-fetch-version
index a7b658b..1e48b45 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -53,6 +53,9 @@ xen-4.*-testing)
repo_tree_rev_fetch_git xen \
$TREE_XEN staging-$branchcore $LOCALREV_XEN
;;
+coverity)
+ repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN
+ ;;
qemu-mainline)
repo_tree_rev_fetch_git $branch \
$TREE_QEMU_MAINLINE master $LOCALREV_QEMU_UPSTREAM
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index e2c6b3b..9d6190f 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -59,6 +59,10 @@ xen-4.*-testing)
repo_tree_rev_fetch_git xen \
$TREE_XEN stable-$branchcore $LOCALREV_XEN
;;
+coverity)
+ #XXX doesn't exist yet, use master for now repo_tree_rev_fetch_git xen $TREE_XEN coverity-scanned $LOCALREV_XEN
+ repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN
+ ;;
qemu-mainline)
repo_tree_rev_fetch_git $branch \
$BASE_TREE_QEMU_UPSTREAM upstream-tested $LOCALREV_QEMU_UPSTREAM
diff --git a/ap-print-url b/ap-print-url
index 4088852..6ca000d 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -31,7 +31,7 @@ if info_linux_tree "$branch"; then
fi
case "$branch" in
-xen-*)
+xen-*|coverity)
echo $TREE_XEN
;;
qemu-mainline)
diff --git a/ap-push b/ap-push
index 8def652..97510c3 100755
--- a/ap-push
+++ b/ap-push
@@ -68,6 +68,10 @@ xen-*-testing)
xenversion=${xenversion#xen-}
git push $TREE_XEN $revision:refs/heads/stable-$xenversion
;;
+coverity)
+ cd $repos/xen
+ git push $TREE_XEN $revision:refs/heads/coverity-scanned
+ ;;
qemu-mainline)
cd $repos/qemu-mainline
git push $TREE_QEMU_UPSTREAM $revision:refs/heads/upstream-tested
diff --git a/cr-daily-branch b/cr-daily-branch
index 364238c..9594e18 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -205,7 +205,22 @@ fi
case "$tree" in
xen)
realtree=$xenbranch
- NEW_REVISION=$REVISION_XEN
+
+ case $branch in
+ coverity)
+ if [ "x$TREE_COVERITY" = x ]; then
+ export TREE_COVERITY=$TREE_XEN
+ fi
+ if [ "x$REVISION_COVERITY" = x ]; then
+ determine_version REVISION_COVERITY coverity COVERITY
+ export REVISION_COVERITY
+ fi
+ NEW_REVISION=$REVISION_COVERITY
+ ;;
+ *)
+ NEW_REVISION=$REVISION_XEN
+ ;;
+ esac
;;
linux)
realtree=linux
@@ -259,6 +274,7 @@ fi
case $branch in
distros-*) makeflight=./make-distros-flight ;;
+coverity) makeflight=./make-coverity-flight ;;
*) makeflight=./make-flight ;;
esac
diff --git a/cri-common b/cri-common
index 6dfe8df..9f8bb0b 100644
--- a/cri-common
+++ b/cri-common
@@ -68,6 +68,7 @@ select_xenbranch () {
case "$branch" in
xen-unstable-smoke) tree=xen; xenbranch=$branch; qemuubranch=qemu-upstream-unstable;;
xen-*) tree=xen; xenbranch=$branch ;;
+ coverity) tree=xen; xenbranch=xen-unstable ;;
qemu-mainline) tree=qemuu; xenbranch=xen-unstable; qemuubranch=qemu-mainline;;
qemu-upstream-*) tree=qemuu; xenbranch=xen-${branch#qemu-upstream-};;
linux) tree=linux; xenbranch=xen-unstable ;;
diff --git a/crontab b/crontab
index 09b8d14..b6ced25 100755
--- a/crontab
+++ b/crontab
@@ -8,6 +8,7 @@ MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
0 * * * * cd testing.git && BRANCHES=xen-unstable-smoke ./cr-for-branches branches -q "./cr-daily-branch --real"
4-59/30 * * * * cd testing.git && ./cr-for-branches branches -q "./cr-daily-branch --real"
18 9 * * 1,3,5 cd testing.git && BRANCHES=linux-next ./cr-for-branches branches -w "./cr-daily-branch --real"
+18 9 * * 7 cd testing.git && BRANCHES=coverity ./cr-for-branches branches -w "./cr-daily-branch --real"
18 4 * * * cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
6-59/15 * * * * cd testing.git && EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w "./cr-try-bisect --real"
#8-59/5 * * * * cd bisects/adhoc.git && with-lock-ex -q data-tree-lock bash -c "./cr-try-bisect-adhoc; exit $?"
diff --git a/make-coverity-flight b/make-coverity-flight
new file mode 100755
index 0000000..dafb61c
--- /dev/null
+++ b/make-coverity-flight
@@ -0,0 +1,52 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e -o posix
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ./cri-common
+. ./ap-common
+. ./mfi-common
+
+defsuite=`getconfig DebianSuite`
+
+arch=amd64
+suite=$defsuite
+
+build_hostflags=share-build-$suite-$arch,arch-$arch,suite-$suite,purpose-build
+
+./cs-job-create $flight coverity-$arch coverity \
+ arch=$arch host_hostflags=$build_hostflags \
+ tree_xen=$TREE_COVERITY \
+ revision_xen=$REVISION_COVERITY \
+ coverity_upload=false
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
diff --git a/production-config b/production-config
index f2f0584..e67a253 100644
--- a/production-config
+++ b/production-config
@@ -100,6 +100,10 @@ TftpGrubVersion XXXX-XX-XX
XenUsePath /usr/groups/xencore/systems/bin/xenuse
XenUseUser osstest
+# Results might include potential vulnerabilities.
+CoverityEmail security@xen.org
+CoverityTools cov-analysis-linux64-7.7.0.4.tar.gz
+
# We use the IP address because Citrix can't manage reliable nameservice
#DebianMirrorHost debian.uk.xensource.com
#DebianMirrorHost 10.80.16.196
diff --git a/sg-run-job b/sg-run-job
index 20ebb64..7e592dd 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -445,6 +445,11 @@ proc prepare-build-host {} {
run-ts . host-build-prep ts-xen-build-prep
}
+proc need-hosts/coverity {} { return BUILD }
+proc run-job/coverity {} {
+ run-ts . = ts-coverity-scan + host
+}
+
#---------- main program ----------
jobdb::set-flight
diff --git a/ts-coverity-scan b/ts-coverity-scan
new file mode 100755
index 0000000..f3cc497
--- /dev/null
+++ b/ts-coverity-scan
@@ -0,0 +1,110 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+# remaining arguments are passed as targets to "make"
+builddirsprops();
+
+# Require explicit opt in from flight construction
+my $coverity_upload = ($r{coverity_upload}//'false') =~ m/true/ ? 1 : 0;
+
+# This must contain exactly and only the token, for example there must
+# be no trailing "\n", otherwise it is included in the literal token,
+# which is then invalid.
+my $tokenfile = "$ENV{HOME}/.xen-osstest/coverity-secret";
+my $submit_url = "https://scan.coverity.com/builds?project=XenProject";
+
+sub checkout () {
+ prepbuilddirs();
+
+ build_clone($ho, 'xen', $builddir, 'xen');
+}
+
+sub covtools () {
+ target_putfile($ho, 100, "$c{Images}/$c{CoverityTools}", "$builddir/covtools.tar.gz");
+ target_cmd($ho, <<END, 100);
+set -xe
+c=$builddir/covtools
+mkdir -p \$c
+cd \$c
+tar --strip-components=1 -xaf $builddir/covtools.tar.gz
+END
+}
+
+sub build () {
+ my $make = "make $makeflags";
+
+ # Pre build things we don't want coverity to scan, but which are
+ # normally built by some other command.
+ target_cmd_build($ho, 1000, $builddir, <<END);
+cd $builddir/xen
+./configure
+$make -C tools/firmware/etherboot all
+$make mini-os-dir
+END
+
+ # Now the stuff we want coverity to look at
+ target_cmd_build($ho, 9000, $builddir, <<END);
+cd $builddir/xen
+export PATH=$builddir/covtools/bin:\$PATH
+cov-build --dir cov-int $make -C extras/mini-os/
+cov-build --dir cov-int $make xen tools
+
+tar czvf xen-coverity.tgz cov-int
+END
+
+ built_stash_file($ho, $builddir,
+ "xen-coverity.tgz", "xen/xen-coverity.tgz", 0);
+}
+
+sub upload() {
+ my $xen_version = target_cmd_output($ho, <<END, 30);
+ cd $builddir/xen
+ make xenversion
+END
+
+ my @form_args;
+ push @form_args, "token=\<$tokenfile";
+ push @form_args, "email=$c{CoverityEmail}";
+ push @form_args, "file=\@$stash/build/xen-coverity.tgz";
+ push @form_args, "version=$xen_version";
+ push @form_args, "description=$flight: $r{tree_xen} $r{built_revision_xen}";
+
+ my @args = map { ("--form", $_) } @form_args;
+ push @args, $submit_url;
+
+ if ($coverity_upload) {
+ system_checked("curl", @args);
+ } else {
+ logm("Not uploading: curl args: ".(join " ", map { qq("$_") } @args));
+ }
+}
+
+checkout();
+covtools();
+build();
+collect_xen_built_versions();
+upload();
diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index b35e91b..c2383db 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -206,7 +206,7 @@ sub prep () {
autoconf automake libtool xsltproc
libxml2-utils libxml2-dev
libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
- ccache nasm checkpolicy ebtables);
+ ccache nasm checkpolicy ebtables curl);
if ($ho->{Suite} =~ m/wheezy|squeeze|lenny/) {
push(@packages, "libnl-dev");
--
2.6.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH 1/3] Move collectversions from ts-xen-build into Osstest::BuildSupport
2016-02-03 9:46 [PATCH 1/3] Move collectversions from ts-xen-build into Osstest::BuildSupport Ian Campbell
2016-02-03 9:46 ` [PATCH 2/3] Add a weekly coverity flight Ian Campbell
@ 2016-02-03 9:59 ` Ian Campbell
2016-02-03 12:02 ` Ian Jackson
2 siblings, 0 replies; 11+ messages in thread
From: Ian Campbell @ 2016-02-03 9:59 UTC (permalink / raw)
To: ian.jackson, xen-devel
On Wed, 2016-02-03 at 09:46 +0000, Ian Campbell wrote:
These 3 should all have been tagged "OSSTEST v2", sorry for the omission.
> I'm going to have a need for it elsewhere.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> Osstest/BuildSupport.pm | 12 ++++++++++++
> ts-xen-build | 13 +------------
> 2 files changed, 13 insertions(+), 12 deletions(-)
>
> diff --git a/Osstest/BuildSupport.pm b/Osstest/BuildSupport.pm
> index 933f6e1..a183546 100644
> --- a/Osstest/BuildSupport.pm
> +++ b/Osstest/BuildSupport.pm
> @@ -42,6 +42,7 @@ BEGIN {
>
> xendist
> $xendist
> + collect_xen_built_versions
>
> submodulefixup submodule_have submodule_find
>
> @@ -84,6 +85,17 @@ sub xendist () {
> ($ho, 'xendist', '', $r{"buildjob"});
> }
>
> +sub collect_xen_built_versions () {
> + my $tools="$builddir/xen/tools";
> + my $extras="$builddir/xen/extras";
> + store_revision($ho, 'qemu', "$tools/ioemu-dir", 1);
> + store_revision($ho, 'qemu', "$tools/qemu-xen-traditional-dir", 1);
> + store_revision($ho, 'qemuu', "$tools/qemu-xen-dir", 1);
> + store_revision($ho, 'seabios', "$tools/firmware/seabios-dir", 1);
> + store_revision($ho, 'ovmf', "$tools/firmware/ovmf-dir", 1);
> + store_revision($ho, 'minios', "$extras/mini-os", 1);
> +}
> +
> #----- submodules -----
>
> sub submodulefixup ($$$$) {
> diff --git a/ts-xen-build b/ts-xen-build
> index 8f92729..382fe62 100755
> --- a/ts-xen-build
> +++ b/ts-xen-build
> @@ -151,17 +151,6 @@ END
> }
> }
>
> -sub collectversions () {
> - my $tools="$builddir/xen/tools";
> - my $extras="$builddir/xen/extras";
> - store_revision($ho, 'qemu', "$tools/ioemu-dir", 1);
> - store_revision($ho, 'qemu', "$tools/qemu-xen-traditional-dir", 1);
> - store_revision($ho, 'qemuu', "$tools/qemu-xen-dir", 1);
> - store_revision($ho, 'seabios', "$tools/firmware/seabios-dir", 1);
> - store_revision($ho, 'ovmf', "$tools/firmware/ovmf-dir", 1);
> - store_revision($ho, 'minios', "$extras/mini-os", 1);
> -}
> -
> sub divide () {
> # Only move hv to xeninstall, so that we can have
> # xenpolicy in tools tarball.
> @@ -248,7 +237,7 @@ sub trapping ($) {
> checkout();
>
> trapping(\&build);
> -trapping(\&collectversions);
> +trapping(\&collect_xen_built_versions);
>
> die "*** something failed:\n\n".(join "\n\n",@probs)."\n** something
> failed"
> if @probs;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread