From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: [PATCH 2/3] Add a weekly coverity flight Date: Wed, 3 Feb 2016 09:46:15 +0000 Message-ID: <1454492776-23788-2-git-send-email-ian.campbell@citrix.com> References: <1454492776-23788-1-git-send-email-ian.campbell@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1454492776-23788-1-git-send-email-ian.campbell@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: ian.jackson@eu.citrix.com, xen-devel@lists.xen.org Cc: Andrew Cooper , Ian Campbell List-Id: xen-devel@lists.xenproject.org This primarily consists of ts-coverity-scan and make-coverity-flight which constructs the sole job. The most recently scanned revision is pushed to a new coverity-scanned branch in the usual xen.git, tests are run on the master branch. For the cr-* integration we treat branch=coverity as a special case of tree=xen. I didn't think tree=coverity made much sense, and would probably reach tendrils into lots of other places (such as the invocations of check_tested). I initially thoughts that $c{CoverityEmail} would need to be an actual account registered with scan, however a manual experiment using email=security@xen.org was accepted by the service. An "analysis complete" message was sent to security@ while individual results mails were sent to each member of the coverity project who was configured to receive them. I think this is what we want. The "analysis complete" mail contained no sensitive data, but also no real information other than "success" (or presumably "failure" if that were to be the case). I think going to security@ is probably OK. I have run this in non-uploading mode on the production infra and then run the curl manually, adjusting the CLI until it works and updated the script to match. I have not yet run in uploading mode but will do so once another upload is allowed by the service, In my experiments the curl command took ~35 minutes to complete (rate in the 100-200k range). Not sure if this is a problem. Note that curl is run on the controller (via system_checked) and consequently has no timeout etc. Note that the token must be supplied with Cc: Andrew Cooper --- v2: - Split move of collect_xen_built_versions() into separate patch - Implemented support for coverity_upload = true (but don't yet set it) - Add host_hostflags to the job so it can actually run somewhere. - Call tsreadconfig() before referencing $r{coverity_upload} so that $r is actually populated. - use token=. + + +set -e -o posix + +branch=$1 +xenbranch=$2 +blessing=$3 +buildflight=$4 + +flight=`./cs-flight-create $blessing $branch` + +. ./cri-common +. ./ap-common +. ./mfi-common + +defsuite=`getconfig DebianSuite` + +arch=amd64 +suite=$defsuite + +build_hostflags=share-build-$suite-$arch,arch-$arch,suite-$suite,purpose-build + +./cs-job-create $flight coverity-$arch coverity \ + arch=$arch host_hostflags=$build_hostflags \ + tree_xen=$TREE_COVERITY \ + revision_xen=$REVISION_COVERITY \ + coverity_upload=false + +echo $flight + +# Local variables: +# mode: sh +# sh-basic-offset: 2 +# indent-tabs-mode: nil +# End: diff --git a/production-config b/production-config index f2f0584..e67a253 100644 --- a/production-config +++ b/production-config @@ -100,6 +100,10 @@ TftpGrubVersion XXXX-XX-XX XenUsePath /usr/groups/xencore/systems/bin/xenuse XenUseUser osstest +# Results might include potential vulnerabilities. +CoverityEmail security@xen.org +CoverityTools cov-analysis-linux64-7.7.0.4.tar.gz + # We use the IP address because Citrix can't manage reliable nameservice #DebianMirrorHost debian.uk.xensource.com #DebianMirrorHost 10.80.16.196 diff --git a/sg-run-job b/sg-run-job index 20ebb64..7e592dd 100755 --- a/sg-run-job +++ b/sg-run-job @@ -445,6 +445,11 @@ proc prepare-build-host {} { run-ts . host-build-prep ts-xen-build-prep } +proc need-hosts/coverity {} { return BUILD } +proc run-job/coverity {} { + run-ts . = ts-coverity-scan + host +} + #---------- main program ---------- jobdb::set-flight diff --git a/ts-coverity-scan b/ts-coverity-scan new file mode 100755 index 0000000..f3cc497 --- /dev/null +++ b/ts-coverity-scan @@ -0,0 +1,110 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2015 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +use strict qw(vars); +use DBI; +use Osstest; +use File::Path; +use POSIX; +use Osstest::TestSupport; +use Osstest::BuildSupport; + +tsreadconfig(); +selectbuildhost(\@ARGV); +# remaining arguments are passed as targets to "make" +builddirsprops(); + +# Require explicit opt in from flight construction +my $coverity_upload = ($r{coverity_upload}//'false') =~ m/true/ ? 1 : 0; + +# This must contain exactly and only the token, for example there must +# be no trailing "\n", otherwise it is included in the literal token, +# which is then invalid. +my $tokenfile = "$ENV{HOME}/.xen-osstest/coverity-secret"; +my $submit_url = "https://scan.coverity.com/builds?project=XenProject"; + +sub checkout () { + prepbuilddirs(); + + build_clone($ho, 'xen', $builddir, 'xen'); +} + +sub covtools () { + target_putfile($ho, 100, "$c{Images}/$c{CoverityTools}", "$builddir/covtools.tar.gz"); + target_cmd($ho, <{Suite} =~ m/wheezy|squeeze|lenny/) { push(@packages, "libnl-dev"); -- 2.6.1