From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C9FC433E7 for ; Thu, 15 Oct 2020 16:09:48 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FC2421D7F for ; Thu, 15 Oct 2020 16:09:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=xenproject.org header.i=@xenproject.org header.b="6nBC+4Wr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FC2421D7F Authentication-Results: mail.kernel.org; dmarc=pass (p=none dis=none) header.from=xenproject.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.7549.19856 (Exim 4.92) (envelope-from ) id 1kT5ow-0000yr-8R; Thu, 15 Oct 2020 16:09:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 7549.19856; Thu, 15 Oct 2020 16:09:38 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kT5ow-0000yd-4E; Thu, 15 Oct 2020 16:09:38 +0000 Received: by outflank-mailman (input) for mailman id 7549; Thu, 15 Oct 2020 16:09:37 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kT5ov-0000xQ-2W for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:37 +0000 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e17d3cb8-6410-454c-a99e-ac964f8ccf48; Thu, 15 Oct 2020 16:09:36 +0000 (UTC) Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kT5ou-0000h7-7W for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:36 +0000 Received: from iwj (helo=mynotebook.example.org) by xenbits.xenproject.org with local-bsmtp (Exim 4.92) (envelope-from ) id 1kT5ou-0006gy-6g for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:36 +0000 Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk) by mariner.uk.xensource.com with esmtp (Exim 4.89) (envelope-from ) id 1kT5WP-0000oB-Uw; Thu, 15 Oct 2020 16:50:30 +0100 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kT5ov-0000xQ-2W for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:37 +0000 X-Inumbo-ID: e17d3cb8-6410-454c-a99e-ac964f8ccf48 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e17d3cb8-6410-454c-a99e-ac964f8ccf48; Thu, 15 Oct 2020 16:09:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xenproject.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=rKhUBECvH4UVMiFTJmADZZZYMWQOR+f/Us5E4XBbMw4=; b=6nBC+4WrrE9VyBaqiUBy++5BXv PMlM1TTv/orJDF/qEz40NHcR1E9G9v+iv8KKje/8Q/h/hhgArylaZlbZB5cmke+cpuFmzhf7CPJTm EWcp1onyN6frRapjHtWJcV7ZP1MK3pA4e1q/fhVyZgMDLpyexhB6PoKTC0hiLmuoWfRE=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kT5ou-0000h7-7W for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:36 +0000 Received: from iwj (helo=mynotebook.example.org) by xenbits.xenproject.org with local-bsmtp (Exim 4.92) (envelope-from ) id 1kT5ou-0006gy-6g for xen-devel@lists.xenproject.org; Thu, 15 Oct 2020 16:09:36 +0000 Received: from [172.18.45.5] (helo=zealot.relativity.greenend.org.uk) by mariner.uk.xensource.com with esmtp (Exim 4.89) (envelope-from ) id 1kT5WP-0000oB-Uw; Thu, 15 Oct 2020 16:50:30 +0100 From: Ian Jackson To: xen-devel@lists.xenproject.org Cc: Ian Jackson , Ian Jackson Subject: [OSSTEST PATCH v2 13/17] cr-daily-branch: Immediately retry failing tests Date: Thu, 15 Oct 2020 16:50:15 +0100 Message-Id: <20201015155019.20705-14-iwj@xenproject.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201015155019.20705-1-iwj@xenproject.org> References: <20201015155019.20705-1-iwj@xenproject.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Ian Jackson We exclude the self-tests because we don't want to miss breakage, and the Xen smoke tests because they will be run again RSN anyway. Signed-off-by: Ian Jackson --- cr-daily-branch | 48 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/cr-daily-branch b/cr-daily-branch index 285ea361..bea8734e 100755 --- a/cr-daily-branch +++ b/cr-daily-branch @@ -472,12 +472,58 @@ esac sgr_args+=" $EXTRA_SGR_ARGS" date >&2 +original_start=`date +%s` + : $flight $branch $OSSTEST_BLESSING $sgr_args eval "$DAILY_BRANCH_PREEXEC_HOOK" execute_flight $flight $OSSTEST_BLESSING date >&2 -start_email $flight $branch "$sgr_args" "$subject_prefix" +default_immediate_retry=$wantpush + +case "$branch" in +*smoke*) default_immediate_retry=false ;; +osstest) default_immediate_retry=false ;; +*) ;; +esac + +: ${OSSTEST_IMMEDIATE_RETRY:-$default_immediate_retry} + +while true; do + start_email $flight $branch "$sgr_args" "$subject_prefix" + if grep '^tolerable$' $mrof >/dev/null 2>&1; then break; fi + if ! $OSSTEST_IMMEDIATE_RETRY; then break; fi + OSSTEST_IMMEDIATE_RETRY=false + retry_jobs=$( + perl <$mrof -wne ' + next unless m/^regression (\S+) /; + my $j = $1; + next if $j =~ m/^build/; + $r{$j}++; + END { + print "copy-jobs '$flight' $_ " + foreach sort keys %r; + }' + ) + if [ "x$retry_jobs" = x ]; then break; fi + + rflight=$( + ./cs-adjust-flight new:$OSSTEST_BLESSING \ + branch-set $branch \ + $retry_jobs + ) + + ./mg-adjust-flight-makexrefs -v $rflight \ + --branch=$branch --revision-osstest=$narness_rev \ + '^build-*' --debug --blessings=real + + export OSSTEST_RESOURCE_WAITSTART=$original_start + execute_flight $rflight $OSSTEST_BLESSING-retest + report_flight $rflight + publish_logs $rflight + + sgr_args+=" --refer-to-flight=$rflight" +done push=false if grep '^tolerable$' $mrof >/dev/null 2>&1; then push=$wantpush; fi -- 2.20.1