Re: [kvm-unit-tests PATCH v4 11/13] scripts/runtime: skip tests that cannot run

From: Andrew Jones <drjones@redhat.com>
To: "Radim Krčmář" <rkrcmar@redhat.com>
Cc: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [kvm-unit-tests PATCH v4 11/13] scripts/runtime: skip tests that cannot run
Date: Wed, 11 May 2016 19:20:42 +0200	[thread overview]
Message-ID: <20160511172042.k63gudr6u4rj73fc@hawk.localdomain> (raw)
In-Reply-To: <1462984243-5783-2-git-send-email-rkrcmar@redhat.com>

On Wed, May 11, 2016 at 06:30:41PM +0200, Radim Krčmář wrote:
> A case where QEMU won't run the kernel should be considered as skipped.
> Hyper-V tests used to FAIL on old QEMUs.  The infamous QEMU=/dev/null
> FAIL streak is covered too.  Some error messages in arch/run were
> reworded, because they are now visible in summary.
> 
> Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
> ---
>  v4:
>  * clarified "kvm only" error message [Drew+Lukáš]
>  * used _NO_FILE_4Uhere_ instead of /dev/null [Drew]
>  * completely reworked output, now prints the last line
> 
>  arm/run              |  2 +-
>  powerpc/run          |  2 +-
>  scripts/runtime.bash | 28 +++++++++++++++++++++++++++-
>  3 files changed, 29 insertions(+), 3 deletions(-)

Reviewed-by: Andrew Jones <drjones@redhat.com>

> 
> diff --git a/arm/run b/arm/run
> index ebf703d5757c..a2f35ef6a7e6 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -19,7 +19,7 @@ if [ -c /dev/kvm ]; then
>  fi
>  
>  if [ "$ACCEL" = "kvm" ] && [ "$kvm_available" != "yes" ]; then
> -	echo "skip $TESTNAME (kvm only)"
> +	echo "KVM is needed, but not available on this host"
>  	exit 2
>  fi
>  
> diff --git a/powerpc/run b/powerpc/run
> index 56698e622592..14988a75b1e2 100755
> --- a/powerpc/run
> +++ b/powerpc/run
> @@ -16,7 +16,7 @@ if [ -c /dev/kvm ]; then
>  fi
>  
>  if [ "$ACCEL" = "kvm" ] && [ "$kvm_available" != "yes" ]; then
> -	echo "skip $TESTNAME (kvm only)"
> +	echo "KVM is needed, but not available on this host"
>  	exit 2
>  fi
>  
> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> index deae077e50a1..c9ab9ba47d81 100644
> --- a/scripts/runtime.bash
> +++ b/scripts/runtime.bash
> @@ -11,6 +11,27 @@ extract_summary()
>      tail -1 | grep '^SUMMARY: ' | sed 's/^SUMMARY: /(/;s/$/)/'
>  }
>  
> +# We assume that QEMU is going to work if it tried to load the kernel
> +premature_failure()
> +{
> +    local log="$(eval $(get_cmdline _NO_FILE_4Uhere_) 2>&1)"
> +    local last_line=$(tail -1 <<< "$log")
> +
> +    echo "$last_line" | grep -qi "could not load kernel" &&
> +        return 1
> +
> +    RUNTIME_log_stderr <<< "$log"
> +
> +    echo "$last_line"
> +    return 0
> +}
> +
> +get_cmdline()
> +{
> +    local kernel=$1
> +    echo "TESTNAME=$testname TIMEOUT=$timeout ACCEL=$accel $RUNTIME_arch_run $kernel -smp $smp $opts"
> +}
> +
>  function run()
>  {
>      local testname="$1"
> @@ -48,7 +69,12 @@ function run()
>          fi
>      done
>  
> -    cmdline="TESTNAME=$testname TIMEOUT=$timeout ACCEL=$accel $RUNTIME_arch_run $kernel -smp $smp $opts"
> +    last_line=$(premature_failure) && {
> +        echo "`SKIP` $1 ($last_line)"
> +        return 77
> +    }
> +
> +    cmdline=$(get_cmdline $kernel)
>      if [ "$verbose" = "yes" ]; then
>          echo $cmdline
>      fi
> -- 
> 2.8.2
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html