Re: [PATCH kvm-unit-tests 0/6] Improve the output of test runners

From: Andrew Jones <drjones@redhat.com>
To: "Radim Krčmář" <rkrcmar@redhat.com>
Cc: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH kvm-unit-tests 0/6] Improve the output of test runners
Date: Mon, 14 Dec 2015 16:20:00 -0600	[thread overview]
Message-ID: <20151214222000.GG3633@hawk.localdomain> (raw)
In-Reply-To: <1450128261-21170-1-git-send-email-rkrcmar@redhat.com>

On Mon, Dec 14, 2015 at 10:24:15PM +0100, Radim Krčmář wrote:
> This series is a mix of patches that change the output of run_tests.sh
> and x86-run.  The output of ./run_tests.sh now looks like this:
> 
> > PASS apic (14 tests, 0 unexpected failures)
> > PASS ioapic (19 tests, 0 unexpected failures)
> > PASS smptest (1 tests, 0 unexpected failures)
> > PASS smptest3 (1 tests, 0 unexpected failures)
> > PASS vmexit_cpuid 
> > PASS vmexit_vmcall 
> > PASS vmexit_mov_from_cr8 
> > PASS vmexit_mov_to_cr8 
> > PASS vmexit_inl_pmtimer 
> > PASS vmexit_ipi 
> > PASS vmexit_ipi_halt 
> > PASS vmexit_ple_round_robin 
> > PASS access 
> > skip smap (0 tests, 0 unexpected failures)
> > skip pku (0 tests, 0 unexpected failures)
> > PASS emulator (132 tests, 0 unexpected failures, 1 skipped)
> > PASS eventinj (13 tests, 0 unexpected failures)
> > PASS hypercall (2 tests, 0 unexpected failures)
> > PASS idt_test (4 tests, 0 unexpected failures)
> > PASS msr (13 tests, 0 unexpected failures)
> > PASS pmu (67 tests, 0 unexpected failures, 1 expected failures)
> > PASS port80 
> > PASS realmode 
> > PASS s3 
> > PASS sieve 
> > PASS tsc (3 tests, 0 unexpected failures)
> > PASS tsc_adjust (5 tests, 0 unexpected failures)
> > PASS xsave (17 tests, 0 unexpected failures)
> > PASS rmap_chain 
> > skip svm (0 tests, 0 unexpected failures)
> > skip svm-disabled (0 tests, 0 unexpected failures)
> > skip taskswitch (i386 only)
> > skip taskswitch2 (i386 only)
> > PASS kvmclock_test 
> > PASS pcid (3 tests, 0 unexpected failures)
> > skip vmx (0 tests, 0 unexpected failures)
> > PASS debug (7 tests, 0 unexpected failures)
> > qemu-kvm: Property '.hv-synic' not found
> > skip hyperv_synic (failed $(echo quit | $qemu -enable-kvm -cpu kvm64,hv_synic -device hyperv-testdev -monitor stdio > /dev/null))

I'm not sure I like the (summary) addition. A summary of why we skip
would be useful, like the (i386 only) stuff, but otherwise it doesn't
seem necessary, and the "(failed $(echo quit | $qemu -enable-kvm -cpu
kvm64,hv_synic -device hyperv-testdev -monitor stdio > /dev/null))"
summary is a bit ugly, wrapping on many terminals.

Another comment that is series wide is that all these changes need to be
tested with 'make standalone' (and many of your patches will require
changes to scripts/mkstandalone.sh, for which you will never forgive
me for having written :-)

Thanks,
drew