* [PATCH] sparse: Reduce 'test-suite' run output
@ 2011-08-26 13:50 Pekka Enberg
2011-08-27 2:11 ` Christopher Li
0 siblings, 1 reply; 6+ messages in thread
From: Pekka Enberg @ 2011-08-26 13:50 UTC (permalink / raw)
To: linux-sparse; +Cc: Pekka Enberg, Christopher Li, Linus Torvalds
There's more than 100 tests in the validation test suite. That makes it
cumbersome to notice test breakage in because the results don't fit on single
page.
We don't expect to see most tests to pass so it's much more convenient to
only be verbose for failing test cases. This patch changes the
'test-suite' output to report progress on a single line unless a test case
fails.
Current master 'make check' output looks as follows:
Makefile:53: Your system does not have libxml, disabling c2xml
TEST preprocessor/preprocessor14.c ( Preprocessor #14)
error: actual output text does not match expected output text.
error: see preprocessor/preprocessor14.c.output.* for further investigation.
--- preprocessor/preprocessor14.c.output.expected 2011-08-26 05:47:22.000000000 +0000
+++ preprocessor/preprocessor14.c.output.got 2011-08-26 05:47:22.000000000 +0000
@@ -1,2 +1,2 @@
-
+,
info: test 'preprocessor/preprocessor14.c' is known to fail
TEST static-forward-decl.c ( static forward declaration)
error: actual error text does not match expected error text.
error: see static-forward-decl.c.error.* for further investigation.
--- static-forward-decl.c.error.expected 2011-08-26 05:47:22.000000000 +0000
+++ static-forward-decl.c.error.got 2011-08-26 05:47:22.000000000 +0000
@@ -0,0 +1 @@
+static-forward-decl.c:3:5: warning: symbol 'f' was not declared. Should it be static?
info: test 'static-forward-decl.c' is known to fail
TEST varargs1.c ( Varargs bogus warning regression test #1)
Out of 102 tests, 100 passed, 2 failed (2 of them are known to fail)
Cc: Christopher Li <sparse@chrisli.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
---
validation/test-suite | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 3c011c6..513d785 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -95,7 +95,7 @@ do_test()
fi
test_name=$last_result
- echo " TEST $test_name ($file)"
+ printf "\r TEST %-80s" "$file ($test_name)"
# does the test provide a specific command ?
cmd=`eval echo $default_path/$default_cmd`
@@ -128,6 +128,7 @@ do_test()
for stream in output error; do
diff -u "$file".$stream.expected "$file".$stream.got > "$file".$stream.diff
if [ "$?" -ne "0" ]; then
+ echo ""
error "actual $stream text does not match expected $stream text."
error "see $file.$stream.* for further investigation."
cat "$file".$stream.diff
@@ -163,6 +164,7 @@ do_test_suite()
# prints some numbers
tests_nr=`expr $ok_tests + $ko_tests`
+ echo ""
echo -n "Out of $tests_nr tests, $ok_tests passed, $ko_tests failed"
echo " ($known_ko_tests of them are known to fail)"
if [ "$unhandled_tests" -ne "0" ]; then
--
1.7.4.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH] sparse: Reduce 'test-suite' run output
2011-08-26 13:50 [PATCH] sparse: Reduce 'test-suite' run output Pekka Enberg
@ 2011-08-27 2:11 ` Christopher Li
2011-08-27 8:13 ` Pekka Enberg
0 siblings, 1 reply; 6+ messages in thread
From: Christopher Li @ 2011-08-27 2:11 UTC (permalink / raw)
To: Pekka Enberg; +Cc: linux-sparse, Linus Torvalds
On Fri, Aug 26, 2011 at 6:50 AM, Pekka Enberg <penberg@kernel.org> wrote:
> There's more than 100 tests in the validation test suite. That makes it
> cumbersome to notice test breakage in because the results don't fit on single
> page.
Looks good in a terminal. But if the output is piped to a file, it will be hard
to read that file. We shouldn't do it for a non-terminal output.
> + echo ""
Why not just "echo" instead. Without any argument, echo will just output "\n".
Chris
--
To unsubscribe from this list: send the line "unsubscribe linux-sparse" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sparse: Reduce 'test-suite' run output
2011-08-27 2:11 ` Christopher Li
@ 2011-08-27 8:13 ` Pekka Enberg
2011-08-27 11:39 ` Jonathan Neuschäfer
2011-08-27 15:15 ` Ben Pfaff
0 siblings, 2 replies; 6+ messages in thread
From: Pekka Enberg @ 2011-08-27 8:13 UTC (permalink / raw)
To: Christopher Li; +Cc: linux-sparse, Linus Torvalds
[-- Attachment #1: Type: TEXT/PLAIN, Size: 696 bytes --]
On Fri, 26 Aug 2011, Christopher Li wrote:
> On Fri, Aug 26, 2011 at 6:50 AM, Pekka Enberg <penberg@kernel.org> wrote:
>> There's more than 100 tests in the validation test suite. That makes it
>> cumbersome to notice test breakage in because the results don't fit on single
>> page.
>
> Looks good in a terminal. But if the output is piped to a file, it will be hard
> to read that file. We shouldn't do it for a non-terminal output.
Right. How can I check if we're outputting to non-terminal?
>> + echo ""
>
> Why not just "echo" instead. Without any argument, echo will just output "\n".
No reason, I'm just really bad at writing shell scripts. :-)
I'll fix that up!
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sparse: Reduce 'test-suite' run output
2011-08-27 8:13 ` Pekka Enberg
@ 2011-08-27 11:39 ` Jonathan Neuschäfer
2011-08-27 15:15 ` Ben Pfaff
1 sibling, 0 replies; 6+ messages in thread
From: Jonathan Neuschäfer @ 2011-08-27 11:39 UTC (permalink / raw)
To: Pekka Enberg; +Cc: Christopher Li, linux-sparse, Linus Torvalds
On Sat, Aug 27, 2011 at 11:13:17AM +0300, Pekka Enberg wrote:
> On Fri, 26 Aug 2011, Christopher Li wrote:
>
> >On Fri, Aug 26, 2011 at 6:50 AM, Pekka Enberg <penberg@kernel.org> wrote:
> >>There's more than 100 tests in the validation test suite. That makes it
> >>cumbersome to notice test breakage in because the results don't fit on single
> >>page.
> >
> >Looks good in a terminal. But if the output is piped to a file, it will be hard
> >to read that file. We shouldn't do it for a non-terminal output.
>
> Right. How can I check if we're outputting to non-terminal?
You probably need a C helper, like this:
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
int main(int argc, char **argv)
{
int fd, tmp;
if (argc < 2) {
printf("usage: %s fd\n", argv[0]);
return EXIT_FAILURE;
}
errno = 0;
fd = (int) strtol(argv[1], NULL, 10);
if (errno == ERANGE) {
perror("bad fd");
return EXIT_FAILURE;
}
tmp = isatty(fd);
if (!tmp && errno == EBADF)
perror("");
return tmp ? EXIT_SUCCESS : EXIT_FAILURE;
}
HTH,
Jonathan Neuschäfer
--
To unsubscribe from this list: send the line "unsubscribe linux-sparse" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] sparse: Reduce 'test-suite' run output
2011-08-27 8:13 ` Pekka Enberg
2011-08-27 11:39 ` Jonathan Neuschäfer
@ 2011-08-27 15:15 ` Ben Pfaff
2011-08-27 18:17 ` Jonathan Neuschäfer
1 sibling, 1 reply; 6+ messages in thread
From: Ben Pfaff @ 2011-08-27 15:15 UTC (permalink / raw)
To: Pekka Enberg; +Cc: Christopher Li, linux-sparse, Linus Torvalds
Pekka Enberg <penberg@kernel.org> writes:
> On Fri, 26 Aug 2011, Christopher Li wrote:
>
>> On Fri, Aug 26, 2011 at 6:50 AM, Pekka Enberg <penberg@kernel.org> wrote:
>>> There's more than 100 tests in the validation test suite. That makes it
>>> cumbersome to notice test breakage in because the results don't fit on single
>>> page.
>>
>> Looks good in a terminal. But if the output is piped to a file, it will be hard
>> to read that file. We shouldn't do it for a non-terminal output.
>
> Right. How can I check if we're outputting to non-terminal?
test -t 1
--
"To the engineer, the world is a toy box full of sub-optimized and
feature-poor toys."
--Scott Adams
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sparse: Reduce 'test-suite' run output
2011-08-27 15:15 ` Ben Pfaff
@ 2011-08-27 18:17 ` Jonathan Neuschäfer
0 siblings, 0 replies; 6+ messages in thread
From: Jonathan Neuschäfer @ 2011-08-27 18:17 UTC (permalink / raw)
To: blp; +Cc: Pekka Enberg, Christopher Li, linux-sparse, Linus Torvalds
On Sat, Aug 27, 2011 at 08:15:44AM -0700, Ben Pfaff wrote:
> Pekka Enberg <penberg@kernel.org> writes:
>
> > On Fri, 26 Aug 2011, Christopher Li wrote:
> >
> >> On Fri, Aug 26, 2011 at 6:50 AM, Pekka Enberg <penberg@kernel.org> wrote:
> >>> There's more than 100 tests in the validation test suite. That makes it
> >>> cumbersome to notice test breakage in because the results don't fit on single
> >>> page.
> >>
> >> Looks good in a terminal. But if the output is piped to a file, it will be hard
> >> to read that file. We shouldn't do it for a non-terminal output.
> >
> > Right. How can I check if we're outputting to non-terminal?
>
> test -t 1
Okay, that's way better than my approach.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-08-27 18:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-08-26 13:50 [PATCH] sparse: Reduce 'test-suite' run output Pekka Enberg
2011-08-27 2:11 ` Christopher Li
2011-08-27 8:13 ` Pekka Enberg
2011-08-27 11:39 ` Jonathan Neuschäfer
2011-08-27 15:15 ` Ben Pfaff
2011-08-27 18:17 ` Jonathan Neuschäfer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.