* [LTP] [PATCH 0/2] Add support for a test tags @ 2019-10-14 11:25 Cyril Hrubis 2019-10-14 11:25 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis 2019-10-14 11:25 ` [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags Cyril Hrubis 0 siblings, 2 replies; 16+ messages in thread From: Cyril Hrubis @ 2019-10-14 11:25 UTC (permalink / raw) To: ltp This is a newest version of the test tags, first patch adds a support for test tags to a test library, the second one annotates tests with tags. Not everything is annotated at the moment but fair amount of tests is included. These patches are also a stepping stone for the test metadata project that will allow us to do much more than this. You can have a look at: https://github.com/metan-ucw/ltp/tree/master/docparse Also if you run a make in the docparse directory it will generate test metadata along with a single html page with nice test metadata description. While the whole project is work-in-progress, the test tags are finished and may add a value even without the rest of the changes, see the first commit for details. Cyril Hrubis (2): lib: Add support for test tags testcases/kernel: Add tags include/tst_test.h | 10 +++ lib/tst_test.c | 77 +++++++++++++++++-- testcases/cve/cve-2014-0196.c | 5 ++ testcases/cve/cve-2015-3290.c | 5 ++ testcases/cve/cve-2016-10044.c | 5 ++ testcases/cve/cve-2016-7042.c | 5 ++ testcases/cve/cve-2016-7117.c | 5 ++ testcases/cve/cve-2017-16939.c | 5 ++ testcases/cve/cve-2017-17052.c | 5 ++ testcases/cve/cve-2017-17053.c | 5 ++ testcases/cve/cve-2017-2618.c | 5 ++ testcases/cve/cve-2017-2671.c | 5 ++ testcases/cve/meltdown.c | 6 +- testcases/cve/stack_clash.c | 4 + testcases/kernel/crypto/af_alg01.c | 5 ++ testcases/kernel/crypto/af_alg02.c | 5 ++ testcases/kernel/crypto/af_alg03.c | 4 + testcases/kernel/crypto/af_alg04.c | 4 + testcases/kernel/crypto/af_alg05.c | 8 +- testcases/kernel/crypto/af_alg06.c | 4 + testcases/kernel/crypto/crypto_user01.c | 6 ++ testcases/kernel/crypto/crypto_user02.c | 4 + testcases/kernel/crypto/pcrypt_aead01.c | 5 ++ testcases/kernel/sound/snd_timer01.c | 6 ++ testcases/kernel/syscalls/accept/accept02.c | 5 ++ testcases/kernel/syscalls/acct/acct02.c | 4 + testcases/kernel/syscalls/add_key/add_key02.c | 5 ++ testcases/kernel/syscalls/bpf/bpf_prog02.c | 4 + .../syscalls/futex/futex_cmp_requeue02.c | 5 ++ testcases/kernel/syscalls/keyctl/keyctl02.c | 4 + testcases/kernel/syscalls/keyctl/keyctl04.c | 5 ++ testcases/kernel/syscalls/keyctl/keyctl05.c | 4 + .../kernel/syscalls/move_pages/move_pages12.c | 6 ++ .../kernel/syscalls/realpath/realpath01.c | 4 + .../syscalls/request_key/request_key05.c | 4 + testcases/kernel/syscalls/sbrk/sbrk03.c | 4 + testcases/kernel/syscalls/sendto/sendto02.c | 4 + .../kernel/syscalls/setrlimit/setrlimit06.c | 4 + 38 files changed, 251 insertions(+), 9 deletions(-) -- 2.21.0 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-14 11:25 [LTP] [PATCH 0/2] Add support for a test tags Cyril Hrubis @ 2019-10-14 11:25 ` Cyril Hrubis 2019-10-15 5:12 ` Li Wang 2019-10-16 13:30 ` Petr Vorel 2019-10-14 11:25 ` [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags Cyril Hrubis 1 sibling, 2 replies; 16+ messages in thread From: Cyril Hrubis @ 2019-10-14 11:25 UTC (permalink / raw) To: ltp The newly introduced test tags are generic name-value pairs that can hold test metadata, the intended use for now is to store kernel commit hashes for kernel reproducers as well as CVE ids. The mechanism is however choosen to be very generic so that it's easy to add basically any information later on. As it is the main purpose is to print hints for a test failures. If a test that has been written as a kernel reproducer fails it prints nice URL pointing to a kernel commit that may be missing. Example output: -------------------------------------------------------------------------- tst_test.c:1145: INFO: Timeout per run is 0h 05m 00s add_key02.c:98: FAIL: unexpected error with key type 'asymmetric': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.idmap': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.spnego': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'pkcs7_test': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc_s': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'user': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'logon': EINVAL HINT: This is a regression test for linux kernel, see commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c HINT: This test also tests for CVE-2017-15274, see: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15274 Summary: passed 0 failed 8 skipped 0 warnings 0 -------------------------------------------------------------------------- Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- include/tst_test.h | 10 ++++++ lib/tst_test.c | 77 +++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 80 insertions(+), 7 deletions(-) diff --git a/include/tst_test.h b/include/tst_test.h index 84acf2c59..4a51b6d16 100644 --- a/include/tst_test.h +++ b/include/tst_test.h @@ -108,6 +108,11 @@ int tst_parse_int(const char *str, int *val, int min, int max); int tst_parse_long(const char *str, long *val, long min, long max); int tst_parse_float(const char *str, float *val, float min, float max); +struct tst_tag { + const char *name; + const char *value; +}; + extern unsigned int tst_variant; struct tst_test { @@ -212,6 +217,11 @@ struct tst_test { * NULL-terminated array of capability settings */ struct tst_cap *caps; + + /* + * {NULL, NULL} terminated array of tags. + */ + const struct tst_tag *tags; }; /* diff --git a/lib/tst_test.c b/lib/tst_test.c index 6239acf89..2129f38cb 100644 --- a/lib/tst_test.c +++ b/lib/tst_test.c @@ -31,6 +31,9 @@ #include "old_device.h" #include "old_tmpdir.h" +#define LINUX_GIT_URL "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=" +#define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-" + struct tst_test *tst_test; static const char *tid; @@ -414,6 +417,9 @@ static void print_help(void) { unsigned int i; + fprintf(stderr, "Options\n"); + fprintf(stderr, "-------\n\n"); + for (i = 0; i < ARRAY_SIZE(options); i++) fprintf(stderr, "%s\n", options[i].help); @@ -424,6 +430,27 @@ static void print_help(void) fprintf(stderr, "%s\n", tst_test->options[i].help); } +static void print_test_info(void) +{ + unsigned int i; + const struct tst_tag *tags = tst_test->tags; + + printf("\nTags\n"); + printf("----\n\n"); + + if (tags) { + for (i = 0; tags[i].name; i++) { + if (!strcmp(tags[i].name, "CVE")) + printf(CVE_DB_URL "%s\n", tags[i].value); + else if (!strcmp(tags[i].name, "linux-git")) + printf(LINUX_GIT_URL "%s\n", tags[i].value); + else + printf("%s: %s\n", tags[i].name, tags[i].value); + printf("\n"); + } + } +} + static void check_option_collision(void) { unsigned int i, j; @@ -499,6 +526,7 @@ static void parse_opts(int argc, char *argv[]) break; case 'h': print_help(); + print_test_info(); exit(0); case 'i': iterations = atoi(optarg); @@ -584,26 +612,61 @@ int tst_parse_float(const char *str, float *val, float min, float max) return 0; } +static void print_colored(const char *str) +{ + if (tst_color_enabled(STDOUT_FILENO)) + printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET); + else + printf("%s", str); +} + +static void print_failure_hints(void) +{ + unsigned int i; + const struct tst_tag *tags = tst_test->tags; + + if (!tags) + return; + + for (i = 0; tags[i].name; i++) { + if (!strcmp(tags[i].name, "linux-git")) { + printf("\n"); + print_colored("HINT: "); + printf("This is a regression test for linux kernel, see commit:\n\n" + LINUX_GIT_URL "%s\n", tags[i].value); + } + + if (!strcmp(tags[i].name, "CVE")) { + printf("\n"); + print_colored("HINT: "); + printf("This test also tests for CVE-%s, see:\n\n" + CVE_DB_URL "%s\n", tags[i].value, tags[i].value); + } + } +} + static void do_exit(int ret) { if (results) { - printf("\nSummary:\n"); - printf("passed %d\n", results->passed); - printf("failed %d\n", results->failed); - printf("skipped %d\n", results->skipped); - printf("warnings %d\n", results->warnings); - if (results->passed && ret == TCONF) ret = 0; - if (results->failed) + if (results->failed) { ret |= TFAIL; + print_failure_hints(); + } if (results->skipped && !results->passed) ret |= TCONF; if (results->warnings) ret |= TWARN; + + printf("\nSummary:\n"); + printf("passed %d\n", results->passed); + printf("failed %d\n", results->failed); + printf("skipped %d\n", results->skipped); + printf("warnings %d\n", results->warnings); } do_cleanup(); -- 2.21.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-14 11:25 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis @ 2019-10-15 5:12 ` Li Wang 2019-10-15 7:42 ` Richard Palethorpe 2019-10-15 9:54 ` Cyril Hrubis 2019-10-16 13:30 ` Petr Vorel 1 sibling, 2 replies; 16+ messages in thread From: Li Wang @ 2019-10-15 5:12 UTC (permalink / raw) To: ltp On Mon, Oct 14, 2019 at 7:25 PM Cyril Hrubis <chrubis@suse.cz> wrote: > The newly introduced test tags are generic name-value pairs that can > hold test metadata, the intended use for now is to store kernel commit > hashes for kernel reproducers as well as CVE ids. The mechanism is > however choosen to be very generic so that it's easy to add basically > any information later on. > > As it is the main purpose is to print hints for a test failures. If a > test that has been written as a kernel reproducer fails it prints nice > URL pointing to a kernel commit that may be missing. > Nice to see this feature patchset achieved, it looks quite clear&good to me. Just have some tiny comments below. > > Example output: > > -------------------------------------------------------------------------- > tst_test.c:1145: INFO: Timeout per run is 0h 05m 00s > add_key02.c:98: FAIL: unexpected error with key type 'asymmetric': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'cifs.idmap': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'cifs.spnego': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'pkcs7_test': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'rxrpc': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'rxrpc_s': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'user': EINVAL > add_key02.c:98: FAIL: unexpected error with key type 'logon': EINVAL > > HINT: This is a regression test for linux kernel, see commit: > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c > > HINT: This test also tests for CVE-2017-15274, see: > > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15274 > > Summary: > passed 0 > failed 8 > skipped 0 > warnings 0 > -------------------------------------------------------------------------- > > Signed-off-by: Cyril Hrubis <chrubis@suse.cz> > --- > include/tst_test.h | 10 ++++++ > lib/tst_test.c | 77 +++++++++++++++++++++++++++++++++++++++++----- > 2 files changed, 80 insertions(+), 7 deletions(-) > > diff --git a/include/tst_test.h b/include/tst_test.h > index 84acf2c59..4a51b6d16 100644 > --- a/include/tst_test.h > +++ b/include/tst_test.h > @@ -108,6 +108,11 @@ int tst_parse_int(const char *str, int *val, int min, > int max); > int tst_parse_long(const char *str, long *val, long min, long max); > int tst_parse_float(const char *str, float *val, float min, float max); > > +struct tst_tag { > + const char *name; > + const char *value; > +}; > + > extern unsigned int tst_variant; > > struct tst_test { > @@ -212,6 +217,11 @@ struct tst_test { > * NULL-terminated array of capability settings > */ > struct tst_cap *caps; > + > + /* > + * {NULL, NULL} terminated array of tags. > + */ > + const struct tst_tag *tags; > }; > > /* > diff --git a/lib/tst_test.c b/lib/tst_test.c > index 6239acf89..2129f38cb 100644 > --- a/lib/tst_test.c > +++ b/lib/tst_test.c > @@ -31,6 +31,9 @@ > #include "old_device.h" > #include "old_tmpdir.h" > > +#define LINUX_GIT_URL " > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id= > " > +#define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-" > + > struct tst_test *tst_test; > > static const char *tid; > @@ -414,6 +417,9 @@ static void print_help(void) > { > unsigned int i; > > + fprintf(stderr, "Options\n"); > + fprintf(stderr, "-------\n\n"); > + > for (i = 0; i < ARRAY_SIZE(options); i++) > fprintf(stderr, "%s\n", options[i].help); > > @@ -424,6 +430,27 @@ static void print_help(void) > fprintf(stderr, "%s\n", tst_test->options[i].help); > } > > +static void print_test_info(void) > print_test_info sounds like general information for the test, maybe print_tags() is a better/precise name? > +{ > + unsigned int i; > + const struct tst_tag *tags = tst_test->tags; > + > + printf("\nTags\n"); > + printf("----\n\n"); > + > + if (tags) { > + for (i = 0; tags[i].name; i++) { > + if (!strcmp(tags[i].name, "CVE")) > + printf(CVE_DB_URL "%s\n", tags[i].value); > + else if (!strcmp(tags[i].name, "linux-git")) > + printf(LINUX_GIT_URL "%s\n", > tags[i].value); > + else > + printf("%s: %s\n", tags[i].name, > tags[i].value); > + printf("\n"); > + } > + } > +} > + > static void check_option_collision(void) > { > unsigned int i, j; > @@ -499,6 +526,7 @@ static void parse_opts(int argc, char *argv[]) > break; > case 'h': > print_help(); > + print_test_info(); > exit(0); > case 'i': > iterations = atoi(optarg); > @@ -584,26 +612,61 @@ int tst_parse_float(const char *str, float *val, > float min, float max) > return 0; > } > > +static void print_colored(const char *str) > +{ > + if (tst_color_enabled(STDOUT_FILENO)) > + printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET); > + else > + printf("%s", str); > +} > + > +static void print_failure_hints(void) > +{ > + unsigned int i; > + const struct tst_tag *tags = tst_test->tags; > + > + if (!tags) > + return; > + > + for (i = 0; tags[i].name; i++) { > + if (!strcmp(tags[i].name, "linux-git")) { > + printf("\n"); > + print_colored("HINT: "); + printf("This is a regression test for linux kernel, > see commit:\n\n" > + LINUX_GIT_URL "%s\n", tags[i].value); > This sentence 'HINT: This is a ...' will be printed many times if there are many commits in tags, I prefer to see only once in front of these linux-kernel links. > + } > + > + if (!strcmp(tags[i].name, "CVE")) { > + printf("\n"); > + print_colored("HINT: "); > + printf("This test also tests for CVE-%s, see:\n\n" > + CVE_DB_URL "%s\n", tags[i].value, > tags[i].value); > Here as well. -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191015/df8ab998/attachment.htm> ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-15 5:12 ` Li Wang @ 2019-10-15 7:42 ` Richard Palethorpe 2019-10-15 9:57 ` Cyril Hrubis 2019-10-15 9:54 ` Cyril Hrubis 1 sibling, 1 reply; 16+ messages in thread From: Richard Palethorpe @ 2019-10-15 7:42 UTC (permalink / raw) To: ltp Hello, Li Wang <liwang@redhat.com> writes: >> } >> >> +static void print_colored(const char *str) >> +{ >> + if (tst_color_enabled(STDOUT_FILENO)) >> + printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET); >> + else >> + printf("%s", str); >> +} >> + >> +static void print_failure_hints(void) >> +{ >> + unsigned int i; >> + const struct tst_tag *tags = tst_test->tags; >> + >> + if (!tags) >> + return; >> + >> + for (i = 0; tags[i].name; i++) { >> + if (!strcmp(tags[i].name, "linux-git")) { >> + printf("\n"); >> + print_colored("HINT: "); > > + printf("This is a regression test for linux kernel, >> see commit:\n\n" It may be better just to print "Associated Linux kernel commit: ..." and "Associated bug ID: CVE-..." in the loop. Then we can avoid any discussions about what type of test it is. Also, a test description tag could be added if necessary. Maybe it would also be a good idea to link back to the test source code on github? Possibly this info could be injected at build time? >> + LINUX_GIT_URL "%s\n", tags[i].value); >> > > This sentence 'HINT: This is a ...' will be printed many times if there are > many commits in tags, I prefer to see only once in front of these > linux-kernel links. > > >> + } >> + >> + if (!strcmp(tags[i].name, "CVE")) { >> + printf("\n"); >> + print_colored("HINT: "); >> + printf("This test also tests for CVE-%s, see:\n\n" >> + CVE_DB_URL "%s\n", tags[i].value, >> tags[i].value); >> > > Here as well. > > -- > Regards, > Li Wang -- Thank you, Richard. ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-15 7:42 ` Richard Palethorpe @ 2019-10-15 9:57 ` Cyril Hrubis 0 siblings, 0 replies; 16+ messages in thread From: Cyril Hrubis @ 2019-10-15 9:57 UTC (permalink / raw) To: ltp Hi! > It may be better just to print "Associated Linux kernel commit: ..." and > "Associated bug ID: CVE-..." in the loop. Then we can avoid any > discussions about what type of test it is. I will try that. > Also, a test description tag could be added if necessary. My plan is actually to keep the description in the top level comment but parse that by the docparse tool, then we will have a browseable test documentation that could be put to the web and we can point to that here. > Maybe it would also be a good idea to link back to the test source code > on github? Possibly this info could be injected at build time? I tried to inject things at a build time and it wasn't pretty, so I would like to avoid that. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-15 5:12 ` Li Wang 2019-10-15 7:42 ` Richard Palethorpe @ 2019-10-15 9:54 ` Cyril Hrubis 1 sibling, 0 replies; 16+ messages in thread From: Cyril Hrubis @ 2019-10-15 9:54 UTC (permalink / raw) To: ltp Hi! > > +static void print_test_info(void) > > > > print_test_info sounds like general information for the test, maybe > print_tags() is a better/precise name? I named it as such in a case that we will add anything else there, but I do not have a strong opinion, I can change that if you insist. > > +static void print_failure_hints(void) > > +{ > > + unsigned int i; > > + const struct tst_tag *tags = tst_test->tags; > > + > > + if (!tags) > > + return; > > + > > + for (i = 0; tags[i].name; i++) { > > + if (!strcmp(tags[i].name, "linux-git")) { > > + printf("\n"); > > + print_colored("HINT: "); > > + printf("This is a regression test for linux kernel, > > see commit:\n\n" > > + LINUX_GIT_URL "%s\n", tags[i].value); > > > > This sentence 'HINT: This is a ...' will be printed many times if there are > many commits in tags, I prefer to see only once in front of these > linux-kernel links. Good catch, I will do something about this. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-14 11:25 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis 2019-10-15 5:12 ` Li Wang @ 2019-10-16 13:30 ` Petr Vorel 2019-10-16 14:46 ` Cyril Hrubis 1 sibling, 1 reply; 16+ messages in thread From: Petr Vorel @ 2019-10-16 13:30 UTC (permalink / raw) To: ltp Hi, > + if (tags) { > + for (i = 0; tags[i].name; i++) { > + if (!strcmp(tags[i].name, "CVE")) > + printf(CVE_DB_URL "%s\n", tags[i].value); > + else if (!strcmp(tags[i].name, "linux-git")) Not sure if it's worth of defining some enums instead of "CVE", "linux-git" (would catch typos). > + printf(LINUX_GIT_URL "%s\n", tags[i].value); > + else > + printf("%s: %s\n", tags[i].name, tags[i].value); > + printf("\n"); Kind regards, Petr ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-16 13:30 ` Petr Vorel @ 2019-10-16 14:46 ` Cyril Hrubis 2019-10-16 19:43 ` Petr Vorel 0 siblings, 1 reply; 16+ messages in thread From: Cyril Hrubis @ 2019-10-16 14:46 UTC (permalink / raw) To: ltp Hi! > > + if (tags) { > > + for (i = 0; tags[i].name; i++) { > > + if (!strcmp(tags[i].name, "CVE")) > > + printf(CVE_DB_URL "%s\n", tags[i].value); > > + else if (!strcmp(tags[i].name, "linux-git")) > Not sure if it's worth of defining some enums instead of "CVE", "linux-git" > (would catch typos). I wanted to avoid enums because they are not flexible enough. The plan here is that docparse would do sanity checks on test metadata and fail the compilation if it founds typos there. With the checks in the docparse tool we can easily check for tags that looks like typos, i.e. check the levenshtein distance from all known tags and print a nice looking error message... -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-16 14:46 ` Cyril Hrubis @ 2019-10-16 19:43 ` Petr Vorel 0 siblings, 0 replies; 16+ messages in thread From: Petr Vorel @ 2019-10-16 19:43 UTC (permalink / raw) To: ltp Hi Cyril, > > > + if (tags) { > > > + for (i = 0; tags[i].name; i++) { > > > + if (!strcmp(tags[i].name, "CVE")) > > > + printf(CVE_DB_URL "%s\n", tags[i].value); > > > + else if (!strcmp(tags[i].name, "linux-git")) > > Not sure if it's worth of defining some enums instead of "CVE", "linux-git" > > (would catch typos). > I wanted to avoid enums because they are not flexible enough. The plan > here is that docparse would do sanity checks on test metadata and fail > the compilation if it founds typos there. With the checks in the > docparse tool we can easily check for tags that looks like typos, i.e. > check the levenshtein distance from all known tags and print a nice > looking error message... OK, sounds good :). Kind regards, Petr ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-14 11:25 [LTP] [PATCH 0/2] Add support for a test tags Cyril Hrubis 2019-10-14 11:25 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis @ 2019-10-14 11:25 ` Cyril Hrubis 2019-10-15 4:54 ` Li Wang 1 sibling, 1 reply; 16+ messages in thread From: Cyril Hrubis @ 2019-10-14 11:25 UTC (permalink / raw) To: ltp Not all tests are annotated at the moment but the majority has been finished for now. Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- testcases/cve/cve-2014-0196.c | 5 +++++ testcases/cve/cve-2015-3290.c | 5 +++++ testcases/cve/cve-2016-10044.c | 5 +++++ testcases/cve/cve-2016-7042.c | 5 +++++ testcases/cve/cve-2016-7117.c | 5 +++++ testcases/cve/cve-2017-16939.c | 5 +++++ testcases/cve/cve-2017-17052.c | 5 +++++ testcases/cve/cve-2017-17053.c | 5 +++++ testcases/cve/cve-2017-2618.c | 5 +++++ testcases/cve/cve-2017-2671.c | 5 +++++ testcases/cve/meltdown.c | 6 +++++- testcases/cve/stack_clash.c | 4 ++++ testcases/kernel/crypto/af_alg01.c | 5 +++++ testcases/kernel/crypto/af_alg02.c | 5 +++++ testcases/kernel/crypto/af_alg03.c | 4 ++++ testcases/kernel/crypto/af_alg04.c | 4 ++++ testcases/kernel/crypto/af_alg05.c | 8 +++++++- testcases/kernel/crypto/af_alg06.c | 4 ++++ testcases/kernel/crypto/crypto_user01.c | 6 ++++++ testcases/kernel/crypto/crypto_user02.c | 4 ++++ testcases/kernel/crypto/pcrypt_aead01.c | 5 +++++ testcases/kernel/sound/snd_timer01.c | 6 ++++++ testcases/kernel/syscalls/accept/accept02.c | 5 +++++ testcases/kernel/syscalls/acct/acct02.c | 4 ++++ testcases/kernel/syscalls/add_key/add_key02.c | 5 +++++ testcases/kernel/syscalls/bpf/bpf_prog02.c | 4 ++++ testcases/kernel/syscalls/futex/futex_cmp_requeue02.c | 5 +++++ testcases/kernel/syscalls/keyctl/keyctl02.c | 4 ++++ testcases/kernel/syscalls/keyctl/keyctl04.c | 5 +++++ testcases/kernel/syscalls/keyctl/keyctl05.c | 4 ++++ testcases/kernel/syscalls/move_pages/move_pages12.c | 6 ++++++ testcases/kernel/syscalls/realpath/realpath01.c | 4 ++++ testcases/kernel/syscalls/request_key/request_key05.c | 4 ++++ testcases/kernel/syscalls/sbrk/sbrk03.c | 4 ++++ testcases/kernel/syscalls/sendto/sendto02.c | 4 ++++ testcases/kernel/syscalls/setrlimit/setrlimit06.c | 4 ++++ 36 files changed, 171 insertions(+), 2 deletions(-) diff --git a/testcases/cve/cve-2014-0196.c b/testcases/cve/cve-2014-0196.c index 6997fffe4..012cbb7cd 100644 --- a/testcases/cve/cve-2014-0196.c +++ b/testcases/cve/cve-2014-0196.c @@ -141,4 +141,9 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "4291086b1f08"}, + {"CVE", "2014-0196"}, + {} + } }; diff --git a/testcases/cve/cve-2015-3290.c b/testcases/cve/cve-2015-3290.c index fe630e4af..6c4fd57ac 100644 --- a/testcases/cve/cve-2015-3290.c +++ b/testcases/cve/cve-2015-3290.c @@ -469,6 +469,11 @@ static struct tst_test test = { .setup = setup, .timeout = TIMEOUT, .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "9b6e6a8334d5"}, + {"CVE", "2015-3290"}, + {} + } }; #else /* HAVE_PERF_EVENT_ATTR && (defined(__x86_64__) || defined(__i386__)) */ diff --git a/testcases/cve/cve-2016-10044.c b/testcases/cve/cve-2016-10044.c index c2e79df31..7519676a1 100644 --- a/testcases/cve/cve-2016-10044.c +++ b/testcases/cve/cve-2016-10044.c @@ -62,4 +62,9 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .min_kver = "2.6.8", + .tags = (const struct tst_tag[]) { + {"linux-git", "22f6b4d34fcf"}, + {"CVE", "2016-10044"}, + {} + } }; diff --git a/testcases/cve/cve-2016-7042.c b/testcases/cve/cve-2016-7042.c index 35950adc7..4434265dd 100644 --- a/testcases/cve/cve-2016-7042.c +++ b/testcases/cve/cve-2016-7042.c @@ -71,4 +71,9 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = do_test, + .tags = (const struct tst_tag[]) { + {"linux-git", "03dab869b7b2"}, + {"CVE", "2016-7042"}, + {} + } }; diff --git a/testcases/cve/cve-2016-7117.c b/testcases/cve/cve-2016-7117.c index 5d9f842aa..dca002924 100644 --- a/testcases/cve/cve-2016-7117.c +++ b/testcases/cve/cve-2016-7117.c @@ -150,4 +150,9 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .min_kver = "2.6.33", + .tags = (const struct tst_tag[]) { + {"linux-git", "a2e2725541fa"}, + {"CVE", "2016-7117"}, + {} + } }; diff --git a/testcases/cve/cve-2017-16939.c b/testcases/cve/cve-2017-16939.c index 4ecc86337..9ab1eb2c3 100644 --- a/testcases/cve/cve-2017-16939.c +++ b/testcases/cve/cve-2017-16939.c @@ -77,4 +77,9 @@ static void run(void) static struct tst_test test = { .setup = setup, .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "1137b5e2529a"}, + {"CVE", "2017-16939"}, + {} + } }; diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c index 81123fce8..d89c62eb6 100644 --- a/testcases/cve/cve-2017-17052.c +++ b/testcases/cve/cve-2017-17052.c @@ -111,4 +111,9 @@ static struct tst_test test = { .cleanup = cleanup, .setup = setup, .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "2b7e8665b4ff"}, + {"CVE", "2017-17052"}, + {} + } }; diff --git a/testcases/cve/cve-2017-17053.c b/testcases/cve/cve-2017-17053.c index 646ec11cc..08a08211d 100644 --- a/testcases/cve/cve-2017-17053.c +++ b/testcases/cve/cve-2017-17053.c @@ -154,6 +154,11 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "ccd5b3235180"}, + {"CVE", "2017-17053"}, + {} + } }; #else diff --git a/testcases/cve/cve-2017-2618.c b/testcases/cve/cve-2017-2618.c index cf018d401..4ab3cfb8f 100644 --- a/testcases/cve/cve-2017-2618.c +++ b/testcases/cve/cve-2017-2618.c @@ -51,4 +51,9 @@ static struct tst_test test = { .forks_child = 1, .setup = setup, .test_all = do_test, + .tags = (const struct tst_tag[]) { + {"linux-git", "0c461cb727d1"}, + {"CVE", "2017-2618"}, + {} + } }; diff --git a/testcases/cve/cve-2017-2671.c b/testcases/cve/cve-2017-2671.c index b92c1d94b..e72795d15 100644 --- a/testcases/cve/cve-2017-2671.c +++ b/testcases/cve/cve-2017-2671.c @@ -109,4 +109,9 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .needs_root = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "43a6684519ab"}, + {"CVE", "2017-2671"}, + {} + } }; diff --git a/testcases/cve/meltdown.c b/testcases/cve/meltdown.c index 863f1a9e4..db00e1811 100644 --- a/testcases/cve/meltdown.c +++ b/testcases/cve/meltdown.c @@ -377,7 +377,11 @@ static struct tst_test test = { .setup = setup, .test_all = run, .cleanup = cleanup, - .min_kver = "2.6.32" + .min_kver = "2.6.32", + .tags = (const struct tst_tag[]) { + {"CVE", "2017-5754"}, + {} + } }; #else /* #if defined(__x86_64__) || defined(__i386__) */ diff --git a/testcases/cve/stack_clash.c b/testcases/cve/stack_clash.c index 3ea3ec491..ecbf5f6da 100644 --- a/testcases/cve/stack_clash.c +++ b/testcases/cve/stack_clash.c @@ -250,4 +250,8 @@ static struct tst_test test = { .needs_root = 1, .setup = setup, .test_all = stack_clash_test, + .tags = (const struct tst_tag[]) { + {"CVE", "2017-1000364"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg01.c b/testcases/kernel/crypto/af_alg01.c index bb6e01738..d3e823e6c 100644 --- a/testcases/kernel/crypto/af_alg01.c +++ b/testcases/kernel/crypto/af_alg01.c @@ -76,4 +76,9 @@ static void do_test(unsigned int i) static struct tst_test test = { .test = do_test, .tcnt = ARRAY_SIZE(hash_algs), + .tags = (const struct tst_tag[]) { + {"linux-git", "af3ff8045bbf"}, + {"CVE", "2017-17806"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg02.c b/testcases/kernel/crypto/af_alg02.c index 08087ecac..fab0010c9 100644 --- a/testcases/kernel/crypto/af_alg02.c +++ b/testcases/kernel/crypto/af_alg02.c @@ -59,4 +59,9 @@ static struct tst_test test = { .test_all = run, .timeout = 20, .needs_checkpoints = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "ecaaab564978"}, + {"CVE", "2017-17805"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg03.c b/testcases/kernel/crypto/af_alg03.c index 240c52835..5f214e48b 100644 --- a/testcases/kernel/crypto/af_alg03.c +++ b/testcases/kernel/crypto/af_alg03.c @@ -28,4 +28,8 @@ static void run(void) static struct tst_test test = { .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "e57121d08c38"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg04.c b/testcases/kernel/crypto/af_alg04.c index 48239e020..df049048a 100644 --- a/testcases/kernel/crypto/af_alg04.c +++ b/testcases/kernel/crypto/af_alg04.c @@ -55,4 +55,8 @@ static void run(void) static struct tst_test test = { .test_all = run, .forks_child = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "bb2964810233"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg05.c b/testcases/kernel/crypto/af_alg05.c index 964adffcc..df2b03546 100644 --- a/testcases/kernel/crypto/af_alg05.c +++ b/testcases/kernel/crypto/af_alg05.c @@ -11,7 +11,7 @@ * boundary. Based on the reproducer from the commit message. Note that this * issue only reproduces on certain architectures, such as arm and arm64. * - * On some older kernel without commit 160544075f2 ("crypto: scatterwalk - Hide + * On some older kernel without commit 160544075f2a ("crypto: scatterwalk - Hide * PageSlab call to optimise away flush_dcache_page") , it doesn't use * ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE macro. It can crash on all architectures. * Without skcipher walk interface, it is also a regresstion test for commit @@ -46,4 +46,10 @@ static void run(void) static struct tst_test test = { .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "8088d3dd4d7c"}, + {"linux-git", "160544075f2a"}, + {"linux-git", "0868def3e410"}, + {} + } }; diff --git a/testcases/kernel/crypto/af_alg06.c b/testcases/kernel/crypto/af_alg06.c index 586d91b1d..6acdf8532 100644 --- a/testcases/kernel/crypto/af_alg06.c +++ b/testcases/kernel/crypto/af_alg06.c @@ -48,4 +48,8 @@ static void run(void) static struct tst_test test = { .test_all = run, + .tags = (const struct tst_tag[]) { + {"linux-git", "8f9c46934848"}, + {} + } }; diff --git a/testcases/kernel/crypto/crypto_user01.c b/testcases/kernel/crypto/crypto_user01.c index 163d7a2dd..47bf9f0d2 100644 --- a/testcases/kernel/crypto/crypto_user01.c +++ b/testcases/kernel/crypto/crypto_user01.c @@ -192,4 +192,10 @@ static struct tst_test test = { .setup = setup, .test_all = run, .cleanup = cleanup, + .tags = (const struct tst_tag[]) { + {"linux-git", "f43f39958beb"}, + {"CVE", "2013-2547"}, + {"CVE", "2018-19854"}, + {} + } }; diff --git a/testcases/kernel/crypto/crypto_user02.c b/testcases/kernel/crypto/crypto_user02.c index 6090b5a4e..384e344f2 100644 --- a/testcases/kernel/crypto/crypto_user02.c +++ b/testcases/kernel/crypto/crypto_user02.c @@ -105,4 +105,8 @@ static struct tst_test test = { .cleanup = cleanup, .needs_root = 1, .forks_child = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "21d4120ec6f5"}, + {} + } }; diff --git a/testcases/kernel/crypto/pcrypt_aead01.c b/testcases/kernel/crypto/pcrypt_aead01.c index d9c27f69d..42ae259fb 100644 --- a/testcases/kernel/crypto/pcrypt_aead01.c +++ b/testcases/kernel/crypto/pcrypt_aead01.c @@ -69,4 +69,9 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .needs_root = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "d76c68109f37"}, + {"CVE", "2017-5754"}, + {} + } }; diff --git a/testcases/kernel/sound/snd_timer01.c b/testcases/kernel/sound/snd_timer01.c index f62696659..ebac6e1c6 100644 --- a/testcases/kernel/sound/snd_timer01.c +++ b/testcases/kernel/sound/snd_timer01.c @@ -140,4 +140,10 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, + .tags = (const struct tst_tag[]) { + {"linux-git", "d11662f4f798"}, + {"linux-git", "ba3021b2c79b"}, + {"CVE", "2017-1000380"}, + {NULL, NULL} + } }; diff --git a/testcases/kernel/syscalls/accept/accept02.c b/testcases/kernel/syscalls/accept/accept02.c index 1a0f625c9..576b91ce8 100644 --- a/testcases/kernel/syscalls/accept/accept02.c +++ b/testcases/kernel/syscalls/accept/accept02.c @@ -142,4 +142,9 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .needs_checkpoints = 1, + .tags = (const struct tst_tag[]) { + {"CVE", "2017-8890"}, + {"linux-git", "657831ff"}, + {NULL, NULL}, + } }; diff --git a/testcases/kernel/syscalls/acct/acct02.c b/testcases/kernel/syscalls/acct/acct02.c index 4d95aafec..890568b08 100644 --- a/testcases/kernel/syscalls/acct/acct02.c +++ b/testcases/kernel/syscalls/acct/acct02.c @@ -263,4 +263,8 @@ static struct tst_test test = { .cleanup = cleanup, .needs_tmpdir = 1, .needs_root = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "4d9570158b626"}, + {} + } }; diff --git a/testcases/kernel/syscalls/add_key/add_key02.c b/testcases/kernel/syscalls/add_key/add_key02.c index 4a5cf054a..159234f67 100644 --- a/testcases/kernel/syscalls/add_key/add_key02.c +++ b/testcases/kernel/syscalls/add_key/add_key02.c @@ -83,4 +83,9 @@ static void verify_add_key(unsigned int i) static struct tst_test test = { .tcnt = ARRAY_SIZE(tcases), .test = verify_add_key, + .tags = (const struct tst_tag[]) { + {"linux-git", "5649645d725c"}, + {"CVE", "2017-15274"}, + {} + } }; diff --git a/testcases/kernel/syscalls/bpf/bpf_prog02.c b/testcases/kernel/syscalls/bpf/bpf_prog02.c index fb8f3f781..47f313a6d 100644 --- a/testcases/kernel/syscalls/bpf/bpf_prog02.c +++ b/testcases/kernel/syscalls/bpf/bpf_prog02.c @@ -190,5 +190,9 @@ static struct tst_test test = { {&attr, .size = sizeof(*attr)}, {&msg, .size = sizeof(MSG)}, {}, + }, + .tags = (const struct tst_tag[]) { + {"linux-git", "3612af783cf5"}, + {} } }; diff --git a/testcases/kernel/syscalls/futex/futex_cmp_requeue02.c b/testcases/kernel/syscalls/futex/futex_cmp_requeue02.c index 3ea19428c..228d8cff6 100644 --- a/testcases/kernel/syscalls/futex/futex_cmp_requeue02.c +++ b/testcases/kernel/syscalls/futex/futex_cmp_requeue02.c @@ -73,4 +73,9 @@ static struct tst_test test = { .cleanup = cleanup, .test = verify_futex_cmp_requeue, .tcnt = ARRAY_SIZE(tcases), + .tags = (const struct tst_tag[]) { + {"CVE", "2018-6927"}, + {"linux-git", "fbe0e839d1e2"}, + {} + } }; diff --git a/testcases/kernel/syscalls/keyctl/keyctl02.c b/testcases/kernel/syscalls/keyctl/keyctl02.c index b034275d2..f4ee03f33 100644 --- a/testcases/kernel/syscalls/keyctl/keyctl02.c +++ b/testcases/kernel/syscalls/keyctl/keyctl02.c @@ -136,4 +136,8 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = do_test, + .tags = (const struct tst_tag[]) { + {"linux-git", "b4a1b4f5047e"}, + {} + } }; diff --git a/testcases/kernel/syscalls/keyctl/keyctl04.c b/testcases/kernel/syscalls/keyctl/keyctl04.c index d30d25b04..1fed23ca6 100644 --- a/testcases/kernel/syscalls/keyctl/keyctl04.c +++ b/testcases/kernel/syscalls/keyctl/keyctl04.c @@ -43,4 +43,9 @@ static void do_test(void) static struct tst_test test = { .test_all = do_test, + .tags = (const struct tst_tag[]) { + {"CVE", "2017-7472"}, + {"linux-git", "c9f838d104fe"}, + {} + } }; diff --git a/testcases/kernel/syscalls/keyctl/keyctl05.c b/testcases/kernel/syscalls/keyctl/keyctl05.c index 653b838f5..c592eb49e 100644 --- a/testcases/kernel/syscalls/keyctl/keyctl05.c +++ b/testcases/kernel/syscalls/keyctl/keyctl05.c @@ -207,4 +207,8 @@ static struct tst_test test = { .tcnt = 3, .test = do_test, .forks_child = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "63a0b0509e70"}, + {} + } }; diff --git a/testcases/kernel/syscalls/move_pages/move_pages12.c b/testcases/kernel/syscalls/move_pages/move_pages12.c index c2927fb86..c906acbe3 100644 --- a/testcases/kernel/syscalls/move_pages/move_pages12.c +++ b/testcases/kernel/syscalls/move_pages/move_pages12.c @@ -330,6 +330,12 @@ static struct tst_test test = { .cleanup = cleanup, .test = do_test, .tcnt = ARRAY_SIZE(tcases), + .tags = (const struct tst_tag[]) { + {"linux-git", "e66f17ff7177"}, + {"linux-git", "c9d398fa2378"}, + {"linux-git", "4643d67e8cb0"}, + {} + } }; #else diff --git a/testcases/kernel/syscalls/realpath/realpath01.c b/testcases/kernel/syscalls/realpath/realpath01.c index fc2af2a1c..c0381e9cb 100644 --- a/testcases/kernel/syscalls/realpath/realpath01.c +++ b/testcases/kernel/syscalls/realpath/realpath01.c @@ -41,4 +41,8 @@ static struct tst_test test = { .setup = setup, .needs_root = 1, .needs_tmpdir = 1, + .tags = (const struct tst_tag[]) { + {"CVE", "2018-1000001"}, + {} + } }; diff --git a/testcases/kernel/syscalls/request_key/request_key05.c b/testcases/kernel/syscalls/request_key/request_key05.c index 129cc0507..a17d1d058 100644 --- a/testcases/kernel/syscalls/request_key/request_key05.c +++ b/testcases/kernel/syscalls/request_key/request_key05.c @@ -31,4 +31,8 @@ static void run(void) static struct tst_test test = { .test_all = run, + .tags = (const struct tst_tag[]) { + {"CVE", "2017-6951"}, + {} + } }; diff --git a/testcases/kernel/syscalls/sbrk/sbrk03.c b/testcases/kernel/syscalls/sbrk/sbrk03.c index 761a01c9d..80d2020ff 100644 --- a/testcases/kernel/syscalls/sbrk/sbrk03.c +++ b/testcases/kernel/syscalls/sbrk/sbrk03.c @@ -66,4 +66,8 @@ static void sbrk_test(void) static struct tst_test test = { .test_all = sbrk_test, + .tags = (const struct tst_tag[]) { + {"linux-git", "473a06572fcd"}, + {} + } }; diff --git a/testcases/kernel/syscalls/sendto/sendto02.c b/testcases/kernel/syscalls/sendto/sendto02.c index 5c9f00125..743101733 100644 --- a/testcases/kernel/syscalls/sendto/sendto02.c +++ b/testcases/kernel/syscalls/sendto/sendto02.c @@ -77,4 +77,8 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = verify_sendto, + .tags = (const struct tst_tag[]) { + {"linux-git", "6e51fe757259"}, + {} + } }; diff --git a/testcases/kernel/syscalls/setrlimit/setrlimit06.c b/testcases/kernel/syscalls/setrlimit/setrlimit06.c index 87e6b0e32..726b26841 100644 --- a/testcases/kernel/syscalls/setrlimit/setrlimit06.c +++ b/testcases/kernel/syscalls/setrlimit/setrlimit06.c @@ -114,4 +114,8 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .forks_child = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "c3bca5d450b62"}, + {} + } }; -- 2.21.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-14 11:25 ` [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags Cyril Hrubis @ 2019-10-15 4:54 ` Li Wang 2019-10-15 9:52 ` Cyril Hrubis 0 siblings, 1 reply; 16+ messages in thread From: Li Wang @ 2019-10-15 4:54 UTC (permalink / raw) To: ltp Cyril Hrubis <chrubis@suse.cz> wrote: > > diff --git a/testcases/cve/cve-2014-0196.c b/testcases/cve/cve-2014-0196.c > index 6997fffe4..012cbb7cd 100644 > --- a/testcases/cve/cve-2014-0196.c > +++ b/testcases/cve/cve-2014-0196.c > @@ -141,4 +141,9 @@ static struct tst_test test = { > .setup = setup, > .cleanup = cleanup, > .test_all = run, > + .tags = (const struct tst_tag[]) { > + {"linux-git", "4291086b1f08"}, > + {"CVE", "2014-0196"}, > + {} > Some of the tests use {NULL, NULL} to terminate tag array but some not, maybe we need to keep consistent in usage. -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191015/bc68b025/attachment.htm> ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-15 4:54 ` Li Wang @ 2019-10-15 9:52 ` Cyril Hrubis 2019-10-16 13:21 ` Petr Vorel 0 siblings, 1 reply; 16+ messages in thread From: Cyril Hrubis @ 2019-10-15 9:52 UTC (permalink / raw) To: ltp Hi! > > diff --git a/testcases/cve/cve-2014-0196.c b/testcases/cve/cve-2014-0196.c > > index 6997fffe4..012cbb7cd 100644 > > --- a/testcases/cve/cve-2014-0196.c > > +++ b/testcases/cve/cve-2014-0196.c > > @@ -141,4 +141,9 @@ static struct tst_test test = { > > .setup = setup, > > .cleanup = cleanup, > > .test_all = run, > > + .tags = (const struct tst_tag[]) { > > + {"linux-git", "4291086b1f08"}, > > + {"CVE", "2014-0196"}, > > + {} > > > > Some of the tests use {NULL, NULL} to terminate tag array but some not, > maybe we need to keep consistent in usage. I guess I missed some when I was converting {NULL, NULL} to just {}. I will fix that in v2. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-15 9:52 ` Cyril Hrubis @ 2019-10-16 13:21 ` Petr Vorel 2019-10-17 3:22 ` Li Wang 2019-10-17 8:38 ` Jan Stancek 0 siblings, 2 replies; 16+ messages in thread From: Petr Vorel @ 2019-10-16 13:21 UTC (permalink / raw) To: ltp Hi, > > > + .tags = (const struct tst_tag[]) { > > > + {"linux-git", "4291086b1f08"}, > > > + {"CVE", "2014-0196"}, > > > + {} > > Some of the tests use {NULL, NULL} to terminate tag array but some not, > > maybe we need to keep consistent in usage. > I guess I missed some when I was converting {NULL, NULL} to just {}. I > will fix that in v2. +1 Nice work, thanks :). Reviewed-by: Petr Vorel <pvorel@suse.cz> nit: I don't that much blank lines, I'd prefer more compact output: OPTIONS -h Prints this help -i n Execute test n times -I x Execute test for n seconds -C ARG Run child process with ARG arguments (used internally) TAGS https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c461cb727d1 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-2618 Kind regards, Petr ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-16 13:21 ` Petr Vorel @ 2019-10-17 3:22 ` Li Wang 2019-10-17 8:38 ` Jan Stancek 1 sibling, 0 replies; 16+ messages in thread From: Li Wang @ 2019-10-17 3:22 UTC (permalink / raw) To: ltp Petr Vorel <pvorel@suse.cz> wrote: ... > nit: I don't that much blank lines, I'd prefer more compact output: > ... > TAGS > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c461cb727d1 > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-2618 > > The explanation(HINT: ...) is needed, otherwise, it will misleading someone not familiar with LTP I think. -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191017/3f153d07/attachment.htm> ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags 2019-10-16 13:21 ` Petr Vorel 2019-10-17 3:22 ` Li Wang @ 2019-10-17 8:38 ` Jan Stancek 1 sibling, 0 replies; 16+ messages in thread From: Jan Stancek @ 2019-10-17 8:38 UTC (permalink / raw) To: ltp ----- Original Message ----- > nit: I don't that much blank lines, I'd prefer more compact output: ... > > TAGS > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c461cb727d1 > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-2618 +1, at least drop some blank lines. With more tags output seems unnecessarily scattered. ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 0/2] Add support for a test tags @ 2019-10-14 11:23 Cyril Hrubis 2019-10-14 11:23 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis 0 siblings, 1 reply; 16+ messages in thread From: Cyril Hrubis @ 2019-10-14 11:23 UTC (permalink / raw) To: ltp From: Cyril Hrubis <chrubis@suse.cz> This is a newest version of the test tags, first patch adds a support for test tags to a test library, the second one annotates tests with tags. Not everything is annotated at the moment but fair amount of tests is included. These patches are also a stepping stone for the test metadata project that will allow us to do much more than this. You can have a look at: https://github.com/metan-ucw/ltp/tree/master/docparse Also if you run a make in the docparse directory it will generate test metadata along with a single html page with nice test metadata description. While the whole project is work-in-progress, the test tags are finished and may add a value even without the rest of the changes, see the first commit for details. Cyril Hrubis (2): lib: Add support for test tags testcases/kernel: Add tags include/tst_test.h | 10 +++ lib/tst_test.c | 77 +++++++++++++++++-- testcases/cve/cve-2014-0196.c | 5 ++ testcases/cve/cve-2015-3290.c | 5 ++ testcases/cve/cve-2016-10044.c | 5 ++ testcases/cve/cve-2016-7042.c | 5 ++ testcases/cve/cve-2016-7117.c | 5 ++ testcases/cve/cve-2017-16939.c | 5 ++ testcases/cve/cve-2017-17052.c | 5 ++ testcases/cve/cve-2017-17053.c | 5 ++ testcases/cve/cve-2017-2618.c | 5 ++ testcases/cve/cve-2017-2671.c | 5 ++ testcases/cve/meltdown.c | 6 +- testcases/cve/stack_clash.c | 4 + testcases/kernel/crypto/af_alg01.c | 5 ++ testcases/kernel/crypto/af_alg02.c | 5 ++ testcases/kernel/crypto/af_alg03.c | 4 + testcases/kernel/crypto/af_alg04.c | 4 + testcases/kernel/crypto/af_alg05.c | 8 +- testcases/kernel/crypto/af_alg06.c | 4 + testcases/kernel/crypto/crypto_user01.c | 6 ++ testcases/kernel/crypto/crypto_user02.c | 4 + testcases/kernel/crypto/pcrypt_aead01.c | 5 ++ testcases/kernel/sound/snd_timer01.c | 6 ++ testcases/kernel/syscalls/accept/accept02.c | 5 ++ testcases/kernel/syscalls/acct/acct02.c | 4 + testcases/kernel/syscalls/add_key/add_key02.c | 5 ++ testcases/kernel/syscalls/bpf/bpf_prog02.c | 4 + .../syscalls/futex/futex_cmp_requeue02.c | 5 ++ testcases/kernel/syscalls/keyctl/keyctl02.c | 4 + testcases/kernel/syscalls/keyctl/keyctl04.c | 5 ++ testcases/kernel/syscalls/keyctl/keyctl05.c | 4 + .../kernel/syscalls/move_pages/move_pages12.c | 6 ++ .../kernel/syscalls/realpath/realpath01.c | 4 + .../syscalls/request_key/request_key05.c | 4 + testcases/kernel/syscalls/sbrk/sbrk03.c | 4 + testcases/kernel/syscalls/sendto/sendto02.c | 4 + .../kernel/syscalls/setrlimit/setrlimit06.c | 4 + 38 files changed, 251 insertions(+), 9 deletions(-) -- 2.21.0 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [LTP] [PATCH 1/2] lib: Add support for test tags 2019-10-14 11:23 [LTP] [PATCH 0/2] Add support for a test tags Cyril Hrubis @ 2019-10-14 11:23 ` Cyril Hrubis 0 siblings, 0 replies; 16+ messages in thread From: Cyril Hrubis @ 2019-10-14 11:23 UTC (permalink / raw) To: ltp From: Cyril Hrubis <chrubis@suse.cz> The newly introduced test tags are generic name-value pairs that can hold test metadata, the intended use for now is to store kernel commit hashes for kernel reproducers as well as CVE ids. The mechanism is however choosen to be very generic so that it's easy to add basically any information later on. As it is the main purpose is to print hints for a test failures. If a test that has been written as a kernel reproducer fails it prints nice URL pointing to a kernel commit that may be missing. Example output: -------------------------------------------------------------------------- tst_test.c:1145: INFO: Timeout per run is 0h 05m 00s add_key02.c:98: FAIL: unexpected error with key type 'asymmetric': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.idmap': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'cifs.spnego': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'pkcs7_test': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'rxrpc_s': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'user': EINVAL add_key02.c:98: FAIL: unexpected error with key type 'logon': EINVAL HINT: This is a regression test for linux kernel, see commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c HINT: This test also tests for CVE-2017-15274, see: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15274 Summary: passed 0 failed 8 skipped 0 warnings 0 -------------------------------------------------------------------------- Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- include/tst_test.h | 10 ++++++ lib/tst_test.c | 77 +++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 80 insertions(+), 7 deletions(-) diff --git a/include/tst_test.h b/include/tst_test.h index 84acf2c59..4a51b6d16 100644 --- a/include/tst_test.h +++ b/include/tst_test.h @@ -108,6 +108,11 @@ int tst_parse_int(const char *str, int *val, int min, int max); int tst_parse_long(const char *str, long *val, long min, long max); int tst_parse_float(const char *str, float *val, float min, float max); +struct tst_tag { + const char *name; + const char *value; +}; + extern unsigned int tst_variant; struct tst_test { @@ -212,6 +217,11 @@ struct tst_test { * NULL-terminated array of capability settings */ struct tst_cap *caps; + + /* + * {NULL, NULL} terminated array of tags. + */ + const struct tst_tag *tags; }; /* diff --git a/lib/tst_test.c b/lib/tst_test.c index 6239acf89..2129f38cb 100644 --- a/lib/tst_test.c +++ b/lib/tst_test.c @@ -31,6 +31,9 @@ #include "old_device.h" #include "old_tmpdir.h" +#define LINUX_GIT_URL "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=" +#define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-" + struct tst_test *tst_test; static const char *tid; @@ -414,6 +417,9 @@ static void print_help(void) { unsigned int i; + fprintf(stderr, "Options\n"); + fprintf(stderr, "-------\n\n"); + for (i = 0; i < ARRAY_SIZE(options); i++) fprintf(stderr, "%s\n", options[i].help); @@ -424,6 +430,27 @@ static void print_help(void) fprintf(stderr, "%s\n", tst_test->options[i].help); } +static void print_test_info(void) +{ + unsigned int i; + const struct tst_tag *tags = tst_test->tags; + + printf("\nTags\n"); + printf("----\n\n"); + + if (tags) { + for (i = 0; tags[i].name; i++) { + if (!strcmp(tags[i].name, "CVE")) + printf(CVE_DB_URL "%s\n", tags[i].value); + else if (!strcmp(tags[i].name, "linux-git")) + printf(LINUX_GIT_URL "%s\n", tags[i].value); + else + printf("%s: %s\n", tags[i].name, tags[i].value); + printf("\n"); + } + } +} + static void check_option_collision(void) { unsigned int i, j; @@ -499,6 +526,7 @@ static void parse_opts(int argc, char *argv[]) break; case 'h': print_help(); + print_test_info(); exit(0); case 'i': iterations = atoi(optarg); @@ -584,26 +612,61 @@ int tst_parse_float(const char *str, float *val, float min, float max) return 0; } +static void print_colored(const char *str) +{ + if (tst_color_enabled(STDOUT_FILENO)) + printf("%s%s%s", ANSI_COLOR_YELLOW, str, ANSI_COLOR_RESET); + else + printf("%s", str); +} + +static void print_failure_hints(void) +{ + unsigned int i; + const struct tst_tag *tags = tst_test->tags; + + if (!tags) + return; + + for (i = 0; tags[i].name; i++) { + if (!strcmp(tags[i].name, "linux-git")) { + printf("\n"); + print_colored("HINT: "); + printf("This is a regression test for linux kernel, see commit:\n\n" + LINUX_GIT_URL "%s\n", tags[i].value); + } + + if (!strcmp(tags[i].name, "CVE")) { + printf("\n"); + print_colored("HINT: "); + printf("This test also tests for CVE-%s, see:\n\n" + CVE_DB_URL "%s\n", tags[i].value, tags[i].value); + } + } +} + static void do_exit(int ret) { if (results) { - printf("\nSummary:\n"); - printf("passed %d\n", results->passed); - printf("failed %d\n", results->failed); - printf("skipped %d\n", results->skipped); - printf("warnings %d\n", results->warnings); - if (results->passed && ret == TCONF) ret = 0; - if (results->failed) + if (results->failed) { ret |= TFAIL; + print_failure_hints(); + } if (results->skipped && !results->passed) ret |= TCONF; if (results->warnings) ret |= TWARN; + + printf("\nSummary:\n"); + printf("passed %d\n", results->passed); + printf("failed %d\n", results->failed); + printf("skipped %d\n", results->skipped); + printf("warnings %d\n", results->warnings); } do_cleanup(); -- 2.21.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-10-17 8:38 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-10-14 11:25 [LTP] [PATCH 0/2] Add support for a test tags Cyril Hrubis 2019-10-14 11:25 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis 2019-10-15 5:12 ` Li Wang 2019-10-15 7:42 ` Richard Palethorpe 2019-10-15 9:57 ` Cyril Hrubis 2019-10-15 9:54 ` Cyril Hrubis 2019-10-16 13:30 ` Petr Vorel 2019-10-16 14:46 ` Cyril Hrubis 2019-10-16 19:43 ` Petr Vorel 2019-10-14 11:25 ` [LTP] [PATCH 2/2] testcases/kernel: Add linux git and CVE tags Cyril Hrubis 2019-10-15 4:54 ` Li Wang 2019-10-15 9:52 ` Cyril Hrubis 2019-10-16 13:21 ` Petr Vorel 2019-10-17 3:22 ` Li Wang 2019-10-17 8:38 ` Jan Stancek -- strict thread matches above, loose matches on Subject: below -- 2019-10-14 11:23 [LTP] [PATCH 0/2] Add support for a test tags Cyril Hrubis 2019-10-14 11:23 ` [LTP] [PATCH 1/2] lib: Add support for " Cyril Hrubis
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.