* [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data
@ 2022-03-16 14:59 Mauro Carvalho Chehab
2022-03-16 14:59 ` [igt-dev] [PATCH v3 01/10] runner: check if it has root permissions Mauro Carvalho Chehab
` (11 more replies)
0 siblings, 12 replies; 14+ messages in thread
From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw)
To: igt-dev
From: Mauro Carvalho Chehab <mchehab@kernel.org>
The gcc compiler has a feature that allows collecting code coverage
in runtime, which is already supported by the Linux Kernel, as described
on its documentation:
https://01.org/linuxgraphics/gfx-docs/drm/dev-tools/gcov.html
Add support for it at the igt_runner and scripts/run-tests.sh.
With such support, once a Kernel is compiled with:
./scripts/config -e DEBUG_FS -e GCOV_KERNEL
And GCOV is enabled by one or more drivers, like doing:
for i in $(find drivers/gpu/drm/ -name Makefile); do sed '1 a GCOV_PROFILE := y' -i $i; done
One can use igt_runner and/or run-tests.sh to collect the code coverage.
The way the implementation works is that the igt_runner has internally
the logic which reset the gcov usage counters. The code capture itself
is done via an external script.
There are two versions of such script:
- scripts/code_cov_capture.sh:
Assumes that the Kernel was built at the same machine, and uses
the lcov tool to generate GCC-independent code coverage data,
in the form of *.info files;
- scripts/code_cov_gather_on_test.py:
Generates a gzipped tarbal with the code coverage counters in
binary format. Such kind of output should then be parsed at
the same machine where the Kernel as built, as its content is not
ony dependent on the Kernel source, but also on the Kernel output
objects.
Either way, both igt_runner and scripts/run-tests.sh could be used
to generate the code coverage results.
So, in order to get the code coverage results, storing results per each
IGT test, one should run either:
./scripts/run-tests.sh -T my_tests.testlist -k ~/linux \
-c scripts/code_cov_gather_on_test.py -P
or:
sudo ./build/runner/igt_runner -o --test-list my_tests.testlist \
--coverage-per-test \
--collect-script scripts/code_cov_gather_on_test.py \
build/tests results
Once the data is captured, it is possible to generate HTML reports
using scripts/code_cov_gen_report.sh:
scripts/code_cov_gen_report.sh -r results/code_cov/i915_selftest_live.info -k ~/linux -o cov_results -i -f
v3:
- Added support at igt_runner to cleanup code coverage results with -o
parameter
v2:
- Applied this commant to patch 8:
sed 's!/home/gta/!/basedir/!;s,\./basedir,/basedir,' -i docs/code_coverage.md
Mauro Carvalho Chehab (9):
runner: check if it has root permissions
runner: Add support for code coverage
runner: cleanup code_cov directory, if any
scripts/code_cov_gather*/sh: add help scripts for code coverage
scripts/code_cov_gather_on_build.sh: Improve the script
scripts/code_cov_capture.sh: add a script to use lcov on build+test
machine
scripts/code_cov_gen_report.sh: add a script to generate code coverage
reports
scripts/run-tests.sh: add code coverage support
docs: add documentation for code coverage
Tomi Sarvela (1):
scripts:code_cov_gather_on_test: use a faster script
docs/code_coverage.md | 293 ++++++++++++++++++++++++++++
runner/executor.c | 235 +++++++++++++++++++++-
runner/resume.c | 4 +-
runner/runner_tests.c | 30 +++
runner/settings.c | 87 ++++++++-
runner/settings.h | 8 +
scripts/code_cov_capture.sh | 25 +++
scripts/code_cov_gather_on_build.sh | 39 ++++
scripts/code_cov_gather_on_test.py | 91 +++++++++
scripts/code_cov_gen_report.sh | 170 ++++++++++++++++
scripts/run-tests.sh | 52 ++++-
11 files changed, 1013 insertions(+), 21 deletions(-)
create mode 100644 docs/code_coverage.md
create mode 100755 scripts/code_cov_capture.sh
create mode 100755 scripts/code_cov_gather_on_build.sh
create mode 100755 scripts/code_cov_gather_on_test.py
create mode 100755 scripts/code_cov_gen_report.sh
--
2.35.1
^ permalink raw reply [flat|nested] 14+ messages in thread* [igt-dev] [PATCH v3 01/10] runner: check if it has root permissions 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 02/10] runner: Add support for code coverage Mauro Carvalho Chehab ` (10 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> Without root permissions, most IGT tests won't actually run, but they would be displayed at the runner's output as if everything went fine. In order to avoid that, check if one attempts to run IGT without root permission. Such check can be disbled with a new command line option: --allow-non-root As runner_tests runs as non-root, most unit tests need to pass --allow-non-root in order for them to not return an error. Reviewed-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- runner/executor.c | 14 +++++++++++++- runner/resume.c | 4 +--- runner/runner_tests.c | 27 +++++++++++++++++++++++++++ runner/settings.c | 8 ++++++++ runner/settings.h | 1 + 5 files changed, 50 insertions(+), 4 deletions(-) diff --git a/runner/executor.c b/runner/executor.c index 9b5821790100..15bd53dd150c 100644 --- a/runner/executor.c +++ b/runner/executor.c @@ -1526,6 +1526,12 @@ bool initialize_execute_state_from_resume(int dirfd, if (!read_settings_from_dir(settings, dirfd) || !read_job_list(list, dirfd)) { close(dirfd); + fprintf(stderr, "Failure reading metadata\n"); + return false; + } + + if (!settings->allow_non_root && (getuid() != 0)) { + fprintf(stderr, "Runner needs to run with UID 0 (root).\n"); return false; } @@ -1573,6 +1579,11 @@ bool initialize_execute_state(struct execute_state *state, struct settings *settings, struct job_list *job_list) { + if (!settings->allow_non_root && (getuid() != 0)) { + fprintf(stderr, "Runner needs to run with UID 0 (root).\n"); + return false; + } + memset(state, 0, sizeof(*state)); if (!validate_settings(settings)) @@ -1840,7 +1851,8 @@ bool execute(struct execute_state *state, } close(sigfd); close(testdirfd); - initialize_execute_state_from_resume(resdirfd, state, settings, job_list); + if (initialize_execute_state_from_resume(resdirfd, state, settings, job_list)) + return false; state->time_left = time_left; return execute(state, settings, job_list); } diff --git a/runner/resume.c b/runner/resume.c index f5d69e218322..eb8074e31ad8 100644 --- a/runner/resume.c +++ b/runner/resume.c @@ -31,10 +31,8 @@ int main(int argc, char **argv) return 127; } - if (!initialize_execute_state_from_resume(dirfd, &state, &settings, &job_list)) { - fprintf(stderr, "Failure reading metadata in %s\n", argv[1]); + if (!initialize_execute_state_from_resume(dirfd, &state, &settings, &job_list)) return 127; - } if (!execute(&state, &settings, &job_list)) { exitcode = 1; diff --git a/runner/runner_tests.c b/runner/runner_tests.c index cd033f6c258f..e67e08a8c99f 100644 --- a/runner/runner_tests.c +++ b/runner/runner_tests.c @@ -112,6 +112,7 @@ static void job_list_filter_test(const char *name, const char *filterarg1, const igt_subtest_f("job-list-filters-%s-%s", name, multiple ? "multiple" : "normal") { struct job_list list; const char *argv[] = { "runner", + "--allow-non-root", /* Ugly but does the trick */ multiple ? "--multiple-mode" : "--sync", filterarg1, filterarg2, @@ -187,6 +188,7 @@ static void assert_settings_equal(struct settings *one, struct settings *two) igt_assert_eqstr(one->test_list, two->test_list); igt_assert_eqstr(one->name, two->name); igt_assert_eq(one->dry_run, two->dry_run); + igt_assert_eq(one->allow_non_root, two->allow_non_root); igt_assert_eq(one->sync, two->sync); igt_assert_eq(one->log_level, two->log_level); igt_assert_eq(one->overwrite, two->overwrite); @@ -264,6 +266,7 @@ igt_main igt_subtest("default-settings") { const char *argv[] = { "runner", + "--allow-non-root", "test-root-dir", "path-to-results", }; @@ -348,6 +351,7 @@ igt_main igt_subtest("absolute-path-usage") { const char *argv[] = { "runner", + "--allow-non-root", "--test-list", pathtotestlist, testdatadir, dirname, @@ -382,6 +386,7 @@ igt_main igt_subtest("environment-overrides-test-root-flag") { const char *argv[] = { "runner", + "--allow-non-root", "test-root-dir", "path-to-results", }; @@ -415,6 +420,7 @@ igt_main igt_subtest("parse-all-settings") { char blacklist_name[PATH_MAX], blacklist2_name[PATH_MAX]; const char *argv[] = { "runner", + "--allow-non-root", "-n", "foo", "--abort-on-monitored-error=taint,lockdep", "--disk-usage-limit=4096", @@ -451,6 +457,7 @@ igt_main igt_assert(strstr(settings->test_list, "path-to-test-list") != NULL); igt_assert_eqstr(settings->name, "foo"); igt_assert(settings->dry_run); + igt_assert(settings->allow_non_root); igt_assert_eq(settings->include_regexes.size, 2); igt_assert_eqstr(settings->include_regexes.regex_strings[0], "pattern1"); igt_assert_eqstr(settings->include_regexes.regex_strings[1], "pattern2"); @@ -484,6 +491,7 @@ igt_main igt_subtest("dmesg-warn-level-inferred") { const char *argv[] = { "runner", + "--allow-non-root", "test-root-dir", "path-to-results", }; @@ -496,6 +504,7 @@ igt_main igt_subtest("dmesg-warn-level-inferred-with-piglit-style") { const char *argv[] = { "runner", + "--allow-non-root", "--piglit-style-dmesg", "test-root-dir", "path-to-results", @@ -509,6 +518,7 @@ igt_main igt_subtest("dmesg-warn-level-overridable-with-piglit-style") { const char *argv[] = { "runner", + "--allow-non-root", "--piglit-style-dmesg", "--dmesg-warn-level=3", "test-root-dir", @@ -560,6 +570,7 @@ igt_main igt_subtest("abort-conditions") { const char *argv[] = { "runner", "--abort-on-monitored-error=taint", + "--allow-non-root", "test-root-dir", "results-path", }; @@ -623,6 +634,7 @@ igt_main const char *argv[] = { "runner", "-n", "foo", "--dry-run", + "--allow-non-root", "test-root-dir", "results-path", }; @@ -656,6 +668,7 @@ igt_main igt_subtest("validate-ok") { const char *argv[] = { "runner", + "--allow-non-root", "--test-list", filename, testdatadir, "path-to-results", @@ -983,6 +996,7 @@ igt_main struct execute_state state; const char *argv[] = { "runner", "--dry-run", + "--allow-non-root", "-x", "^abort", testdatadir, dirname, @@ -1055,6 +1069,7 @@ igt_main igt_subtest("execute-initialize-new-run") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", testdatadir, dirname, }; @@ -1102,6 +1117,7 @@ igt_main igt_subtest("execute-initialize-subtest-started") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", "--multiple-mode", "-t", "successtest", testdatadir, @@ -1158,6 +1174,7 @@ igt_main igt_subtest("execute-initialize-all-subtests-started") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", "--multiple-mode", "-t", "successtest@first-subtest", "-t", "successtest@second-subtest", @@ -1213,6 +1230,7 @@ igt_main igt_subtest("execute-initialize-subtests-complete") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", "--multiple-mode", testdatadir, dirname, @@ -1278,6 +1296,7 @@ igt_main igt_subtest_f("execute-subtests-%s", multiple ? "multiple" : "normal") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", multiple ? "--multiple-mode" : "--sync", "-t", "successtest.*-subtest", testdatadir, @@ -1413,6 +1432,7 @@ igt_main igt_subtest_f("execute-skipper-journal-%s", multiple ? "multiple" : "normal") { struct execute_state state; const char *argv[] = { "runner", + "--allow-non-root", multiple ? "--multiple-mode" : "--sync", "-t", "skippers", testdatadir, @@ -1499,6 +1519,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", + "--allow-non-root", "--test-list", filename, testdatadir, dirname, @@ -1554,6 +1575,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", + "--allow-non-root", "-t", "^dynamic$", testdatadir, dirname, @@ -1599,6 +1621,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", + "--allow-non-root", "-t", "^abort-simple$", testdatadir, dirname, @@ -1646,6 +1669,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", + "--allow-non-root", "-t", "^abort$", multiple ? "--multiple-mode" : "--sync", testdatadir, @@ -1701,6 +1725,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", multiple ? "--multiple-mode" : "--sync", + "--allow-non-root", "-t", "^abort-fixture$", testdatadir, dirname, @@ -1767,6 +1792,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", multiple ? "--multiple-mode" : "--sync", + "--allow-non-root", "--test-list", filename, testdatadir, dirname, @@ -1822,6 +1848,7 @@ igt_main struct execute_state state; struct json_object *results, *tests; const char *argv[] = { "runner", multiple ? "--multiple-mode" : "--sync", + "--allow-non-root", "-t", "^abort-dynamic$", testdatadir, dirname, diff --git a/runner/settings.c b/runner/settings.c index 200f1ce59ed6..ac090b081e3b 100644 --- a/runner/settings.c +++ b/runner/settings.c @@ -24,6 +24,7 @@ enum { OPT_DMESG_WARN_LEVEL, OPT_OVERALL_TIMEOUT, OPT_PER_TEST_TIMEOUT, + OPT_ALLOW_NON_ROOT, OPT_VERSION, OPT_HELP = 'h', OPT_NAME = 'n', @@ -199,6 +200,7 @@ static const char *usage_str = " --ignore-missing Ignored but accepted, for piglit compatibility\n" "\n" " Incompatible options:\n" + " --allow-non-root Allow running tests without being the root user.\n" " -m, --multiple-mode Run multiple subtests in the same binary execution.\n" " If a testlist file is given, consecutive subtests are\n" " run in the same execution if they are from the same\n" @@ -381,6 +383,7 @@ bool parse_options(int argc, char **argv, {"help", no_argument, NULL, OPT_HELP}, {"name", required_argument, NULL, OPT_NAME}, {"dry-run", no_argument, NULL, OPT_DRY_RUN}, + {"allow-non-root", no_argument, NULL, OPT_ALLOW_NON_ROOT}, {"include-tests", required_argument, NULL, OPT_INCLUDE}, {"exclude-tests", required_argument, NULL, OPT_EXCLUDE}, {"abort-on-monitored-error", optional_argument, NULL, OPT_ABORT_ON_ERROR}, @@ -423,6 +426,9 @@ bool parse_options(int argc, char **argv, case OPT_DRY_RUN: settings->dry_run = true; break; + case OPT_ALLOW_NON_ROOT: + settings->allow_non_root = true; + break; case OPT_INCLUDE: if (!add_regex(&settings->include_regexes, strdup(optarg))) goto error; @@ -693,6 +699,7 @@ bool serialize_settings(struct settings *settings) if (settings->name) SERIALIZE_LINE(f, settings, name, "%s"); SERIALIZE_LINE(f, settings, dry_run, "%d"); + SERIALIZE_LINE(f, settings, allow_non_root, "%d"); SERIALIZE_LINE(f, settings, sync, "%d"); SERIALIZE_LINE(f, settings, log_level, "%d"); SERIALIZE_LINE(f, settings, overwrite, "%d"); @@ -740,6 +747,7 @@ bool read_settings_from_file(struct settings *settings, FILE *f) PARSE_LINE(settings, name, val, test_list, val ? strdup(val) : NULL); PARSE_LINE(settings, name, val, name, val ? strdup(val) : NULL); PARSE_LINE(settings, name, val, dry_run, numval); + PARSE_LINE(settings, name, val, allow_non_root, numval); PARSE_LINE(settings, name, val, sync, numval); PARSE_LINE(settings, name, val, log_level, numval); PARSE_LINE(settings, name, val, overwrite, numval); diff --git a/runner/settings.h b/runner/settings.h index 409391f9b9de..bc61faeb6c86 100644 --- a/runner/settings.h +++ b/runner/settings.h @@ -32,6 +32,7 @@ struct settings { char *test_list; char *name; bool dry_run; + bool allow_non_root; struct regex_list include_regexes; struct regex_list exclude_regexes; bool sync; -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 02/10] runner: Add support for code coverage 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 01/10] runner: check if it has root permissions Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 03/10] runner: cleanup code_cov directory, if any Mauro Carvalho Chehab ` (9 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> The gcc compiler has a feature that enables checking the code coverage in runtime[1]. [1] See https://www.kernel.org/doc/html/latest/dev-tools/gcov.html The Linux Kernel comes with an option to enable such feature: ./scripts/config -e DEBUG_FS -e GCOV_KERNEL The driver's Makefile also needs change to enable it. For instance, in order to enable GCOV for all DRM drivers, one would need to run: for i in $(find drivers/gpu/drm/ -name Makefile); do sed '1 a GCOV_PROFILE := y' -i $i done This patch adds support for it by: a) Implementing a logic to cleanup the code coverage counters via sysfs; b) Calling a script responsible for collecging code coverage data. The implementation works with two modes: 1) It zeroes the counters, run all IGT tests and collects the code coverage results at the end. This implies that no tests would crash the driver, as otherwise the results won't be collected; This is faster, as collecting code coverage data can take several seconds. 2) For each test, it will clean the code coverage counters, run the and collect the results. This is more reliable, as a Kernel crash/OOPS won't affect the results of the previously ran tests. Reviewed-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- runner/executor.c | 186 ++++++++++++++++++++++++++++++++++++++++-- runner/runner_tests.c | 3 + runner/settings.c | 79 +++++++++++++++++- runner/settings.h | 7 ++ 4 files changed, 265 insertions(+), 10 deletions(-) diff --git a/runner/executor.c b/runner/executor.c index 15bd53dd150c..ab00900c6d44 100644 --- a/runner/executor.c +++ b/runner/executor.c @@ -1,3 +1,4 @@ +#include <ctype.h> #include <errno.h> #include <fcntl.h> #include <glib.h> @@ -1693,6 +1694,140 @@ static bool should_die_because_signal(int sigfd) return false; } +static char *code_coverage_name(struct settings *settings) +{ + const char *start, *end, *fname; + char *name; + int size; + + if (settings->name && *settings->name) + return settings->name; + else if (!settings->test_list) + return NULL; + + /* Use only the base of the test_list, without path and extension */ + fname = settings->test_list; + + start = strrchr(fname,'/'); + if (!start) + start = fname; + + end = strrchr(start, '.'); + if (end) + size = end - start; + else + size = strlen(start); + + name = malloc(size + 1); + strncpy(name, fname, size); + name[size] = '\0'; + + return name; +} + +static void run_as_root(char * const argv[], int sigfd, char **abortreason) +{ + struct signalfd_siginfo siginfo; + int status = 0, ret; + pid_t child; + + child = fork(); + if (child < 0) { + *abortreason = strdup("Failed to fork"); + return; + } + + if (child == 0) { + execv(argv[0], argv); + perror (argv[0]); + exit(IGT_EXIT_INVALID); + } + + if (sigfd >= 0) { + while (1) { + ret = read(sigfd, &siginfo, sizeof(siginfo)); + if (ret < 0) { + errf("Error reading from signalfd: %m\n"); + continue; + } else if (siginfo.ssi_signo == SIGCHLD) { + if (child != waitpid(child, &status, WNOHANG)) { + errf("Failed to reap child\n"); + status = 9999; + continue; + } + break; + } + } + } else { + waitpid(child, &status, 0); + } + + if (WIFSIGNALED(status)) + asprintf(abortreason, "%s received signal %d while running\n",argv[0], WTERMSIG(status)); + else if (!WIFEXITED(status)) + asprintf(abortreason, "%s aborted with unknown status\n", argv[0]); + else if (WEXITSTATUS(status)) + asprintf(abortreason, "%s returned error %d\n", argv[0], WEXITSTATUS(status)); +} + +static void code_coverage_start(struct settings *settings, int sigfd, char **abortreason) +{ + int fd; + + fd = open(GCOV_RESET, O_WRONLY); + if (fd < 0) { + asprintf(abortreason, "Failed to open %s", GCOV_RESET); + return; + } + if (write(fd, "0\n", 2) < 0) + *abortreason = strdup("Failed to reset gcov counters"); + + close(fd); +} + +static void code_coverage_stop(struct settings *settings, const char *job_name, + int sigfd, char **abortreason) +{ + int i, j = 0, last_was_escaped = 1; + char fname[PATH_MAX]; + char name[PATH_MAX]; + char *argv[3] = {}; + + /* If name is empty, use a default */ + if (!job_name || !*job_name) + job_name = "code_coverage"; + + /* + * Use only letters, numbers and '_' + * + * This way, the tarball name can be used as testname when lcov runs + */ + for (i = 0; i < strlen(job_name); i++) { + if (!isalpha(job_name[i]) && !isalnum(job_name[i])) { + if (last_was_escaped) + continue; + name[j++] = '_'; + last_was_escaped = 1; + } else { + name[j++] = job_name[i]; + last_was_escaped = 0; + } + } + if (j && last_was_escaped) + j--; + name[j] = '\0'; + + strcpy(fname, settings->results_path); + strcat(fname, CODE_COV_RESULTS_PATH "/"); + strcat(fname, name); + + argv[0] = settings->code_coverage_script; + argv[1] = fname; + + outf("Storing code coverage results...\n"); + run_as_root(argv, sigfd, abortreason); +} + bool execute(struct execute_state *state, struct settings *settings, struct job_list *job_list) @@ -1709,6 +1844,17 @@ bool execute(struct execute_state *state, return true; } + if (settings->enable_code_coverage && !settings->cov_results_per_test) { + char *reason = NULL; + + code_coverage_start(settings, -1, &reason); + if (reason != NULL) { + errf("%s\n", reason); + free(reason); + status = false; + } + } + if ((resdirfd = open(settings->results_path, O_DIRECTORY | O_RDONLY)) < 0) { /* Initialize state should have done this */ errf("Error: Failure opening results path %s\n", @@ -1795,6 +1941,7 @@ bool execute(struct execute_state *state, for (; state->next < job_list->size; state->next++) { char *reason = NULL; + char *job_name; int result; if (should_die_because_signal(sigfd)) { @@ -1802,14 +1949,26 @@ bool execute(struct execute_state *state, goto end; } - result = execute_next_entry(state, - job_list->size, - &time_spent, - settings, - &job_list->entries[state->next], - testdirfd, resdirfd, - sigfd, &sigmask, - &reason); + if (settings->cov_results_per_test) { + code_coverage_start(settings, sigfd, &reason); + job_name = entry_display_name(&job_list->entries[state->next]); + } + + if (reason == NULL) { + result = execute_next_entry(state, + job_list->size, + &time_spent, + settings, + &job_list->entries[state->next], + testdirfd, resdirfd, + sigfd, &sigmask, + &reason); + + if (settings->cov_results_per_test) { + code_coverage_stop(settings, job_name, sigfd, &reason); + free(job_name); + } + } if (reason != NULL || (reason = need_to_abort(settings)) != NULL) { char *prev = entry_display_name(&job_list->entries[state->next]); @@ -1864,6 +2023,17 @@ bool execute(struct execute_state *state, } end: + if (settings->enable_code_coverage && !settings->cov_results_per_test) { + char *reason = NULL; + + code_coverage_stop(settings, code_coverage_name(settings), -1, &reason); + if (reason != NULL) { + errf("%s\n", reason); + free(reason); + status = false; + } + } + close_watchdogs(settings); sigprocmask(SIG_UNBLOCK, &sigmask, NULL); /* make sure that we do not leave any signals unhandled */ diff --git a/runner/runner_tests.c b/runner/runner_tests.c index e67e08a8c99f..96ffbf1ff398 100644 --- a/runner/runner_tests.c +++ b/runner/runner_tests.c @@ -443,6 +443,9 @@ igt_main "--use-watchdog", "--piglit-style-dmesg", "--dmesg-warn-level=3", + "--collect-code-cov", + "--coverage-per-test", + "--collect-script", "/usr/bin/true", "test-root-dir", "path-to-results", }; diff --git a/runner/settings.c b/runner/settings.c index ac090b081e3b..a7a12f506a05 100644 --- a/runner/settings.c +++ b/runner/settings.c @@ -25,6 +25,9 @@ enum { OPT_OVERALL_TIMEOUT, OPT_PER_TEST_TIMEOUT, OPT_ALLOW_NON_ROOT, + OPT_CODE_COV_SCRIPT, + OPT_ENABLE_CODE_COVERAGE, + OPT_COV_RESULTS_PER_TEST, OPT_VERSION, OPT_HELP = 'h', OPT_NAME = 'n', @@ -240,6 +243,13 @@ static const char *usage_str = " Exclude all test matching to regexes from FILENAME\n" " (can be used more than once)\n" " -L, --list-all List all matching subtests instead of running\n" + " --collect-code-cov Enables gcov-based collect of code coverage for tests.\n" + " Requires --collect-script FILENAME\n" + " --coverage-per-test Stores code coverage results per each test.\n" + " Requires --collect-script FILENAME\n" + " --collect-script FILENAME\n" + " Use FILENAME as script to collect code coverage data.\n" + "\n" " [test_root] Directory that contains the IGT tests. The environment\n" " variable IGT_TEST_ROOT will be used if set, overriding\n" " this option if given.\n" @@ -338,11 +348,21 @@ static void free_regexes(struct regex_list *regexes) free(regexes->regexes); } -static bool readable_file(char *filename) +static bool readable_file(const char *filename) { return !access(filename, R_OK); } +static bool writeable_file(const char *filename) +{ + return !access(filename, W_OK); +} + +static bool executable_file(const char *filename) +{ + return !access(filename, X_OK); +} + static void print_version(void) { struct utsname uts; @@ -393,6 +413,9 @@ bool parse_options(int argc, char **argv, {"test-list", required_argument, NULL, OPT_TEST_LIST}, {"overwrite", no_argument, NULL, OPT_OVERWRITE}, {"ignore-missing", no_argument, NULL, OPT_IGNORE_MISSING}, + {"collect-code-cov", no_argument, NULL, OPT_ENABLE_CODE_COVERAGE}, + {"coverage-per-test", no_argument, NULL, OPT_COV_RESULTS_PER_TEST}, + {"collect-script", required_argument, NULL, OPT_CODE_COV_SCRIPT}, {"multiple-mode", no_argument, NULL, OPT_MULTIPLE}, {"inactivity-timeout", required_argument, NULL, OPT_TIMEOUT}, {"per-test-timeout", required_argument, NULL, OPT_PER_TEST_TIMEOUT}, @@ -465,6 +488,16 @@ bool parse_options(int argc, char **argv, case OPT_IGNORE_MISSING: /* Ignored, piglit compatibility */ break; + case OPT_ENABLE_CODE_COVERAGE: + settings->enable_code_coverage = true; + break; + case OPT_COV_RESULTS_PER_TEST: + settings->cov_results_per_test = true; + break; + case OPT_CODE_COV_SCRIPT: + settings->code_coverage_script = absolute_path(optarg); + break; + case OPT_MULTIPLE: settings->multiple_mode = true; break; @@ -597,6 +630,29 @@ bool validate_settings(struct settings *settings) close(fd); close(dirfd); + /* enables code coverage when --coverage-per-test is used */ + if (settings->cov_results_per_test) + settings->enable_code_coverage = true; + + if (!settings->allow_non_root && (getuid() != 0)) { + fprintf(stderr, "Runner needs to run with UID 0 (root).\n"); + return false; + } + + if (settings->enable_code_coverage) { + if (!executable_file(settings->code_coverage_script)) { + fprintf(stderr, "%s doesn't exist or is not executable\n", settings->code_coverage_script); + return false; + } + if (!writeable_file(GCOV_RESET)) { + if (getuid() != 0) + fprintf(stderr, "Code coverage requires root.\n"); + else + fprintf(stderr, "Is GCOV enabled? Can't access %s stat.\n", GCOV_RESET); + return false; + } + } + return true; } @@ -645,7 +701,8 @@ bool serialize_settings(struct settings *settings) { #define SERIALIZE_LINE(f, s, name, format) fprintf(f, "%s : " format "\n", #name, s->name) - int dirfd, fd; + int dirfd, covfd, fd; + char path[PATH_MAX]; FILE *f; if (!settings->results_path) { @@ -660,6 +717,18 @@ bool serialize_settings(struct settings *settings) return false; } } + if (settings->enable_code_coverage) { + strcpy(path, settings->results_path); + strcat(path, CODE_COV_RESULTS_PATH); + if ((covfd = open(path, O_DIRECTORY | O_RDONLY)) < 0) { + if (mkdir(path, 0755)) { + usage("Creating code coverage path failed", stderr); + return false; + } + } else { + close(covfd); + } + } if (!settings->overwrite && faccessat(dirfd, settings_filename, F_OK, 0) == 0) { @@ -712,6 +781,9 @@ bool serialize_settings(struct settings *settings) SERIALIZE_LINE(f, settings, dmesg_warn_level, "%d"); SERIALIZE_LINE(f, settings, test_root, "%s"); SERIALIZE_LINE(f, settings, results_path, "%s"); + SERIALIZE_LINE(f, settings, enable_code_coverage, "%d"); + SERIALIZE_LINE(f, settings, cov_results_per_test, "%d"); + SERIALIZE_LINE(f, settings, code_coverage_script, "%s"); if (settings->sync) { fsync(fd); @@ -760,6 +832,9 @@ bool read_settings_from_file(struct settings *settings, FILE *f) PARSE_LINE(settings, name, val, dmesg_warn_level, numval); PARSE_LINE(settings, name, val, test_root, val ? strdup(val) : NULL); PARSE_LINE(settings, name, val, results_path, val ? strdup(val) : NULL); + PARSE_LINE(settings, name, val, enable_code_coverage, numval); + PARSE_LINE(settings, name, val, cov_results_per_test, numval); + PARSE_LINE(settings, name, val, code_coverage_script, val ? strdup(val) : NULL); printf("Warning: Unknown field in settings file: %s = %s\n", name, val); diff --git a/runner/settings.h b/runner/settings.h index bc61faeb6c86..3c2a3ee9c336 100644 --- a/runner/settings.h +++ b/runner/settings.h @@ -20,6 +20,10 @@ enum { _Static_assert(ABORT_ALL == (ABORT_TAINT | ABORT_LOCKDEP | ABORT_PING), "ABORT_ALL must be all conditions bitwise or'd"); +#define GCOV_DIR "/sys/kernel/debug/gcov" +#define GCOV_RESET GCOV_DIR "/reset" +#define CODE_COV_RESULTS_PATH "/code_cov" + struct regex_list { char **regex_strings; GRegex **regexes; @@ -48,6 +52,9 @@ struct settings { bool piglit_style_dmesg; int dmesg_warn_level; bool list_all; + char *code_coverage_script; + bool enable_code_coverage; + bool cov_results_per_test; }; /** -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 03/10] runner: cleanup code_cov directory, if any 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 01/10] runner: check if it has root permissions Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 02/10] runner: Add support for code coverage Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 04/10] scripts/code_cov_gather*/sh: add help scripts for code coverage Mauro Carvalho Chehab ` (8 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> Ensure that "-o" parameter will also cleanup the contents of the code coverage results directory. Reviewed-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- runner/executor.c | 59 ++++++++++++++++++++++++++++++++++++----------- runner/settings.h | 2 +- 2 files changed, 47 insertions(+), 14 deletions(-) diff --git a/runner/executor.c b/runner/executor.c index ab00900c6d44..b4efc4fdbf81 100644 --- a/runner/executor.c +++ b/runner/executor.c @@ -20,6 +20,7 @@ #include <sys/types.h> #include <sys/utsname.h> #include <sys/wait.h> +#include <dirent.h> #include <time.h> #include <unistd.h> @@ -1446,8 +1447,11 @@ static bool clear_test_result_directory(int dirfd) static bool clear_old_results(char *path) { + struct dirent *entry; + char name[PATH_MAX]; int dirfd; size_t i; + DIR *dir; if ((dirfd = open(path, O_DIRECTORY | O_RDONLY)) < 0) { if (errno == ENOENT) { @@ -1469,7 +1473,6 @@ static bool clear_old_results(char *path) } for (i = 0; true; i++) { - char name[32]; int resdirfd; snprintf(name, sizeof(name), "%zd", i); @@ -1488,6 +1491,31 @@ static bool clear_old_results(char *path) } } + strcpy(name, path); + strcat(name, "/" CODE_COV_RESULTS_PATH); + if ((dir = opendir(name)) != NULL) { + char *p; + + strcat(name, "/"); + p = name + strlen(name); + + while ((entry = readdir(dir)) != NULL) { + if (!strcmp(entry->d_name, ".") || !strcmp(entry->d_name, "..")) + continue; + + strcpy(p, entry->d_name); + if (unlink(name)) { + errf("Error removing %s\n", name); + } + } + + closedir(dir); + if (unlinkat(dirfd, CODE_COV_RESULTS_PATH, AT_REMOVEDIR)) { + errf("Warning: Result directory %s/%s contains extra files\n", + path, CODE_COV_RESULTS_PATH); + } + } + close(dirfd); return true; @@ -1818,7 +1846,7 @@ static void code_coverage_stop(struct settings *settings, const char *job_name, name[j] = '\0'; strcpy(fname, settings->results_path); - strcat(fname, CODE_COV_RESULTS_PATH "/"); + strcat(fname, "/" CODE_COV_RESULTS_PATH "/"); strcat(fname, name); argv[0] = settings->code_coverage_script; @@ -1844,17 +1872,6 @@ bool execute(struct execute_state *state, return true; } - if (settings->enable_code_coverage && !settings->cov_results_per_test) { - char *reason = NULL; - - code_coverage_start(settings, -1, &reason); - if (reason != NULL) { - errf("%s\n", reason); - free(reason); - status = false; - } - } - if ((resdirfd = open(settings->results_path, O_DIRECTORY | O_RDONLY)) < 0) { /* Initialize state should have done this */ errf("Error: Failure opening results path %s\n", @@ -1862,6 +1879,22 @@ bool execute(struct execute_state *state, return false; } + if (settings->enable_code_coverage) { + if (!settings->cov_results_per_test) { + char *reason = NULL; + + code_coverage_start(settings, -1, &reason); + if (reason != NULL) { + errf("%s\n", reason); + free(reason); + close(resdirfd); + return false; + } + } + + mkdirat(resdirfd, CODE_COV_RESULTS_PATH, 0755); + } + if ((testdirfd = open(settings->test_root, O_DIRECTORY | O_RDONLY)) < 0) { errf("Error: Failure opening test root %s\n", settings->test_root); diff --git a/runner/settings.h b/runner/settings.h index 3c2a3ee9c336..bbd965d27882 100644 --- a/runner/settings.h +++ b/runner/settings.h @@ -22,7 +22,7 @@ _Static_assert(ABORT_ALL == (ABORT_TAINT | ABORT_LOCKDEP | ABORT_PING), "ABORT_A #define GCOV_DIR "/sys/kernel/debug/gcov" #define GCOV_RESET GCOV_DIR "/reset" -#define CODE_COV_RESULTS_PATH "/code_cov" +#define CODE_COV_RESULTS_PATH "code_cov" struct regex_list { char **regex_strings; -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 04/10] scripts/code_cov_gather*/sh: add help scripts for code coverage 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (2 preceding siblings ...) 2022-03-16 14:59 ` [igt-dev] [PATCH v3 03/10] runner: cleanup code_cov directory, if any Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 05/10] scripts/code_cov_gather_on_build.sh: Improve the script Mauro Carvalho Chehab ` (7 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> When a Linux Kernel is built with gcov support, the compiler creates some other object files (*.gcno) that are dependent of the compiler version, and contain references to the source files. At Kernel runtime at the test machine, counter files will be visible at sysfs (*.gcda files), which are also on a compiler specific format. In order to be able to properly parse the contents of the counters, the information from 3 different places should be merged altogether: - Runtime counters: /sys/.../*.gcda and hiperlinks to *.gcno - Compile-time cross-reference object files: *.gcno - Kernel source code. If the build machine is different than the source machine, some special scripts are needed in order to either copy the source files to the runtime machine or vice versa. This is described at: https://www.kernel.org/doc/html/latest/dev-tools/gcov.html Copy the two scripts described there as-is. Further patches will modify them as needed. Acked-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/code_cov_gather_on_build.sh | 23 +++++++++++++++++++++++ scripts/code_cov_gather_on_test.sh | 20 ++++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100755 scripts/code_cov_gather_on_build.sh create mode 100755 scripts/code_cov_gather_on_test.sh diff --git a/scripts/code_cov_gather_on_build.sh b/scripts/code_cov_gather_on_build.sh new file mode 100755 index 000000000000..99c436f76fb8 --- /dev/null +++ b/scripts/code_cov_gather_on_build.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +KSRC=$1 +KOBJ=$2 +DEST=$3 + +if [ -z "$KSRC" ] || [ -z "$KOBJ" ] || [ -z "$DEST" ]; then + echo "Usage: $0 <ksrc directory> <kobj directory> <output.tar.gz>" >&2 + exit 1 +fi + +KSRC=$(cd $KSRC; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) +KOBJ=$(cd $KOBJ; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) + +find $KSRC $KOBJ \( -name '*.gcno' -o -name '*.[ch]' -o -type l \) -a \ + -perm /u+r,g+r | tar cfz $DEST -P -T - + +if [ $? -eq 0 ] ; then + echo "$DEST successfully created, copy to test system and unpack with:" + echo " tar xfz $DEST -P" +else + echo "Could not create file $DEST" +fi diff --git a/scripts/code_cov_gather_on_test.sh b/scripts/code_cov_gather_on_test.sh new file mode 100755 index 000000000000..8834aa0d78af --- /dev/null +++ b/scripts/code_cov_gather_on_test.sh @@ -0,0 +1,20 @@ +#!/bin/bash -e + +DEST=$1 +GCDA=/sys/kernel/debug/gcov + +if [ -z "$DEST" ] ; then + echo "Usage: $0 <output.tar.gz>" >&2 + exit 1 +fi + +TEMPDIR=$(mktemp -d) +echo Collecting data.. +find $GCDA -type d -exec mkdir -p $TEMPDIR/\{\} \; +find $GCDA -name '*.gcda' -exec sh -c 'cat < $0 > '$TEMPDIR'/$0' {} \; +find $GCDA -name '*.gcno' -exec sh -c 'cp -d $0 '$TEMPDIR'/$0' {} \; +tar czf $DEST -C $TEMPDIR sys +rm -rf $TEMPDIR + +echo "$DEST successfully created, copy to build system and unpack with:" +echo " tar xfz $DEST" -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 05/10] scripts/code_cov_gather_on_build.sh: Improve the script 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (3 preceding siblings ...) 2022-03-16 14:59 ` [igt-dev] [PATCH v3 04/10] scripts/code_cov_gather*/sh: add help scripts for code coverage Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 06/10] scripts/code_cov_capture.sh: add a script to use lcov on build+test machine Mauro Carvalho Chehab ` (6 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> - speedup the script by allowing Sparse file detection instead of libz compression; - use realpath for the paths to allow running them from everywhere; - Speedup tarball generation by limiting the source file scope to the DRM subsystem; - Indent with tabs instead of spaces. Reviewed-by: Tomi Sarvela <tomi.p.sarvela@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/code_cov_gather_on_build.sh | 34 +++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/scripts/code_cov_gather_on_build.sh b/scripts/code_cov_gather_on_build.sh index 99c436f76fb8..eebee89f9f1c 100755 --- a/scripts/code_cov_gather_on_build.sh +++ b/scripts/code_cov_gather_on_build.sh @@ -4,20 +4,36 @@ KSRC=$1 KOBJ=$2 DEST=$3 +# Limit scope in order to speedup tarball creation +GCOV_SCOPE=drivers/gpu/drm/ + if [ -z "$KSRC" ] || [ -z "$KOBJ" ] || [ -z "$DEST" ]; then - echo "Usage: $0 <ksrc directory> <kobj directory> <output.tar.gz>" >&2 - exit 1 + echo "Usage: $0 <ksrc directory> <kobj directory> <output.tar[.gz]>" >&2 + exit 1 +fi + +if [ "x$(echo $DEST|grep '\.gz')" != "x" ]; then + TAR_COMPRESS="z" +else + # gcno files are sparsed. So, if no compression is used, store the + # results as a sparse file, in order to save disk space + TAR_COMPRESS="S" fi -KSRC=$(cd $KSRC; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) -KOBJ=$(cd $KOBJ; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) +KSRC=$(realpath $KSRC) +KOBJ=$(realpath $KOBJ) + +# Source files +SRCS="${KSRC}/include ${KSRC}/arch/x86/include $(find ${KSRC}/${GCOV_SCOPE} -name '*.[ch]')" + +# Generated gcno files and links +OBJS="$(find $KOBJ/${GCOV_SCOPE} \( -name '*.gcno' -o -name '*.[ch]' -o -type l \) -a -perm /u+r,g+r)" -find $KSRC $KOBJ \( -name '*.gcno' -o -name '*.[ch]' -o -type l \) -a \ - -perm /u+r,g+r | tar cfz $DEST -P -T - +tar cf${TAR_COMPRESS} $DEST $SRCS $OBJS if [ $? -eq 0 ] ; then - echo "$DEST successfully created, copy to test system and unpack with:" - echo " tar xfz $DEST -P" + echo "$DEST successfully created, copy to test system and unpack with:" + echo " tar xf${TAR_COMPRESS} $DEST" else - echo "Could not create file $DEST" + echo "Could not create file $DEST" fi -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 06/10] scripts/code_cov_capture.sh: add a script to use lcov on build+test machine 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (4 preceding siblings ...) 2022-03-16 14:59 ` [igt-dev] [PATCH v3 05/10] scripts/code_cov_gather_on_build.sh: Improve the script Mauro Carvalho Chehab @ 2022-03-16 14:59 ` Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 07/10] scripts/code_cov_gen_report.sh: add a script to generate code coverage reports Mauro Carvalho Chehab ` (5 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 14:59 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> When the test machine is also the build machine, it is possible to run lcov directly on it. That makes the IGT tests faster, as it won't need to generate a results tarball, and the produced results will be independent on the Kernel object files, making the output files more portable. It should be noticed that, in order to generate html files, the Kernel source will still be needed, due to the detailed view. Reviewed-by: Tomi Sarvela <tomi.p.sarvela@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/code_cov_capture.sh | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100755 scripts/code_cov_capture.sh diff --git a/scripts/code_cov_capture.sh b/scripts/code_cov_capture.sh new file mode 100755 index 000000000000..8662b0f145b8 --- /dev/null +++ b/scripts/code_cov_capture.sh @@ -0,0 +1,25 @@ +#!/bin/bash + +trap 'catch $LINENO' ERR +catch() { + echo "$0: error on line $1. Code coverage not stored." >&2 + exit 1 +} + +if [ -z "$IGT_KERNEL_TREE" ] ; then + echo "Error! IGT_KERNEL_TREE environment var was not defined." >&2 + exit 1 +fi + +if [ -z "$1" ] ; then + echo "Usage: $0 <output>" >&2 + echo " Please notice that this script assumes $IGT_KERNEL_TREE is used as both Kernel source and object dir." >&2 + exit 1 +fi + +lcov -q --rc lcov_branch_coverage=1 \ + --test-name "$(basename $1)" -b $IGT_KERNEL_TREE --capture \ + --output-file $1.info + +uptime=$(cat /proc/uptime|cut -d' ' -f 1) +echo "[$uptime] Code coverage wrote to $1.info" -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 07/10] scripts/code_cov_gen_report.sh: add a script to generate code coverage reports 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (5 preceding siblings ...) 2022-03-16 14:59 ` [igt-dev] [PATCH v3 06/10] scripts/code_cov_capture.sh: add a script to use lcov on build+test machine Mauro Carvalho Chehab @ 2022-03-16 15:00 ` Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 08/10] scripts/run-tests.sh: add code coverage support Mauro Carvalho Chehab ` (4 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 15:00 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> The proper sequence to process the results at the build machine is not trivial. Add a script automating such steps. Reviewed-by: Tomi Sarvela <tomi.p.sarvela@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/code_cov_gen_report.sh | 170 +++++++++++++++++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100755 scripts/code_cov_gen_report.sh diff --git a/scripts/code_cov_gen_report.sh b/scripts/code_cov_gen_report.sh new file mode 100755 index 000000000000..05efebe19ee4 --- /dev/null +++ b/scripts/code_cov_gen_report.sh @@ -0,0 +1,170 @@ +#!/bin/bash + +MERGED_INFO="merged" +GATHER_ON_BUILD="code_cov_gather_on_build.sh" +PARSE_INFO="code_cov_parse_info.pl" + +trap 'catch $LINENO' ERR +catch() { + echo "$0: error on line $1. HTML report not generated." + exit $1 +} + +usage() { + printf >&2 "\ +Usage: + $(basename $0) + --read <file or dir> --kernel-source <dir> --kernel-object <dir> + --output-dir <dir> [--info or --tar] [--force-override] + +--kernel-object is only needed when Kernel was built with make O=dir +" + exit $1 +} + +MODE= +RESULTS= +KSRC= +KOBJ= +DEST_DIR= +FORCE= + +while [ "$1" != "" ]; do + case $1 in + --info|-i) + MODE=info + ;; + --tar|--tarball|-t) + MODE=tar.gz + ;; + --kernel-source|-k) + if [ "$2" == "" ]; then + usage 1 + else + KSRC=$(realpath $2) + shift + fi + ;; + --kernel-object|-O) + if [ "$2" == "" ]; then + usage 1 + else + KOBJ=$(realpath $2) + shift + fi + ;; + --output-dir|-o) + if [ "$2" == "" ]; then + usage 1 + else + DEST_DIR=$(realpath $2) + shift + fi + ;; + --read|-r) + if [ "$2" == "" ]; then + usage 1 + else + RESULTS=$(realpath $2) + shift + fi + ;; + --force-override|-f) + FORCE=1 + ;; + --help) + usage 0 + ;; + + *) + echo "Unknown argument '$1'" + usage 1 + ;; + esac + shift +done + +if [ "x$RESULTS" == "x" -o "x$KSRC" == "x" -o "x$DEST_DIR" == "x" -o "x$MODE" == "x" ]; then + echo "Missing a mandatory argument" + usage 1 +fi + +if [ -z "$KOBJ" ]; then + KOBJ=$KSRC +fi + +SCRIPT_DIR=$(dirname $(realpath $0)) +RESULTS=$(realpath $RESULTS) +KSRC=$(realpath $KSRC) +KOBJ=$(realpath $KOBJ) +DEST_DIR=$(realpath $DEST_DIR) + +if [ -e "$DEST_DIR" ]; then + if [ "x$FORCE" != "x" -a -d "$DEST_DIR" ]; then + rm -rf $DEST_DIR/ + else + echo "Directory exists. Won't override." + exit 1 + fi +fi + +mkdir -p $DEST_DIR +cd $DEST_DIR + +if [ "$MODE" != "info" ]; then + echo "Generating source tarball from $KSRC (O=$KOBJ)..." + ${SCRIPT_DIR}/${GATHER_ON_BUILD} $KSRC $KOBJ source.tar.gz + + echo "Adding source files..." + tar xf source.tar.gz + + if [ -d "$RESULTS" ]; then + echo "Creating per-file info files..." + echo -n "" >${MERGED_INFO}.info + for i in $RESULTS/*.tar.gz; do + TITLE=$(basename $i) + TITLE=${TITLE/.tar.gz/} + + echo "Adding results from $i..." + tar xf $i + + echo "Generating $TITLE.info..." + lcov -q -t ${TITLE} --rc lcov_branch_coverage=1 -o $TITLE.info -c -d . + + cat $TITLE.info >>${MERGED_INFO}.info + + # Remove the contents of the results tarball + rm -rf sys/ + done + + TITLE=${MERGED_INFO} + else + TITLE=$(basename $RESULTS) + TITLE=${TITLE/.tar.gz/} + + echo "Adding results from $RESULTS..." + tar xf $RESULTS + + echo "Generating $TITLE.info..." + lcov -q -t ${TITLE} --rc lcov_branch_coverage=1 -o $TITLE.info -c -d . + fi +else + if [ -d "$RESULTS" ]; then + echo "Merging info files..." + echo -n "" >${MERGED_INFO}.info + for i in $RESULTS/*.info; do + cat $i >>${MERGED_INFO}.info + done + + TITLE=${MERGED_INFO} + else + echo "Copying $RESULTS to $DEST_DIR..." + cp $RESULTS . + + TITLE=$(basename $RESULTS) + TITLE=${TITLE/.info/} + fi +fi + +echo "Generating HTML files..." +genhtml -q ${TITLE}.info -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 08/10] scripts/run-tests.sh: add code coverage support 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (6 preceding siblings ...) 2022-03-16 15:00 ` [igt-dev] [PATCH v3 07/10] scripts/code_cov_gen_report.sh: add a script to generate code coverage reports Mauro Carvalho Chehab @ 2022-03-16 15:00 ` Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 09/10] scripts:code_cov_gather_on_test: use a faster script Mauro Carvalho Chehab ` (3 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 15:00 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> When the Kernel is built with GCOV_KERNEL and one or more drivers are built with GCOV_PROFILE enabled[1], the Kernel will collect code coverage usage at the same time as a test runs. Add support at run-tests.sh to collect usage data and store them on .info files. That actually require two new options: -c <test_name>: will store the tests under test_name.info; -k <kernel_tree>: Points to the source code of the built Kernel. The Kernel tree can be a partial tree, provided that it contains at least all *.h files used by the drivers, plus the *.c files that were built with gcov support enabled. [1] See https://01.org/linuxgraphics/gfx-docs/drm/dev-tools/gcov.html Reviewed-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/run-tests.sh | 52 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 46 insertions(+), 6 deletions(-) diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh index 8399b6d15aa9..6986232e8eb4 100755 --- a/scripts/run-tests.sh +++ b/scripts/run-tests.sh @@ -29,6 +29,11 @@ RESULTS="$ROOT/results" PIGLIT=`which piglit 2> /dev/null` IGT_RUNNER= IGT_RESUME= +IGT_KERNEL_TREE= +COV_ARGS= +COV_PER_TEST= +LCOV_CMD="lcov" +KERNEL_TREE= USE_PIGLIT=0 RUNNER= RESUME= @@ -84,6 +89,15 @@ function find_runner_binary # basename return 1 } +function find_lcov_binary # basename +{ + if command -v $LCOV_CMD &> /dev/null; then + return 0 + fi + + return 1 +} + function download_piglit { git clone https://anongit.freedesktop.org/git/piglit.git "$ROOT/piglit" } @@ -96,10 +110,10 @@ function execute_runner # as-root <runner> <args> shift local sudo - export IGT_TEST_ROOT IGT_CONFIG_PATH + export IGT_TEST_ROOT IGT_CONFIG_PATH IGT_KERNEL_TREE if [ "$need_root" -ne 0 -a "$EUID" -ne 0 ]; then - sudo="sudo --preserve-env=IGT_TEST_ROOT,IGT_CONFIG_PATH" + sudo="sudo --preserve-env=IGT_TEST_ROOT,IGT_CONFIG_PATH,IGT_KERNEL_TREE" fi $sudo $runner "$@" @@ -108,8 +122,14 @@ function execute_runner # as-root <runner> <args> function print_help { echo "Usage: run-tests.sh [options]" echo "Available options:" + echo " -c <capture_script>" + echo " capture gcov code coverage using the <capture_script>." + echo " -P store code coverage results per each test. Should be" + echo " used together with -k option" echo " -d download Piglit to $ROOT/piglit" echo " -h display this help message" + echo " -k <kernel_dir> Linux Kernel source code directory used to generate code" + echo " coverage builds." echo " -l list all available tests" echo " -r <directory> store the results in directory" echo " (default: $RESULTS)" @@ -134,11 +154,14 @@ function print_help { echo "Useful patterns for test filtering are described in the API documentation." } -while getopts ":dhlr:st:T:vx:Rnpb:" opt; do +while getopts ":c:dhk:lPr:st:T:vx:Rnpb:" opt; do case $opt in + c) COV_ARGS="$COV_ARGS --collect-code-cov --collect-script $OPTARG " ;; d) download_piglit; exit ;; h) print_help; exit ;; + k) IGT_KERNEL_TREE="$OPTARG" ;; l) LIST_TESTS="true" ;; + P) COV_ARGS="$COV_ARGS --coverage-per-test"; COV_PER_TEST=1 ;; r) RESULTS="$OPTARG" ;; s) SUMMARY="html" ;; t) FILTER="$FILTER -t $OPTARG" ;; @@ -172,6 +195,18 @@ if [ "x$PIGLIT" == "x" ]; then PIGLIT="$ROOT/piglit/piglit" fi +if [ "x$COV_ARGS" != "x" ]; then + if [ "$USE_PIGLIT" -eq "1" ]; then + echo "Cannot collect code coverage when running tests with Piglit. Use igt_runner instead." + exit 1 + fi + + if ! $(find_lcov_binary); then + echo "Can't check code coverage, as 'lcov' is not installed" + exit 1 + fi +fi + RUN_ARGS= RESUME_ARGS= LIST_ARGS= @@ -201,15 +236,20 @@ else fi if [ "x$LIST_TESTS" != "x" ]; then - execute_runner 0 $RUNNER $LIST_ARGS $FILTER + execute_runner 0 $RUNNER $LIST_ARGS $FILTER $COV_ARGS exit fi if [ "x$RESUME_RUN" != "x" ]; then - execute_runner 1 $RESUME $RESUME_ARGS "$RESULTS" + if [ "x$COV_ARGS" != "x" -a "x$COV_PER_TEST" == "x" ]; then + echo "Can't continue collecting coverage tests. Next time, run" + echo "$0 with '-P' in order to generate separate code coverage results". + exit 1 + fi + execute_runner 1 $RESUME $RESUME_ARGS $COV_ARGS "$RESULTS" else mkdir -p "$RESULTS" - execute_runner 1 $RUNNER $RUN_ARGS -o -s "$RESULTS" $VERBOSE $FILTER + execute_runner 1 $RUNNER $RUN_ARGS -o -s "$RESULTS" $COV_ARGS $VERBOSE $FILTER fi if [ "$SUMMARY" == "html" ]; then -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 09/10] scripts:code_cov_gather_on_test: use a faster script 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (7 preceding siblings ...) 2022-03-16 15:00 ` [igt-dev] [PATCH v3 08/10] scripts/run-tests.sh: add code coverage support Mauro Carvalho Chehab @ 2022-03-16 15:00 ` Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 10/10] docs: add documentation for code coverage Mauro Carvalho Chehab ` (2 subsequent siblings) 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 15:00 UTC (permalink / raw) To: igt-dev From: Tomi Sarvela <tomi.p.sarvela@intel.com> The original shell script takes too long to complete (~7-10 seconds), while the python version requires only ~500 ms. As this has a relevant impact when doing the tests, use the faster version. Signed-off-by: Tomi Sarvela <tomi.p.sarvela@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- scripts/code_cov_gather_on_test.py | 91 ++++++++++++++++++++++++++++++ scripts/code_cov_gather_on_test.sh | 20 ------- 2 files changed, 91 insertions(+), 20 deletions(-) create mode 100755 scripts/code_cov_gather_on_test.py delete mode 100755 scripts/code_cov_gather_on_test.sh diff --git a/scripts/code_cov_gather_on_test.py b/scripts/code_cov_gather_on_test.py new file mode 100755 index 000000000000..b4356aa1397b --- /dev/null +++ b/scripts/code_cov_gather_on_test.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 OR MIT +# +# Copyright (C) 2022 Intel Corporation +# +# gather_on_test.py +# +# by Tomi Sarvela <tomi.p.sarvela@intel.com> +# +# Faster implementation for linux kernel GCOV data collection +# Command line compatible with original gather_on_test.sh +# +# Refs: +# https://www.kernel.org/doc/html/latest/dev-tools/gcov.html +# +import argparse +import errno +import io +import os +import sys +import tarfile + +def parse_args() -> argparse.Namespace: + '''Command line arguments''' + ap = argparse.ArgumentParser(description="Gather Linux kernel GCOV data") + ap.add_argument('output', + help="Output file name (.tar.gz will be added)") + ap.add_argument('--gcov', default='/sys/kernel/debug/gcov', + help="GCOV directory, default: /sys/kernel/debug/gcov") + return ap.parse_args() + +def tar_add_link(tar:tarfile.TarFile, filename:str): + '''Add filename as symlink to tarfile''' + info = tarfile.TarInfo(filename) + if info.name[0] == '/': info.name = info.name[1:] + info.type = tarfile.SYMTYPE + info.linkname = os.readlink(filename) + tar.addfile(info) + +def tar_add_file(tar:tarfile.TarFile, filename:str): + '''Add filename to tarfile, file size expected to be invalid''' + try: + with open(filename, "rb") as fp: + data = fp.read() # big gulp + except OSError as e: + print(f"ERROR: {filename}: {e}", file=sys.stderr) + return + info = tarfile.TarInfo(filename) + if info.name[0] == '/': info.name = info.name[1:] + info.size = len(data) + tar.addfile(info, io.BytesIO(data)) + +def tar_add_tree(tar:tarfile.TarFile, tree:str): + '''Add gcov files in directory tree to tar''' + # FIXME: should dirs be added to tar for compatibility? + for root, _, files in os.walk(tree, followlinks=False): + for file in files: + filepath = os.path.join(root, file) + if file.endswith('.gcda'): tar_add_file(tar, filepath) + if file.endswith('.gcno'): tar_add_link(tar, filepath) + +def main() -> int: + '''MAIN''' + if not os.path.isdir(args.gcov): + print(f"ERROR: [Errno {errno.ENOTDIR}] {os.strerror(errno.ENOTDIR)}: '{args.gcov}'", + file=sys.stderr) + return errno.ENOTDIR + if args.output == '-': + # reopen stdout as bytes for tarfile + fp = os.fdopen(sys.stdout.fileno(), "wb", closefd=False) + else: + if not args.output.endswith('.tgz') and \ + not args.output.endswith('.tar.gz'): + args.output+='.tar.gz' + try: + fp = open(args.output, 'wb') + except OSError as e: + print(f"ERROR: {e}", file=sys.stderr) + return e.errno + with tarfile.open(fileobj=fp, mode='w:gz') as tar: + tar_add_tree(tar, args.gcov) + fp.close() + return 0 + +if __name__ == '__main__': + try: + args = parse_args() + sys.exit(main()) + except KeyboardInterrupt: + print("Interrupted", file=sys.stderr) + sys.exit(errno.EINTR) diff --git a/scripts/code_cov_gather_on_test.sh b/scripts/code_cov_gather_on_test.sh deleted file mode 100755 index 8834aa0d78af..000000000000 --- a/scripts/code_cov_gather_on_test.sh +++ /dev/null @@ -1,20 +0,0 @@ -#!/bin/bash -e - -DEST=$1 -GCDA=/sys/kernel/debug/gcov - -if [ -z "$DEST" ] ; then - echo "Usage: $0 <output.tar.gz>" >&2 - exit 1 -fi - -TEMPDIR=$(mktemp -d) -echo Collecting data.. -find $GCDA -type d -exec mkdir -p $TEMPDIR/\{\} \; -find $GCDA -name '*.gcda' -exec sh -c 'cat < $0 > '$TEMPDIR'/$0' {} \; -find $GCDA -name '*.gcno' -exec sh -c 'cp -d $0 '$TEMPDIR'/$0' {} \; -tar czf $DEST -C $TEMPDIR sys -rm -rf $TEMPDIR - -echo "$DEST successfully created, copy to build system and unpack with:" -echo " tar xfz $DEST" -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v3 10/10] docs: add documentation for code coverage 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (8 preceding siblings ...) 2022-03-16 15:00 ` [igt-dev] [PATCH v3 09/10] scripts:code_cov_gather_on_test: use a faster script Mauro Carvalho Chehab @ 2022-03-16 15:00 ` Mauro Carvalho Chehab 2022-03-16 15:44 ` [igt-dev] ✓ Fi.CI.BAT: success for Add support to collect code coverage data Patchwork 2022-03-16 16:53 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork 11 siblings, 0 replies; 14+ messages in thread From: Mauro Carvalho Chehab @ 2022-03-16 15:00 UTC (permalink / raw) To: igt-dev From: Mauro Carvalho Chehab <mchehab@kernel.org> Document the IGT runner features related to code coverage data capture. Acked-by: Petri Latvala <petri.latvala@intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> --- docs/code_coverage.md | 293 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 293 insertions(+) create mode 100644 docs/code_coverage.md diff --git a/docs/code_coverage.md b/docs/code_coverage.md new file mode 100644 index 000000000000..34d56d283a75 --- /dev/null +++ b/docs/code_coverage.md @@ -0,0 +1,293 @@ +# Collecting code coverage data from IGT tests + +## Introduction + +Ensuring that a test plan covers all the driver code is not trivial. Also, +as time goes by, changes at both the tests and drivers may badly affect +the code coverage. So, some tools are needed in order to be able to verify +and improve the driver test coverage. While static analyzers can help +checking the driver's code, it is not as effective as runtime tests. + +Thankfully gcc has a feature which allows capturing such data in realtime, +called gcov. LLVM/clang also has a similar feature (llvm-cov). Such feature +is available at the Linux Kernel since 2009. + +## Building a Kernel with GCOV support + +Enabling GCOV at the Linux Kernel requires two steps: + +1. Enable GCOV_KERNEL: + + ``` + ./scripts/config -e DEBUG_FS -e GCOV_KERNEL + ``` + + +2. Enable per-driver or per-makefile GCOV support. In order to enable support + for all DRM drivers: + + ``` + for i in $(find drivers/gpu/drm/ -name Makefile); do + sed '1 a GCOV_PROFILE := y' -i $i + done + ``` + +When gcov is enabled for a given driver or directory, GCC will generate +some special object files, like: + +``` +... +drivers/gpu/drm/drm_probe_helper.gcno +drivers/gpu/drm/drm_dp_dual_mode_helper.gcno +drivers/gpu/drm/drm_plane.gcno +drivers/gpu/drm/drm_lease.gcno +drivers/gpu/drm/drm_mipi_dsi.gcno +drivers/gpu/drm/drm_dsc.gcno +drivers/gpu/drm/drm_property.gcno +drivers/gpu/drm/drm_dp_aux_dev.gcno +drivers/gpu/drm/drm_blend.gcno +... +``` + +Those will be stored at the Kernel object directory, which is usually +the same as the Kernel source directory, except if the Kernel was built +with: + +``` +make O=kernel_output_dir +``` + +Such compile-time files are compiler-dependent and they're needed in order +to properly decode the code coverage counters that will be produced in +runtime. + +## Collecting GCOV data in runtime + +Once a GCOV-enabled Kernel boots, the Kernel will keep track of the code +monitored via GCOV under sysfs, at `/sys/kernel/debug/gcov/`. + +There is a special file there: `/sys/kernel/debug/gcov/reset`. When something +is written to it, all counters will be cleaned. + +There are also driver-related counters and softlinks stored there: + +``` +ls -la /basedir/linux/drivers/gpu/drm/ +... +-rw------- 1 root root 0 Feb 16 07:03 drm_probe_helper.gcda +lrwxrwxrwx 1 root root 0 Feb 16 07:03 drm_probe_helper.gcno -> /basedir/linux/drivers/gpu/drm/drm_probe_helper.gcno +-rw------- 1 root root 0 Feb 16 07:03 drm_property.gcda +lrwxrwxrwx 1 root root 0 Feb 16 07:03 drm_property.gcno -> /basedir/linux/drivers/gpu/drm/drm_property.gcno +-rw------- 1 root root 0 Feb 16 07:03 drm_rect.gcda +lrwxrwxrwx 1 root root 0 Feb 16 07:03 drm_rect.gcno -> /basedir/linux/drivers/gpu/drm/drm_rect.gcno +... +``` + +The actual counters are stored at the *.gcda files on a compiler-dependent +format. + +### calling `igt_runner` directly + +When code coverage support is enabled, the `igt_runner` tool will internally +clean up the counters before starting test(s). Once test(s) finish, it will +also run an external script that will be responsible for collecting the data +and store on some file. + +Enabling code coverage data collect can be done either per test or as +a hole for an entire test list, by using those command line options: + +- `--collect-code-cov` + + Enables gcov-based collect of code coverage for tests. + +- `--coverage-per-test` + + Stores code coverage results per each test. This option implies + `--collect-code-cov`. + +For those options to work, it is mandatory to specifiy what script will +be used to collect the data with `--collect-script` _file_name_. + +### calling `./scripts/run-tests.sh` script + +The `run-tests.sh` script can used instead as a frontend for igt_runner. +It has the following options: + +- `-c <capture_script>` + + Capture gcov code coverage using the _capture_script_ + +- `-P` + + Store code coverage results per each test. + +- `-k` _kernel_dir_ + + Linux Kernel source code directory used to generate code coverage builds. + This is passed through the capture script via the `IGT_KERNEL_TREE` + shell environment variable. + +So, for instance, if one wans to capture code coverage data from the +Kernel that was built at the same machine, at the directory `~/linux`, +and wants to capture one file per test, it would use: + +``` +./scripts/run-tests.sh -T my.testlist -k ~/linux -c scripts/code_cov_capture.sh -P +``` + +### Code Coverage Collect script + +While any script could in thesis be used, currently, there are two ones +under `scripts/`: + +- `scripts/code_cov_capture.sh`: + + Assumes that the Kernel was built at the same machine, and uses + the lcov tool to generate GCC-independent code coverage data, + in the form of `*.info` files. Internally, it uses an shell environment + variable (`IGT_KERNEL_TREE`), which points to the place where the Kernel + source and objects are contained. + + Such script requires `lcov` tool to be installed at the test machine. + +- `scripts/code_cov_gather_on_test.py`: + + Generates a gzipped tarbal with the code coverage counters in + binary format. Such kind of output should then be parsed at + the same machine where the Kernel as built, as its content is not + ony dependent on the Kernel source, but also on the Kernel output + objects. + +For each script, the igt_runner passes just one parameter: the results +directory + the test name. + +For instance, if it is needed to run a test called +`debugfs_test (read_all_entries)` using `scripts/code_cov_capture.sh` +parameter, e. g.: + +``` +$ echo "igt@debugfs_test@read_all_entries" > my.testlist +$ ./scripts/run-tests.sh -T my.testlist -k ~/linux -c scripts/code_cov_capture.sh -P +Found test list: "/basedir/igt/build/tests/test-list.txt" +[31410.499969] [1/1] debugfs_test (read_all_entries) +[31411.060446] Storing code coverage results... +[31418.01] Code coverage wrote to /basedir/igt/results/code_cov/debugfs_test_read_all_entries.info +Done. +``` + +The script will be called as: + +``` +scripts/code_cov_capture.sh /basedir/igt/results/code_cov/debugfs_test_read_all_entries +``` + +Please notice that any character that it is not a number nor a letter at the +test name will be converted into '_', as other characters are not supported +as titles at the lcov files. + +### Passing extra arguments to the script + +If any extra global parameters are needed by the script, those can be sent +via shell's environment var. + +### The `*.info` file format + +The `*.info` files contain several fields on it, grouped into records. +An info file looks like: + +``` +TN:fbdev_eof +... +SF:/basedir/linux/drivers/gpu/drm/i915/intel_runtime_pm.c +... +FN:158,__intel_runtime_pm_get +FNDA:2,__intel_runtime_pm_get +... +end_of_record +SF:<some other file> +... +end_of_record +... +``` + +The main fields at the above record are: + +- `TN:` Test name +- `SF:` Source file +- `FN:` line_number function_name +- `FNDA:` call_count function_name + +So, the above example means that, inside +`drivers/gpu/drm/i915/intel_runtime_pm.c` there's a function +`__intel_runtime_pm_get()` which it was called 2 times. + +## Generating code coverage documentation + +The `lcov` package contains the needed tools to parse and generate code +coverage documentation. It is used by `scripts/code_cov_capture.sh` script +to convery from compiler-dependent `*.gcno` counters into a +compiler-independent format (`*.info`). + +Grouping multiple `*.info` files is as easy as running: + +``` +cat core*.info > all_core.info +``` + +The `lcov` package also contains a tool which converts a given `*.info` file +into html patches, called `genhtml`. + +As the output can actually show the code source file, `genhtml` need access +not only to the info file, but also to the Kernel directory with the +source files. Some optional arguments can be used at the command line, or +can be stored at `/etc/lcovrc` or `~/.lcovrc` files. + +As generating the documentation depends wheather the results were generated +as with a single or multiple `*.info` files by `scripts/code_cov_capture.sh` +or stored in raw formats inside `*.tar.gz` file(s) by +`scripts/code_cov_gather_on_test.py`, there's a script that does all the +required steps to build the code coverage html reports: +`scripts/code_cov_gen_report.sh`. + +It requires the following arguments: + +- `--read` _file or dir_ (or `-r` _file or dir_) + + File or directory where the code coverage capture file(s) is(are) located. + +- `--kernel-source` _dir_ (or `-k` _dir_) + + Kernel source directory. + +- `--kernel-object` _dir_ (or `-O` _dir_) + + Kernel object directory. Only needed when Kernel was built with `make O=dir`. + +- `--output-dir` _dir_ (or `-o` _dir) + + Directory where the html output will be stored. By default, the script + won't let re-use an already existing directory. + +- `--info` + + The files specified by `--read` parameter are at lcov's `*.info` format. + +- `--tar` + + The files specified by `--read` are gzipped tarballs containing all + `*.gcno` files and all `*.gcda` softlinks from the `/sys/kernel/debug/gcov/` + directory at the test machine, created by + `scripts/code_cov_gather_on_test.py` script. + +- `--force-override` + + Allow using a non-empty directory for `--output-dir`. + +`--info` and `--tar` are mutually exclusive and at least one of them should +be specified. + +## References + +More information is available at Kernel gcov documentation: +[Using gcov with the Linux kernel](https://www.kernel.org/doc/html/latest/dev-tools/gcov.html). + -- 2.35.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [igt-dev] ✓ Fi.CI.BAT: success for Add support to collect code coverage data 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (9 preceding siblings ...) 2022-03-16 15:00 ` [igt-dev] [PATCH v3 10/10] docs: add documentation for code coverage Mauro Carvalho Chehab @ 2022-03-16 15:44 ` Patchwork 2022-03-16 16:53 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork 11 siblings, 0 replies; 14+ messages in thread From: Patchwork @ 2022-03-16 15:44 UTC (permalink / raw) To: Mauro Carvalho Chehab; +Cc: igt-dev [-- Attachment #1: Type: text/plain, Size: 18402 bytes --] == Series Details == Series: Add support to collect code coverage data URL : https://patchwork.freedesktop.org/series/101434/ State : success == Summary == CI Bug Log - changes from IGT_6383 -> IGTPW_6794 ==================================================== Summary ------- **SUCCESS** No regressions found. External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/index.html Participating hosts (41 -> 46) ------------------------------ Additional (8): fi-kbl-soraka bat-dg1-6 bat-dg1-5 bat-adlm-1 bat-dg2-9 fi-kbl-8809g fi-pnv-d510 bat-rpls-1 Missing (3): fi-ctg-p8600 fi-bsw-cyan fi-hsw-4200u Possible new issues ------------------- Here are the unknown changes that may have been introduced in IGTPW_6794: ### IGT changes ### #### Suppressed #### The following results come from untrusted machines, tests, or statuses. They do not affect the overall result. * igt@kms_flip@basic-plain-flip: - {bat-adlm-1}: NOTRUN -> [INCOMPLETE][1] [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-adlm-1/igt@kms_flip@basic-plain-flip.html Known issues ------------ Here are the changes found in IGTPW_6794 that come from known issues: ### IGT changes ### #### Issues hit #### * igt@amdgpu/amd_basic@cs-gfx: - fi-hsw-4770: NOTRUN -> [SKIP][2] ([fdo#109271] / [fdo#109315]) +17 similar issues [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-hsw-4770/igt@amdgpu/amd_basic@cs-gfx.html * igt@amdgpu/amd_basic@cs-sdma: - fi-cfl-8109u: NOTRUN -> [SKIP][3] ([fdo#109271]) +27 similar issues [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@amdgpu/amd_basic@cs-sdma.html * igt@fbdev@write: - bat-dg1-5: NOTRUN -> [SKIP][4] ([i915#2582]) +4 similar issues [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@fbdev@write.html * igt@gem_exec_fence@basic-busy@bcs0: - fi-kbl-soraka: NOTRUN -> [SKIP][5] ([fdo#109271]) +9 similar issues [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@gem_exec_fence@basic-busy@bcs0.html * igt@gem_exec_gttfill@basic: - bat-dg1-6: NOTRUN -> [SKIP][6] ([i915#4086]) [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@gem_exec_gttfill@basic.html - bat-dg1-5: NOTRUN -> [SKIP][7] ([i915#4086]) [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@gem_exec_gttfill@basic.html * igt@gem_exec_suspend@basic-s0@smem: - fi-kbl-8809g: NOTRUN -> [DMESG-WARN][8] ([i915#4962]) +1 similar issue [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@gem_exec_suspend@basic-s0@smem.html * igt@gem_huc_copy@huc-copy: - fi-kbl-soraka: NOTRUN -> [SKIP][9] ([fdo#109271] / [i915#2190]) [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@gem_huc_copy@huc-copy.html - fi-kbl-8809g: NOTRUN -> [SKIP][10] ([fdo#109271] / [i915#2190]) [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@gem_huc_copy@huc-copy.html - fi-cfl-8109u: NOTRUN -> [SKIP][11] ([fdo#109271] / [i915#2190]) [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@gem_huc_copy@huc-copy.html * igt@gem_lmem_swapping@basic: - fi-kbl-soraka: NOTRUN -> [SKIP][12] ([fdo#109271] / [i915#4613]) +3 similar issues [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@gem_lmem_swapping@basic.html * igt@gem_lmem_swapping@random-engines: - fi-kbl-8809g: NOTRUN -> [SKIP][13] ([fdo#109271] / [i915#4613]) +3 similar issues [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@gem_lmem_swapping@random-engines.html - fi-cfl-8109u: NOTRUN -> [SKIP][14] ([fdo#109271] / [i915#4613]) +3 similar issues [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@gem_lmem_swapping@random-engines.html * igt@gem_mmap@basic: - bat-dg1-6: NOTRUN -> [SKIP][15] ([i915#4083]) [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@gem_mmap@basic.html - bat-dg1-5: NOTRUN -> [SKIP][16] ([i915#4083]) [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@gem_mmap@basic.html * igt@gem_tiled_blits@basic: - bat-dg1-6: NOTRUN -> [SKIP][17] ([i915#4077]) +2 similar issues [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@gem_tiled_blits@basic.html * igt@gem_tiled_fence_blits@basic: - bat-dg1-5: NOTRUN -> [SKIP][18] ([i915#4077]) +2 similar issues [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@gem_tiled_fence_blits@basic.html * igt@gem_tiled_pread_basic: - bat-dg1-5: NOTRUN -> [SKIP][19] ([i915#4079]) +1 similar issue [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@gem_tiled_pread_basic.html - bat-dg1-6: NOTRUN -> [SKIP][20] ([i915#4079]) +1 similar issue [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@gem_tiled_pread_basic.html * igt@i915_pm_backlight@basic-brightness: - bat-dg1-5: NOTRUN -> [SKIP][21] ([i915#1155]) [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@i915_pm_backlight@basic-brightness.html - bat-dg1-6: NOTRUN -> [SKIP][22] ([i915#1155]) [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@i915_pm_backlight@basic-brightness.html * igt@i915_pm_rps@basic-api: - bat-dg1-6: NOTRUN -> [FAIL][23] ([i915#4032]) [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@i915_pm_rps@basic-api.html * igt@i915_selftest@live@gt_pm: - fi-kbl-soraka: NOTRUN -> [DMESG-FAIL][24] ([i915#1886]) [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@i915_selftest@live@gt_pm.html * igt@i915_selftest@live@hangcheck: - bat-dg1-5: NOTRUN -> [DMESG-FAIL][25] ([i915#4494] / [i915#4957]) [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@i915_selftest@live@hangcheck.html - bat-dg1-6: NOTRUN -> [DMESG-FAIL][26] ([i915#4494] / [i915#4957]) [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@i915_selftest@live@hangcheck.html * igt@kms_addfb_basic@addfb25-x-tiled-legacy: - bat-dg1-6: NOTRUN -> [SKIP][27] ([i915#4212]) +7 similar issues [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_addfb_basic@addfb25-x-tiled-legacy.html * igt@kms_addfb_basic@basic-y-tiled-legacy: - bat-dg1-5: NOTRUN -> [SKIP][28] ([i915#4215]) [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_addfb_basic@basic-y-tiled-legacy.html - bat-dg1-6: NOTRUN -> [SKIP][29] ([i915#4215]) [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_addfb_basic@basic-y-tiled-legacy.html * igt@kms_addfb_basic@tile-pitch-mismatch: - bat-dg1-5: NOTRUN -> [SKIP][30] ([i915#4212]) +7 similar issues [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_addfb_basic@tile-pitch-mismatch.html * igt@kms_busy@basic: - bat-dg1-5: NOTRUN -> [SKIP][31] ([i915#4303]) [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_busy@basic.html * igt@kms_chamelium@common-hpd-after-suspend: - fi-kbl-soraka: NOTRUN -> [SKIP][32] ([fdo#109271] / [fdo#111827]) +8 similar issues [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@kms_chamelium@common-hpd-after-suspend.html * igt@kms_chamelium@hdmi-edid-read: - fi-kbl-8809g: NOTRUN -> [SKIP][33] ([fdo#109271] / [fdo#111827]) +8 similar issues [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@kms_chamelium@hdmi-edid-read.html - bat-dg1-6: NOTRUN -> [SKIP][34] ([fdo#111827]) +8 similar issues [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_chamelium@hdmi-edid-read.html * igt@kms_chamelium@hdmi-hpd-fast: - bat-dg1-5: NOTRUN -> [SKIP][35] ([fdo#111827]) +8 similar issues [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_chamelium@hdmi-hpd-fast.html - fi-cfl-8109u: NOTRUN -> [SKIP][36] ([fdo#109271] / [fdo#111827]) +8 similar issues [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@kms_chamelium@hdmi-hpd-fast.html * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic: - bat-dg1-6: NOTRUN -> [SKIP][37] ([i915#4103] / [i915#4213]) +1 similar issue [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic.html * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy: - bat-dg1-5: NOTRUN -> [SKIP][38] ([i915#4103] / [i915#4213]) +1 similar issue [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html * igt@kms_flip@basic-flip-vs-dpms: - bat-dg1-5: NOTRUN -> [SKIP][39] ([i915#4078]) +23 similar issues [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_flip@basic-flip-vs-dpms.html * igt@kms_force_connector_basic@force-load-detect: - bat-dg1-6: NOTRUN -> [SKIP][40] ([fdo#109285]) [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_force_connector_basic@force-load-detect.html - bat-dg1-5: NOTRUN -> [SKIP][41] ([fdo#109285]) [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_force_connector_basic@force-load-detect.html * igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c: - fi-pnv-d510: NOTRUN -> [SKIP][42] ([fdo#109271] / [i915#5341]) [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-pnv-d510/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c.html - bat-dg1-5: NOTRUN -> [SKIP][43] ([i915#4078] / [i915#5341]) [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c.html - fi-kbl-8809g: NOTRUN -> [SKIP][44] ([fdo#109271] / [i915#5341]) [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c.html * igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d: - fi-kbl-soraka: NOTRUN -> [SKIP][45] ([fdo#109271] / [i915#533]) [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-soraka/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html - fi-cfl-8109u: NOTRUN -> [SKIP][46] ([fdo#109271] / [i915#533]) [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html - fi-kbl-8809g: NOTRUN -> [SKIP][47] ([fdo#109271] / [i915#533]) [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html * igt@kms_psr@cursor_plane_move: - fi-kbl-8809g: NOTRUN -> [SKIP][48] ([fdo#109271]) +54 similar issues [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-kbl-8809g/igt@kms_psr@cursor_plane_move.html * igt@kms_psr@primary_page_flip: - bat-dg1-5: NOTRUN -> [SKIP][49] ([i915#1072] / [i915#4078]) +3 similar issues [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_psr@primary_page_flip.html * igt@kms_psr@sprite_plane_onoff: - bat-dg1-6: NOTRUN -> [SKIP][50] ([i915#1072] / [i915#4078]) +3 similar issues [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_psr@sprite_plane_onoff.html * igt@kms_setmode@basic-clone-single-crtc: - bat-dg1-6: NOTRUN -> [SKIP][51] ([i915#3555]) [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@kms_setmode@basic-clone-single-crtc.html - bat-dg1-5: NOTRUN -> [SKIP][52] ([i915#3555]) [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@kms_setmode@basic-clone-single-crtc.html * igt@prime_vgem@basic-fence-mmap: - bat-dg1-5: NOTRUN -> [SKIP][53] ([i915#3708] / [i915#4077]) +1 similar issue [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@prime_vgem@basic-fence-mmap.html * igt@prime_vgem@basic-gtt: - bat-dg1-6: NOTRUN -> [SKIP][54] ([i915#3708] / [i915#4077]) +1 similar issue [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@prime_vgem@basic-gtt.html * igt@prime_vgem@basic-userptr: - fi-pnv-d510: NOTRUN -> [SKIP][55] ([fdo#109271]) +57 similar issues [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-pnv-d510/igt@prime_vgem@basic-userptr.html - bat-dg1-6: NOTRUN -> [SKIP][56] ([i915#3708] / [i915#4873]) [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@prime_vgem@basic-userptr.html - bat-dg1-5: NOTRUN -> [SKIP][57] ([i915#3708] / [i915#4873]) [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@prime_vgem@basic-userptr.html * igt@prime_vgem@basic-write: - bat-dg1-5: NOTRUN -> [SKIP][58] ([i915#3708]) +3 similar issues [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-5/igt@prime_vgem@basic-write.html - bat-dg1-6: NOTRUN -> [SKIP][59] ([i915#3708]) +3 similar issues [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/bat-dg1-6/igt@prime_vgem@basic-write.html #### Possible fixes #### * igt@gem_exec_suspend@basic-s0@smem: - fi-cfl-8109u: [INCOMPLETE][60] -> [PASS][61] [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/fi-cfl-8109u/igt@gem_exec_suspend@basic-s0@smem.html [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-cfl-8109u/igt@gem_exec_suspend@basic-s0@smem.html * igt@i915_selftest@live@hangcheck: - fi-hsw-4770: [INCOMPLETE][62] ([i915#3303]) -> [PASS][63] [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/fi-hsw-4770/igt@i915_selftest@live@hangcheck.html [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/fi-hsw-4770/igt@i915_selftest@live@hangcheck.html {name}: This element is suppressed. This means it is ignored when computing the status of the difference (SUCCESS, WARNING, or FAILURE). [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271 [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285 [fdo#109308]: https://bugs.freedesktop.org/show_bug.cgi?id=109308 [fdo#109315]: https://bugs.freedesktop.org/show_bug.cgi?id=109315 [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825 [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827 [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072 [i915#1155]: https://gitlab.freedesktop.org/drm/intel/issues/1155 [i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845 [i915#1849]: https://gitlab.freedesktop.org/drm/intel/issues/1849 [i915#1886]: https://gitlab.freedesktop.org/drm/intel/issues/1886 [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190 [i915#2582]: https://gitlab.freedesktop.org/drm/intel/issues/2582 [i915#2867]: https://gitlab.freedesktop.org/drm/intel/issues/2867 [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282 [i915#3303]: https://gitlab.freedesktop.org/drm/intel/issues/3303 [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555 [i915#3576]: https://gitlab.freedesktop.org/drm/intel/issues/3576 [i915#3595]: https://gitlab.freedesktop.org/drm/intel/issues/3595 [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637 [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708 [i915#4032]: https://gitlab.freedesktop.org/drm/intel/issues/4032 [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077 [i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078 [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079 [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083 [i915#4086]: https://gitlab.freedesktop.org/drm/intel/issues/4086 [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103 [i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212 [i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213 [i915#4215]: https://gitlab.freedesktop.org/drm/intel/issues/4215 [i915#4303]: https://gitlab.freedesktop.org/drm/intel/issues/4303 [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312 [i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391 [i915#4494]: https://gitlab.freedesktop.org/drm/intel/issues/4494 [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613 [i915#4873]: https://gitlab.freedesktop.org/drm/intel/issues/4873 [i915#4957]: https://gitlab.freedesktop.org/drm/intel/issues/4957 [i915#4962]: https://gitlab.freedesktop.org/drm/intel/issues/4962 [i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983 [i915#5127]: https://gitlab.freedesktop.org/drm/intel/issues/5127 [i915#5192]: https://gitlab.freedesktop.org/drm/intel/issues/5192 [i915#5193]: https://gitlab.freedesktop.org/drm/intel/issues/5193 [i915#5323]: https://gitlab.freedesktop.org/drm/intel/issues/5323 [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533 [i915#5338]: https://gitlab.freedesktop.org/drm/intel/issues/5338 [i915#5339]: https://gitlab.freedesktop.org/drm/intel/issues/5339 [i915#5341]: https://gitlab.freedesktop.org/drm/intel/issues/5341 [i915#5342]: https://gitlab.freedesktop.org/drm/intel/issues/5342 Build changes ------------- * CI: CI-20190529 -> None * IGT: IGT_6383 -> IGTPW_6794 CI-20190529: 20190529 CI_DRM_11368: 66b3d1ac616565206cddf4327ca7c102b651b032 @ git://anongit.freedesktop.org/gfx-ci/linux IGTPW_6794: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/index.html IGT_6383: 1e16f23f496c37b7a5678ddebe89c9482b351bb9 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git == Logs == For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/index.html [-- Attachment #2: Type: text/html, Size: 22783 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [igt-dev] ✓ Fi.CI.IGT: success for Add support to collect code coverage data 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab ` (10 preceding siblings ...) 2022-03-16 15:44 ` [igt-dev] ✓ Fi.CI.BAT: success for Add support to collect code coverage data Patchwork @ 2022-03-16 16:53 ` Patchwork 11 siblings, 0 replies; 14+ messages in thread From: Patchwork @ 2022-03-16 16:53 UTC (permalink / raw) To: Mauro Carvalho Chehab; +Cc: igt-dev [-- Attachment #1: Type: text/plain, Size: 30260 bytes --] == Series Details == Series: Add support to collect code coverage data URL : https://patchwork.freedesktop.org/series/101434/ State : success == Summary == CI Bug Log - changes from IGT_6383_full -> IGTPW_6794_full ==================================================== Summary ------- **SUCCESS** No regressions found. External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/index.html Participating hosts (7 -> 7) ------------------------------ No changes in participating hosts Known issues ------------ Here are the changes found in IGTPW_6794_full that come from known issues: ### IGT changes ### #### Issues hit #### * igt@feature_discovery@display-3x: - shard-glk: NOTRUN -> [SKIP][1] ([fdo#109271]) +83 similar issues [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk8/igt@feature_discovery@display-3x.html - shard-iclb: NOTRUN -> [SKIP][2] ([i915#1839]) [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@feature_discovery@display-3x.html * igt@feature_discovery@psr2: - shard-iclb: [PASS][3] -> [SKIP][4] ([i915#658]) [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb2/igt@feature_discovery@psr2.html [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb7/igt@feature_discovery@psr2.html * igt@gem_create@create-massive: - shard-iclb: NOTRUN -> [DMESG-WARN][5] ([i915#4991]) [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@gem_create@create-massive.html - shard-kbl: NOTRUN -> [DMESG-WARN][6] ([i915#4991]) [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl3/igt@gem_create@create-massive.html - shard-tglb: NOTRUN -> [DMESG-WARN][7] ([i915#4991]) [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb7/igt@gem_create@create-massive.html * igt@gem_ctx_isolation@preservation-s3@vcs0: - shard-apl: NOTRUN -> [DMESG-WARN][8] ([i915#180]) [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl1/igt@gem_ctx_isolation@preservation-s3@vcs0.html * igt@gem_ctx_persistence@processes: - shard-snb: NOTRUN -> [SKIP][9] ([fdo#109271] / [i915#1099]) [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-snb5/igt@gem_ctx_persistence@processes.html * igt@gem_ctx_shared@q-in-order: - shard-snb: NOTRUN -> [SKIP][10] ([fdo#109271]) +157 similar issues [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-snb7/igt@gem_ctx_shared@q-in-order.html * igt@gem_ctx_sseu@engines: - shard-tglb: NOTRUN -> [SKIP][11] ([i915#280]) [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb6/igt@gem_ctx_sseu@engines.html * igt@gem_exec_balancer@parallel-balancer: - shard-iclb: [PASS][12] -> [SKIP][13] ([i915#4525]) [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb2/igt@gem_exec_balancer@parallel-balancer.html [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@gem_exec_balancer@parallel-balancer.html * igt@gem_exec_balancer@parallel-keep-in-fence: - shard-tglb: NOTRUN -> [DMESG-WARN][14] ([i915#5076]) [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb2/igt@gem_exec_balancer@parallel-keep-in-fence.html - shard-kbl: NOTRUN -> [DMESG-WARN][15] ([i915#5076]) [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl6/igt@gem_exec_balancer@parallel-keep-in-fence.html * igt@gem_exec_fair@basic-none-vip@rcs0: - shard-kbl: NOTRUN -> [FAIL][16] ([i915#2842]) +1 similar issue [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl7/igt@gem_exec_fair@basic-none-vip@rcs0.html - shard-tglb: NOTRUN -> [FAIL][17] ([i915#2842]) +2 similar issues [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@gem_exec_fair@basic-none-vip@rcs0.html * igt@gem_exec_fair@basic-pace-solo@rcs0: - shard-iclb: NOTRUN -> [FAIL][18] ([i915#2842]) +2 similar issues [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb8/igt@gem_exec_fair@basic-pace-solo@rcs0.html - shard-glk: NOTRUN -> [FAIL][19] ([i915#2842]) +1 similar issue [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk6/igt@gem_exec_fair@basic-pace-solo@rcs0.html * igt@gem_exec_fair@basic-pace@vecs0: - shard-kbl: [PASS][20] -> [FAIL][21] ([i915#2842]) [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-kbl7/igt@gem_exec_fair@basic-pace@vecs0.html [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl4/igt@gem_exec_fair@basic-pace@vecs0.html - shard-iclb: [PASS][22] -> [FAIL][23] ([i915#2842]) [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb7/igt@gem_exec_fair@basic-pace@vecs0.html [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb6/igt@gem_exec_fair@basic-pace@vecs0.html * igt@gem_exec_params@secure-non-master: - shard-iclb: NOTRUN -> [SKIP][24] ([fdo#112283]) [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@gem_exec_params@secure-non-master.html * igt@gem_exec_whisper@basic-normal-all: - shard-glk: NOTRUN -> [DMESG-WARN][25] ([i915#118]) [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk1/igt@gem_exec_whisper@basic-normal-all.html * igt@gem_huc_copy@huc-copy: - shard-apl: NOTRUN -> [SKIP][26] ([fdo#109271] / [i915#2190]) [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl2/igt@gem_huc_copy@huc-copy.html - shard-glk: NOTRUN -> [SKIP][27] ([fdo#109271] / [i915#2190]) [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk3/igt@gem_huc_copy@huc-copy.html - shard-iclb: NOTRUN -> [SKIP][28] ([i915#2190]) [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@gem_huc_copy@huc-copy.html - shard-kbl: NOTRUN -> [SKIP][29] ([fdo#109271] / [i915#2190]) [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl4/igt@gem_huc_copy@huc-copy.html * igt@gem_lmem_swapping@heavy-random: - shard-apl: NOTRUN -> [SKIP][30] ([fdo#109271] / [i915#4613]) +1 similar issue [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl4/igt@gem_lmem_swapping@heavy-random.html * igt@gem_lmem_swapping@smem-oom: - shard-kbl: NOTRUN -> [SKIP][31] ([fdo#109271] / [i915#4613]) +1 similar issue [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl3/igt@gem_lmem_swapping@smem-oom.html - shard-iclb: NOTRUN -> [SKIP][32] ([i915#4613]) +1 similar issue [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@gem_lmem_swapping@smem-oom.html * igt@gem_lmem_swapping@verify-random: - shard-tglb: NOTRUN -> [SKIP][33] ([i915#4613]) +1 similar issue [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@gem_lmem_swapping@verify-random.html - shard-glk: NOTRUN -> [SKIP][34] ([fdo#109271] / [i915#4613]) [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk4/igt@gem_lmem_swapping@verify-random.html * igt@gem_media_vme: - shard-tglb: NOTRUN -> [SKIP][35] ([i915#284]) [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb6/igt@gem_media_vme.html * igt@gem_pxp@reject-modify-context-protection-off-1: - shard-iclb: NOTRUN -> [SKIP][36] ([i915#4270]) +2 similar issues [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@gem_pxp@reject-modify-context-protection-off-1.html * igt@gem_pxp@verify-pxp-stale-ctx-execution: - shard-tglb: NOTRUN -> [SKIP][37] ([i915#4270]) +2 similar issues [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb2/igt@gem_pxp@verify-pxp-stale-ctx-execution.html * igt@gem_render_copy@y-tiled-mc-ccs-to-vebox-y-tiled: - shard-iclb: NOTRUN -> [SKIP][38] ([i915#768]) +3 similar issues [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb3/igt@gem_render_copy@y-tiled-mc-ccs-to-vebox-y-tiled.html * igt@gem_userptr_blits@create-destroy-unsync: - shard-tglb: NOTRUN -> [SKIP][39] ([i915#3297]) +1 similar issue [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb2/igt@gem_userptr_blits@create-destroy-unsync.html - shard-iclb: NOTRUN -> [SKIP][40] ([i915#3297]) [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@gem_userptr_blits@create-destroy-unsync.html * igt@gen3_render_linear_blits: - shard-tglb: NOTRUN -> [SKIP][41] ([fdo#109289]) +4 similar issues [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@gen3_render_linear_blits.html * igt@gen7_exec_parse@batch-without-end: - shard-iclb: NOTRUN -> [SKIP][42] ([fdo#109289]) +3 similar issues [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb7/igt@gen7_exec_parse@batch-without-end.html * igt@gen9_exec_parse@bb-start-param: - shard-tglb: NOTRUN -> [SKIP][43] ([i915#2527] / [i915#2856]) +2 similar issues [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@gen9_exec_parse@bb-start-param.html - shard-iclb: NOTRUN -> [SKIP][44] ([i915#2856]) +1 similar issue [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb6/igt@gen9_exec_parse@bb-start-param.html * igt@i915_pm_rc6_residency@rc6-idle: - shard-tglb: NOTRUN -> [WARN][45] ([i915#2681] / [i915#2684]) [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@i915_pm_rc6_residency@rc6-idle.html * igt@i915_pm_rpm@modeset-pc8-residency-stress: - shard-tglb: NOTRUN -> [SKIP][46] ([fdo#109506] / [i915#2411]) [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@i915_pm_rpm@modeset-pc8-residency-stress.html * igt@kms_addfb_basic@invalid-smem-bo-on-discrete: - shard-tglb: NOTRUN -> [SKIP][47] ([i915#3826]) [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_addfb_basic@invalid-smem-bo-on-discrete.html - shard-iclb: NOTRUN -> [SKIP][48] ([i915#3826]) [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@kms_addfb_basic@invalid-smem-bo-on-discrete.html * igt@kms_atomic_transition@plane-all-modeset-transition-fencing: - shard-iclb: NOTRUN -> [SKIP][49] ([i915#1769]) [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb8/igt@kms_atomic_transition@plane-all-modeset-transition-fencing.html - shard-tglb: NOTRUN -> [SKIP][50] ([i915#1769]) [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_atomic_transition@plane-all-modeset-transition-fencing.html * igt@kms_big_fb@4-tiled-8bpp-rotate-180: - shard-iclb: NOTRUN -> [SKIP][51] ([i915#5286]) +3 similar issues [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb6/igt@kms_big_fb@4-tiled-8bpp-rotate-180.html * igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip: - shard-tglb: NOTRUN -> [SKIP][52] ([i915#5286]) +4 similar issues [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html * igt@kms_big_fb@linear-16bpp-rotate-90: - shard-iclb: NOTRUN -> [SKIP][53] ([fdo#110725] / [fdo#111614]) +5 similar issues [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb6/igt@kms_big_fb@linear-16bpp-rotate-90.html * igt@kms_big_fb@linear-32bpp-rotate-180: - shard-glk: [PASS][54] -> [DMESG-WARN][55] ([i915#118]) +2 similar issues [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-glk6/igt@kms_big_fb@linear-32bpp-rotate-180.html [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk1/igt@kms_big_fb@linear-32bpp-rotate-180.html * igt@kms_big_fb@x-tiled-32bpp-rotate-90: - shard-tglb: NOTRUN -> [SKIP][56] ([fdo#111614]) +4 similar issues [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb6/igt@kms_big_fb@x-tiled-32bpp-rotate-90.html * igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip: - shard-apl: NOTRUN -> [SKIP][57] ([fdo#109271] / [i915#3777]) +1 similar issue [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl7/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html * igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip: - shard-kbl: NOTRUN -> [SKIP][58] ([fdo#109271] / [i915#3777]) [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl7/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip.html * igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip: - shard-glk: NOTRUN -> [SKIP][59] ([fdo#109271] / [i915#3777]) +1 similar issue [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk3/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html * igt@kms_big_fb@yf-tiled-8bpp-rotate-180: - shard-tglb: NOTRUN -> [SKIP][60] ([fdo#111615]) +3 similar issues [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_big_fb@yf-tiled-8bpp-rotate-180.html * igt@kms_big_fb@yf-tiled-8bpp-rotate-270: - shard-iclb: NOTRUN -> [SKIP][61] ([fdo#110723]) +3 similar issues [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@kms_big_fb@yf-tiled-8bpp-rotate-270.html * igt@kms_big_joiner@invalid-modeset: - shard-iclb: NOTRUN -> [SKIP][62] ([i915#2705]) [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@kms_big_joiner@invalid-modeset.html - shard-tglb: NOTRUN -> [SKIP][63] ([i915#2705]) +1 similar issue [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_big_joiner@invalid-modeset.html * igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs: - shard-glk: NOTRUN -> [SKIP][64] ([fdo#109271] / [i915#3886]) +7 similar issues [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk1/igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs.html * igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_ccs: - shard-tglb: NOTRUN -> [SKIP][65] ([i915#3689]) +4 similar issues [65]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_ccs.html * igt@kms_ccs@pipe-b-bad-rotation-90-y_tiled_gen12_mc_ccs: - shard-tglb: NOTRUN -> [SKIP][66] ([i915#3689] / [i915#3886]) +5 similar issues [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb2/igt@kms_ccs@pipe-b-bad-rotation-90-y_tiled_gen12_mc_ccs.html * igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc: - shard-kbl: NOTRUN -> [SKIP][67] ([fdo#109271] / [i915#3886]) +9 similar issues [67]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl1/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html * igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_mc_ccs: - shard-iclb: NOTRUN -> [SKIP][68] ([fdo#109278] / [i915#3886]) +8 similar issues [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_mc_ccs.html * igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs: - shard-apl: NOTRUN -> [SKIP][69] ([fdo#109271] / [i915#3886]) +10 similar issues [69]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl4/igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs.html * igt@kms_ccs@pipe-c-crc-primary-rotation-180-yf_tiled_ccs: - shard-tglb: NOTRUN -> [SKIP][70] ([fdo#111615] / [i915#3689]) +5 similar issues [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_ccs@pipe-c-crc-primary-rotation-180-yf_tiled_ccs.html * igt@kms_ccs@pipe-d-bad-rotation-90-y_tiled_gen12_rc_ccs_cc: - shard-iclb: NOTRUN -> [SKIP][71] ([fdo#109278]) +36 similar issues [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb7/igt@kms_ccs@pipe-d-bad-rotation-90-y_tiled_gen12_rc_ccs_cc.html * igt@kms_chamelium@hdmi-mode-timings: - shard-kbl: NOTRUN -> [SKIP][72] ([fdo#109271] / [fdo#111827]) +9 similar issues [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl3/igt@kms_chamelium@hdmi-mode-timings.html - shard-iclb: NOTRUN -> [SKIP][73] ([fdo#109284] / [fdo#111827]) +7 similar issues [73]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb3/igt@kms_chamelium@hdmi-mode-timings.html * igt@kms_chamelium@vga-hpd: - shard-apl: NOTRUN -> [SKIP][74] ([fdo#109271] / [fdo#111827]) +6 similar issues [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl8/igt@kms_chamelium@vga-hpd.html * igt@kms_color_chamelium@pipe-c-ctm-max: - shard-snb: NOTRUN -> [SKIP][75] ([fdo#109271] / [fdo#111827]) +1 similar issue [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-snb2/igt@kms_color_chamelium@pipe-c-ctm-max.html * igt@kms_color_chamelium@pipe-d-ctm-limited-range: - shard-tglb: NOTRUN -> [SKIP][76] ([fdo#109284] / [fdo#111827]) +11 similar issues [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb6/igt@kms_color_chamelium@pipe-d-ctm-limited-range.html - shard-glk: NOTRUN -> [SKIP][77] ([fdo#109271] / [fdo#111827]) +1 similar issue [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk7/igt@kms_color_chamelium@pipe-d-ctm-limited-range.html - shard-iclb: NOTRUN -> [SKIP][78] ([fdo#109278] / [fdo#109284] / [fdo#111827]) [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@kms_color_chamelium@pipe-d-ctm-limited-range.html * igt@kms_content_protection@legacy: - shard-iclb: NOTRUN -> [SKIP][79] ([fdo#109300] / [fdo#111066]) [79]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@kms_content_protection@legacy.html - shard-kbl: NOTRUN -> [TIMEOUT][80] ([i915#1319]) [80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl6/igt@kms_content_protection@legacy.html - shard-tglb: NOTRUN -> [SKIP][81] ([i915#1063]) [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_content_protection@legacy.html - shard-apl: NOTRUN -> [TIMEOUT][82] ([i915#1319]) [82]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl1/igt@kms_content_protection@legacy.html * igt@kms_cursor_crc@pipe-a-cursor-32x32-sliding: - shard-tglb: NOTRUN -> [SKIP][83] ([i915#3319]) +2 similar issues [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_cursor_crc@pipe-a-cursor-32x32-sliding.html * igt@kms_cursor_crc@pipe-a-cursor-max-size-rapid-movement: - shard-tglb: NOTRUN -> [SKIP][84] ([i915#3359]) +6 similar issues [84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb1/igt@kms_cursor_crc@pipe-a-cursor-max-size-rapid-movement.html * igt@kms_cursor_crc@pipe-c-cursor-512x512-random: - shard-tglb: NOTRUN -> [SKIP][85] ([fdo#109279] / [i915#3359]) +1 similar issue [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb7/igt@kms_cursor_crc@pipe-c-cursor-512x512-random.html - shard-iclb: NOTRUN -> [SKIP][86] ([fdo#109278] / [fdo#109279]) [86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@kms_cursor_crc@pipe-c-cursor-512x512-random.html * igt@kms_cursor_legacy@2x-long-cursor-vs-flip-atomic: - shard-iclb: NOTRUN -> [SKIP][87] ([fdo#109274] / [fdo#109278]) +4 similar issues [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb5/igt@kms_cursor_legacy@2x-long-cursor-vs-flip-atomic.html * igt@kms_cursor_legacy@pipe-d-torture-bo: - shard-kbl: NOTRUN -> [SKIP][88] ([fdo#109271] / [i915#533]) +3 similar issues [88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl4/igt@kms_cursor_legacy@pipe-d-torture-bo.html - shard-glk: NOTRUN -> [SKIP][89] ([fdo#109271] / [i915#533]) +1 similar issue [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk5/igt@kms_cursor_legacy@pipe-d-torture-bo.html * igt@kms_cursor_legacy@short-busy-flip-before-cursor-toggle: - shard-tglb: NOTRUN -> [SKIP][90] ([i915#4103]) +1 similar issue [90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_cursor_legacy@short-busy-flip-before-cursor-toggle.html * igt@kms_display_modes@extended-mode-basic: - shard-iclb: NOTRUN -> [SKIP][91] ([fdo#109274]) +7 similar issues [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@kms_display_modes@extended-mode-basic.html - shard-tglb: NOTRUN -> [SKIP][92] ([fdo#109274]) [92]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb6/igt@kms_display_modes@extended-mode-basic.html * igt@kms_draw_crc@draw-method-xrgb2101010-blt-4tiled: - shard-tglb: NOTRUN -> [SKIP][93] ([i915#5287]) +2 similar issues [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_draw_crc@draw-method-xrgb2101010-blt-4tiled.html * igt@kms_draw_crc@draw-method-xrgb8888-pwrite-4tiled: - shard-iclb: NOTRUN -> [SKIP][94] ([i915#5287]) +2 similar issues [94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@kms_draw_crc@draw-method-xrgb8888-pwrite-4tiled.html * igt@kms_dsc@basic-dsc-enable: - shard-iclb: NOTRUN -> [SKIP][95] ([i915#3840]) [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@kms_dsc@basic-dsc-enable.html * igt@kms_flip@2x-blocking-absolute-wf_vblank-interruptible: - shard-tglb: NOTRUN -> [SKIP][96] ([fdo#109274] / [fdo#111825]) +11 similar issues [96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_flip@2x-blocking-absolute-wf_vblank-interruptible.html * igt@kms_flip@flip-vs-suspend@a-dp1: - shard-apl: [PASS][97] -> [DMESG-WARN][98] ([i915#180]) +1 similar issue [97]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-apl7/igt@kms_flip@flip-vs-suspend@a-dp1.html [98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl7/igt@kms_flip@flip-vs-suspend@a-dp1.html * igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-upscaling: - shard-tglb: NOTRUN -> [SKIP][99] ([i915#2587]) [99]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb8/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-upscaling.html * igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling: - shard-iclb: [PASS][100] -> [SKIP][101] ([i915#3701]) [100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb1/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling.html [101]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling.html * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-spr-indfb-draw-mmap-cpu: - shard-iclb: NOTRUN -> [SKIP][102] ([fdo#109280]) +24 similar issues [102]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-spr-indfb-draw-mmap-cpu.html * igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-cpu: - shard-tglb: NOTRUN -> [SKIP][103] ([fdo#109280] / [fdo#111825]) +34 similar issues [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb7/igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-cpu.html * igt@kms_hdr@static-toggle-dpms: - shard-tglb: NOTRUN -> [SKIP][104] ([i915#3555]) +3 similar issues [104]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@kms_hdr@static-toggle-dpms.html * igt@kms_pipe_crc_basic@hang-read-crc-pipe-d: - shard-apl: NOTRUN -> [SKIP][105] ([fdo#109271] / [i915#533]) [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl3/igt@kms_pipe_crc_basic@hang-read-crc-pipe-d.html * igt@kms_plane_alpha_blend@pipe-b-constant-alpha-max: - shard-kbl: NOTRUN -> [FAIL][106] ([fdo#108145] / [i915#265]) [106]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl4/igt@kms_plane_alpha_blend@pipe-b-constant-alpha-max.html * igt@kms_plane_lowres@pipe-a-tiling-y: - shard-tglb: NOTRUN -> [SKIP][107] ([i915#3536]) [107]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb8/igt@kms_plane_lowres@pipe-a-tiling-y.html * igt@kms_plane_lowres@pipe-c-tiling-4: - shard-iclb: NOTRUN -> [SKIP][108] ([i915#5288]) [108]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@kms_plane_lowres@pipe-c-tiling-4.html * igt@kms_plane_multiple@atomic-pipe-c-tiling-4: - shard-tglb: NOTRUN -> [SKIP][109] ([i915#5288]) +1 similar issue [109]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_plane_multiple@atomic-pipe-c-tiling-4.html * igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-75@pipe-b-edp-1-downscale-with-pixel-format: - shard-iclb: [PASS][110] -> [INCOMPLETE][111] ([i915#5293]) [110]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb3/igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-75@pipe-b-edp-1-downscale-with-pixel-format.html [111]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb2/igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-75@pipe-b-edp-1-downscale-with-pixel-format.html * igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-a-dp-1-downscale-with-rotation: - shard-kbl: NOTRUN -> [SKIP][112] ([fdo#109271]) +165 similar issues [112]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl1/igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-a-dp-1-downscale-with-rotation.html * igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-a-edp-1-downscale-with-rotation: - shard-iclb: NOTRUN -> [SKIP][113] ([i915#5176]) +2 similar issues [113]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb7/igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-a-edp-1-downscale-with-rotation.html * igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-b-edp-1-downscale-with-rotation: - shard-tglb: NOTRUN -> [SKIP][114] ([i915#5176]) +3 similar issues [114]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb2/igt@kms_plane_scaling@downscale-with-rotation-factor-0-5@pipe-b-edp-1-downscale-with-rotation.html * igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-25@pipe-d-edp-1-planes-upscale-downscale: - shard-tglb: NOTRUN -> [SKIP][115] ([i915#5235]) +7 similar issues [115]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb7/igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-25@pipe-d-edp-1-planes-upscale-downscale.html * igt@kms_psr2_sf@overlay-plane-update-continuous-sf: - shard-iclb: NOTRUN -> [SKIP][116] ([fdo#111068] / [i915#658]) [116]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb7/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html - shard-glk: NOTRUN -> [SKIP][117] ([fdo#109271] / [i915#658]) [117]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-glk7/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html * igt@kms_psr2_sf@primary-plane-update-sf-dmg-area: - shard-kbl: NOTRUN -> [SKIP][118] ([fdo#109271] / [i915#658]) +1 similar issue [118]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-kbl1/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area.html - shard-tglb: NOTRUN -> [SKIP][119] ([i915#2920]) +1 similar issue [119]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area.html * igt@kms_psr2_su@page_flip-nv12: - shard-apl: NOTRUN -> [SKIP][120] ([fdo#109271] / [i915#658]) +1 similar issue [120]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-apl8/igt@kms_psr2_su@page_flip-nv12.html - shard-iclb: NOTRUN -> [SKIP][121] ([fdo#109642] / [fdo#111068] / [i915#658]) [121]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb3/igt@kms_psr2_su@page_flip-nv12.html * igt@kms_psr@psr2_cursor_plane_onoff: - shard-iclb: [PASS][122] -> [SKIP][123] ([fdo#109441]) [122]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6383/shard-iclb2/igt@kms_psr@psr2_cursor_plane_onoff.html [123]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb4/igt@kms_psr@psr2_cursor_plane_onoff.html * igt@kms_psr@psr2_primary_page_flip: - shard-iclb: NOTRUN -> [SKIP][124] ([fdo#109441]) +2 similar issues [124]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-iclb1/igt@kms_psr@psr2_primary_page_flip.html * igt@kms_psr@psr2_sprite_mmap_cpu: - shard-tglb: NOTRUN -> [FAIL][125] ([i915#132] / [i915#3467]) +3 similar issues [125]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb3/igt@kms_psr@psr2_sprite_mmap_cpu.html * igt@kms_scaling_modes@scaling-mode-none@edp-1-pipe-a: - shard-tglb: NOTRUN -> [SKIP][126] ([i915#5030]) +3 similar issues [126]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/shard-tglb5/igt@kms_scaling_modes@scaling-mode-none@edp-1-pipe-a.html * igt@kms_scaling_modes@scaling-mode-none@edp-1-pipe-c: - shard-iclb: NOTRUN -> [SKIP][127] ([i915#5030]) +2 similar issues [ == Logs == For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6794/index.html [-- Attachment #2: Type: text/html, Size: 33945 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* [igt-dev] [PATCH v2 0/9] Add support to collect code coverage data
@ 2022-03-01 7:59 Mauro Carvalho Chehab
2022-03-01 15:48 ` [igt-dev] ✓ Fi.CI.IGT: success for " Patchwork
0 siblings, 1 reply; 14+ messages in thread
From: Mauro Carvalho Chehab @ 2022-03-01 7:59 UTC (permalink / raw)
To: igt-dev
From: Mauro Carvalho Chehab <mchehab@kernel.org>
The gcc compiler has a feature that allows collecting code coverage
in runtime, which is already supported by the Linux Kernel, as described
on its documentation:
https://01.org/linuxgraphics/gfx-docs/drm/dev-tools/gcov.html
Add support for it at the igt_runner and scripts/run-tests.sh.
With such support, once a Kernel is compiled with:
./scripts/config -e DEBUG_FS -e GCOV_KERNEL
And GCOV is enabled by one or more drivers, like doing:
for i in $(find drivers/gpu/drm/ -name Makefile); do sed '1 a GCOV_PROFILE := y' -i $i; done
One can use igt_runner and/or run-tests.sh to collect the code coverage.
The way the implementation works is that the igt_runner has internally
the logic which reset the gcov usage counters. The code capture itself
is done via an external script.
There are two versions of such script:
- scripts/code_cov_capture.sh:
Assumes that the Kernel was built at the same machine, and uses
the lcov tool to generate GCC-independent code coverage data,
in the form of *.info files;
- scripts/code_cov_gather_on_test.py:
Generates a gzipped tarbal with the code coverage counters in
binary format. Such kind of output should then be parsed at
the same machine where the Kernel as built, as its content is not
ony dependent on the Kernel source, but also on the Kernel output
objects.
Either way, both igt_runner and scripts/run-tests.sh could be used
to generate the code coverage results.
So, in order to get the code coverage results, storing results per each
IGT test, one should run either:
./scripts/run-tests.sh -T my_tests.testlist -k ~/linux \
-c scripts/code_cov_gather_on_test.py -P
or:
sudo ./build/runner/igt_runner -o --test-list my_tests.testlist \
--coverage-per-test \
--collect-script scripts/code_cov_gather_on_test.py \
build/tests results
Once the data is captured, it is possible to generate HTML reports
using scripts/code_cov_gen_report.sh:
scripts/code_cov_gen_report.sh -r results/code_cov/i915_selftest_live.info -k ~/linux -o cov_results -i -f
v2:
- Applied this commant to patch 8:
sed 's!/home/gta/!/basedir/!;s,\./basedir,/basedir,' -i docs/code_coverage.md
Mauro Carvalho Chehab (8):
runner: check if it has root permissions
runner: Add support for code coverage
scripts/code_cov_gather*/sh: add help scripts for code coverage
scripts/code_cov_gather_on_build.sh: Improve the script
scripts/code_cov_capture.sh: add a script to use lcov on build+test
machine
scripts/code_cov_gen_report.sh: add a script to generate code coverage
reports
scripts/run-tests.sh: add code coverage support
docs: add documentation for code coverage
Tomi Sarvela (1):
scripts:code_cov_gather_on_test: use a faster script
docs/code_coverage.md | 293 ++++++++++++++++++++++++++++
runner/executor.c | 200 ++++++++++++++++++-
runner/resume.c | 4 +-
runner/runner_tests.c | 30 +++
runner/settings.c | 87 ++++++++-
runner/settings.h | 8 +
scripts/code_cov_capture.sh | 25 +++
scripts/code_cov_gather_on_build.sh | 39 ++++
scripts/code_cov_gather_on_test.py | 91 +++++++++
scripts/code_cov_gen_report.sh | 170 ++++++++++++++++
scripts/run-tests.sh | 52 ++++-
11 files changed, 979 insertions(+), 20 deletions(-)
create mode 100644 docs/code_coverage.md
create mode 100755 scripts/code_cov_capture.sh
create mode 100755 scripts/code_cov_gather_on_build.sh
create mode 100755 scripts/code_cov_gather_on_test.py
create mode 100755 scripts/code_cov_gen_report.sh
--
2.35.1
^ permalink raw reply [flat|nested] 14+ messages in thread* [igt-dev] ✓ Fi.CI.IGT: success for Add support to collect code coverage data 2022-03-01 7:59 [igt-dev] [PATCH v2 0/9] " Mauro Carvalho Chehab @ 2022-03-01 15:48 ` Patchwork 0 siblings, 0 replies; 14+ messages in thread From: Patchwork @ 2022-03-01 15:48 UTC (permalink / raw) To: Mauro Carvalho Chehab; +Cc: igt-dev [-- Attachment #1: Type: text/plain, Size: 30260 bytes --] == Series Details == Series: Add support to collect code coverage data URL : https://patchwork.freedesktop.org/series/100875/ State : success == Summary == CI Bug Log - changes from CI_DRM_11301_full -> IGTPW_6721_full ==================================================== Summary ------- **SUCCESS** No regressions found. External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/index.html Participating hosts (13 -> 8) ------------------------------ Missing (5): pig-kbl-iris pig-glk-j5005 pig-skl-6260u shard-rkl shard-dg1 Possible new issues ------------------- Here are the unknown changes that may have been introduced in IGTPW_6721_full: ### IGT changes ### #### Possible regressions #### * {igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-d-edp-1-downscale-with-pixel-format} (NEW): - shard-tglb: NOTRUN -> [SKIP][1] [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-d-edp-1-downscale-with-pixel-format.html #### Suppressed #### The following results come from untrusted machines, tests, or statuses. They do not affect the overall result. * {igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-a-edp-1-downscale-with-pixel-format}: - shard-iclb: NOTRUN -> [SKIP][2] +2 similar issues [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-a-edp-1-downscale-with-pixel-format.html * {igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-b-edp-1-downscale-with-pixel-format}: - shard-tglb: NOTRUN -> [SKIP][3] +2 similar issues [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-b-edp-1-downscale-with-pixel-format.html * {igt@kms_plane_scaling@planes-downscale-factor-0-5@pipe-a-edp-1-planes-downscale}: - shard-iclb: [PASS][4] -> [SKIP][5] +2 similar issues [4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-iclb1/igt@kms_plane_scaling@planes-downscale-factor-0-5@pipe-a-edp-1-planes-downscale.html [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@kms_plane_scaling@planes-downscale-factor-0-5@pipe-a-edp-1-planes-downscale.html New tests --------- New tests have been introduced between CI_DRM_11301_full and IGTPW_6721_full: ### New IGT tests (24) ### * igt@gem_create@create-massive: - Statuses : 7 dmesg-warn(s) - Exec time: [0.00] s * igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-25@pipe-d-edp-1-downscale-with-pixel-format: - Statuses : 1 skip(s) - Exec time: [0.02] s * igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-75@pipe-b-hdmi-a-1-downscale-with-pixel-format: - Statuses : 1 pass(s) - Exec time: [22.83] s * igt@kms_plane_scaling@downscale-with-pixel-format-factor-0-75@pipe-d-hdmi-a-1-downscale-with-pixel-format: - Statuses : 1 pass(s) - Exec time: [0.28] s * igt@kms_plane_scaling@invalid-num-scalers@pipe-d-edp-1-invalid-num-scalers: - Statuses : 1 pass(s) - Exec time: [0.02] s * igt@kms_plane_scaling@planes-scaling-unity-scaling@pipe-b-hdmi-a-1-planes-unity-scaling: - Statuses : 1 pass(s) - Exec time: [0.12] s * igt@kms_plane_scaling@planes-scaling-unity-scaling@pipe-d-edp-1-planes-unity-scaling: - Statuses : 1 pass(s) - Exec time: [1.28] s * igt@kms_plane_scaling@planes-scaling-unity-scaling@pipe-d-hdmi-a-1-planes-unity-scaling: - Statuses : 1 pass(s) - Exec time: [0.12] s * igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-75@pipe-b-hdmi-a-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [0.16] s * igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-75@pipe-d-edp-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [1.28] s * igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-75@pipe-d-hdmi-a-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [0.15] s * igt@kms_plane_scaling@planes-upscale-20x20@pipe-b-hdmi-a-1-planes-upscale: - Statuses : 1 pass(s) - Exec time: [0.12] s * igt@kms_plane_scaling@planes-upscale-20x20@pipe-d-edp-1-planes-upscale: - Statuses : 1 pass(s) - Exec time: [1.22] s * igt@kms_plane_scaling@planes-upscale-20x20@pipe-d-hdmi-a-1-planes-upscale: - Statuses : 1 pass(s) - Exec time: [0.10] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-a-edp-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [0.14] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-a-hdmi-a-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [0.35] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-a-vga-1-planes-upscale-downscale: - Statuses : 1 skip(s) - Exec time: [0.05] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-b-edp-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [1.28] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-b-hdmi-a-2-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [0.35] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-b-vga-1-planes-upscale-downscale: - Statuses : 1 skip(s) - Exec time: [0.03] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-c-edp-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [1.28] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-c-hdmi-a-1-planes-upscale-downscale: - Statuses : 1 skip(s) - Exec time: [0.09] s * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-75@pipe-d-edp-1-planes-upscale-downscale: - Statuses : 1 pass(s) - Exec time: [1.22] s * igt@kms_plane_scaling@planes-upscale-factor-0-25@pipe-d-edp-1-planes-upscale: - Statuses : 1 pass(s) - Exec time: [1.22] s Known issues ------------ Here are the changes found in IGTPW_6721_full that come from known issues: ### IGT changes ### #### Issues hit #### * igt@gem_ctx_param@set-priority-not-supported: - shard-tglb: NOTRUN -> [SKIP][6] ([fdo#109314]) [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@gem_ctx_param@set-priority-not-supported.html - shard-iclb: NOTRUN -> [SKIP][7] ([fdo#109314]) [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@gem_ctx_param@set-priority-not-supported.html * igt@gem_ctx_persistence@legacy-engines-queued: - shard-snb: NOTRUN -> [SKIP][8] ([fdo#109271] / [i915#1099]) +1 similar issue [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-snb2/igt@gem_ctx_persistence@legacy-engines-queued.html * igt@gem_eio@in-flight-1us: - shard-tglb: [PASS][9] -> [TIMEOUT][10] ([i915#3063]) [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-tglb5/igt@gem_eio@in-flight-1us.html [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb5/igt@gem_eio@in-flight-1us.html * igt@gem_exec_balancer@parallel-keep-submit-fence: - shard-kbl: NOTRUN -> [DMESG-WARN][11] ([i915#5076]) [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl3/igt@gem_exec_balancer@parallel-keep-submit-fence.html * igt@gem_exec_fair@basic-none@vcs0: - shard-tglb: NOTRUN -> [FAIL][12] ([i915#2842]) +4 similar issues [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@gem_exec_fair@basic-none@vcs0.html * igt@gem_exec_fair@basic-none@vecs0: - shard-iclb: NOTRUN -> [FAIL][13] ([i915#2842]) +3 similar issues [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb3/igt@gem_exec_fair@basic-none@vecs0.html * igt@gem_exec_fair@basic-pace-share@rcs0: - shard-glk: [PASS][14] -> [FAIL][15] ([i915#2842]) [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-glk1/igt@gem_exec_fair@basic-pace-share@rcs0.html [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk9/igt@gem_exec_fair@basic-pace-share@rcs0.html * igt@gem_exec_params@secure-non-root: - shard-iclb: NOTRUN -> [SKIP][16] ([fdo#112283]) [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb6/igt@gem_exec_params@secure-non-root.html * igt@gem_lmem_swapping@basic: - shard-iclb: NOTRUN -> [SKIP][17] ([i915#4613]) [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@gem_lmem_swapping@basic.html * igt@gem_lmem_swapping@heavy-random: - shard-glk: NOTRUN -> [SKIP][18] ([fdo#109271] / [i915#4613]) [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk2/igt@gem_lmem_swapping@heavy-random.html * igt@gem_lmem_swapping@heavy-verify-random: - shard-kbl: NOTRUN -> [SKIP][19] ([fdo#109271] / [i915#4613]) +1 similar issue [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl3/igt@gem_lmem_swapping@heavy-verify-random.html - shard-tglb: NOTRUN -> [SKIP][20] ([i915#4613]) +1 similar issue [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb7/igt@gem_lmem_swapping@heavy-verify-random.html * igt@gem_lmem_swapping@parallel-random-verify: - shard-apl: NOTRUN -> [SKIP][21] ([fdo#109271] / [i915#4613]) [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl2/igt@gem_lmem_swapping@parallel-random-verify.html * igt@gem_pxp@create-protected-buffer: - shard-iclb: NOTRUN -> [SKIP][22] ([i915#4270]) +1 similar issue [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@gem_pxp@create-protected-buffer.html * igt@gem_pxp@create-regular-buffer: - shard-tglb: NOTRUN -> [SKIP][23] ([i915#4270]) +4 similar issues [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb3/igt@gem_pxp@create-regular-buffer.html * igt@gem_userptr_blits@unsync-unmap-cycles: - shard-iclb: NOTRUN -> [SKIP][24] ([i915#3297]) [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb7/igt@gem_userptr_blits@unsync-unmap-cycles.html * igt@gen3_render_tiledy_blits: - shard-tglb: NOTRUN -> [SKIP][25] ([fdo#109289]) +3 similar issues [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@gen3_render_tiledy_blits.html * igt@gen7_exec_parse@chained-batch: - shard-iclb: NOTRUN -> [SKIP][26] ([fdo#109289]) [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@gen7_exec_parse@chained-batch.html * igt@gen9_exec_parse@batch-without-end: - shard-iclb: NOTRUN -> [SKIP][27] ([i915#2856]) +1 similar issue [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@gen9_exec_parse@batch-without-end.html * igt@gen9_exec_parse@shadow-peek: - shard-tglb: NOTRUN -> [SKIP][28] ([i915#2527] / [i915#2856]) +2 similar issues [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb5/igt@gen9_exec_parse@shadow-peek.html * igt@i915_pm_rc6_residency@media-rc6-accuracy: - shard-tglb: NOTRUN -> [SKIP][29] ([fdo#109289] / [fdo#111719]) [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb6/igt@i915_pm_rc6_residency@media-rc6-accuracy.html * igt@i915_pm_rc6_residency@rc6-idle: - shard-tglb: NOTRUN -> [WARN][30] ([i915#2681] / [i915#2684]) [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb3/igt@i915_pm_rc6_residency@rc6-idle.html * igt@i915_pm_rpm@gem-execbuf-stress-pc8: - shard-iclb: NOTRUN -> [SKIP][31] ([fdo#109293] / [fdo#109506]) [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@i915_pm_rpm@gem-execbuf-stress-pc8.html - shard-tglb: NOTRUN -> [SKIP][32] ([fdo#109506] / [i915#2411]) [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb8/igt@i915_pm_rpm@gem-execbuf-stress-pc8.html * igt@i915_pm_sseu@full-enable: - shard-tglb: NOTRUN -> [SKIP][33] ([i915#4387]) [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb8/igt@i915_pm_sseu@full-enable.html * igt@i915_selftest@live@hangcheck: - shard-snb: [PASS][34] -> [INCOMPLETE][35] ([i915#3921]) [34]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-snb6/igt@i915_selftest@live@hangcheck.html [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-snb7/igt@i915_selftest@live@hangcheck.html * igt@i915_suspend@debugfs-reader: - shard-apl: NOTRUN -> [DMESG-WARN][36] ([i915#180]) [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl3/igt@i915_suspend@debugfs-reader.html - shard-kbl: NOTRUN -> [DMESG-WARN][37] ([i915#180]) [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl6/igt@i915_suspend@debugfs-reader.html * igt@kms_atomic_transition@plane-all-modeset-transition: - shard-iclb: NOTRUN -> [SKIP][38] ([i915#1769]) [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb6/igt@kms_atomic_transition@plane-all-modeset-transition.html - shard-tglb: NOTRUN -> [SKIP][39] ([i915#1769]) [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb3/igt@kms_atomic_transition@plane-all-modeset-transition.html * igt@kms_big_fb@linear-32bpp-rotate-0: - shard-glk: [PASS][40] -> [DMESG-WARN][41] ([i915#118]) [40]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-glk1/igt@kms_big_fb@linear-32bpp-rotate-0.html [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk6/igt@kms_big_fb@linear-32bpp-rotate-0.html * igt@kms_big_fb@linear-32bpp-rotate-180: - shard-glk: [PASS][42] -> [DMESG-WARN][43] ([i915#118] / [i915#1982]) [42]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-glk6/igt@kms_big_fb@linear-32bpp-rotate-180.html [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk3/igt@kms_big_fb@linear-32bpp-rotate-180.html * igt@kms_big_fb@linear-8bpp-rotate-270: - shard-tglb: NOTRUN -> [SKIP][44] ([fdo#111614]) +3 similar issues [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb6/igt@kms_big_fb@linear-8bpp-rotate-270.html * igt@kms_big_fb@x-tiled-64bpp-rotate-270: - shard-iclb: NOTRUN -> [SKIP][45] ([fdo#110725] / [fdo#111614]) [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@kms_big_fb@x-tiled-64bpp-rotate-270.html * igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip: - shard-kbl: NOTRUN -> [SKIP][46] ([fdo#109271] / [i915#3777]) +2 similar issues [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl1/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip.html * igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip: - shard-apl: NOTRUN -> [SKIP][47] ([fdo#109271] / [i915#3777]) +1 similar issue [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl6/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html * igt@kms_big_fb@yf-tiled-64bpp-rotate-270: - shard-tglb: NOTRUN -> [SKIP][48] ([fdo#111615]) +8 similar issues [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@kms_big_fb@yf-tiled-64bpp-rotate-270.html * igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0: - shard-apl: NOTRUN -> [SKIP][49] ([fdo#109271]) +189 similar issues [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl2/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html - shard-iclb: NOTRUN -> [SKIP][50] ([fdo#110723]) +2 similar issues [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb8/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html * igt@kms_big_joiner@basic: - shard-tglb: NOTRUN -> [SKIP][51] ([i915#2705]) [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@kms_big_joiner@basic.html - shard-iclb: NOTRUN -> [SKIP][52] ([i915#2705]) [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@kms_big_joiner@basic.html * igt@kms_ccs@pipe-a-bad-aux-stride-y_tiled_gen12_rc_ccs_cc: - shard-glk: NOTRUN -> [SKIP][53] ([fdo#109271] / [i915#3886]) +7 similar issues [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk7/igt@kms_ccs@pipe-a-bad-aux-stride-y_tiled_gen12_rc_ccs_cc.html - shard-iclb: NOTRUN -> [SKIP][54] ([fdo#109278] / [i915#3886]) +8 similar issues [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb1/igt@kms_ccs@pipe-a-bad-aux-stride-y_tiled_gen12_rc_ccs_cc.html * igt@kms_ccs@pipe-a-crc-primary-rotation-180-y_tiled_gen12_rc_ccs_cc: - shard-apl: NOTRUN -> [SKIP][55] ([fdo#109271] / [i915#3886]) +8 similar issues [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl7/igt@kms_ccs@pipe-a-crc-primary-rotation-180-y_tiled_gen12_rc_ccs_cc.html * igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_ccs: - shard-tglb: NOTRUN -> [SKIP][56] ([i915#3689]) +8 similar issues [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb6/igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_ccs.html * igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_mc_ccs: - shard-kbl: NOTRUN -> [SKIP][57] ([fdo#109271] / [i915#3886]) +7 similar issues [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl1/igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html * igt@kms_ccs@pipe-b-bad-pixel-format-y_tiled_ccs: - shard-snb: NOTRUN -> [SKIP][58] ([fdo#109271]) +137 similar issues [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-snb5/igt@kms_ccs@pipe-b-bad-pixel-format-y_tiled_ccs.html * igt@kms_ccs@pipe-b-crc-primary-basic-yf_tiled_ccs: - shard-tglb: NOTRUN -> [SKIP][59] ([fdo#111615] / [i915#3689]) +5 similar issues [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb7/igt@kms_ccs@pipe-b-crc-primary-basic-yf_tiled_ccs.html * igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs: - shard-tglb: NOTRUN -> [SKIP][60] ([i915#3689] / [i915#3886]) +4 similar issues [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb8/igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs.html * igt@kms_cdclk@plane-scaling: - shard-tglb: NOTRUN -> [SKIP][61] ([i915#3742]) [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@kms_cdclk@plane-scaling.html * igt@kms_chamelium@dp-crc-fast: - shard-iclb: NOTRUN -> [SKIP][62] ([fdo#109284] / [fdo#111827]) +9 similar issues [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@kms_chamelium@dp-crc-fast.html * igt@kms_chamelium@dp-mode-timings: - shard-apl: NOTRUN -> [SKIP][63] ([fdo#109271] / [fdo#111827]) +21 similar issues [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl3/igt@kms_chamelium@dp-mode-timings.html * igt@kms_chamelium@hdmi-hpd: - shard-glk: NOTRUN -> [SKIP][64] ([fdo#109271] / [fdo#111827]) +9 similar issues [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk3/igt@kms_chamelium@hdmi-hpd.html * igt@kms_chamelium@vga-hpd-for-each-pipe: - shard-kbl: NOTRUN -> [SKIP][65] ([fdo#109271] / [fdo#111827]) +18 similar issues [65]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl6/igt@kms_chamelium@vga-hpd-for-each-pipe.html * igt@kms_color@pipe-d-ctm-negative: - shard-iclb: NOTRUN -> [SKIP][66] ([fdo#109278] / [i915#1149]) [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb8/igt@kms_color@pipe-d-ctm-negative.html * igt@kms_color_chamelium@pipe-b-ctm-limited-range: - shard-tglb: NOTRUN -> [SKIP][67] ([fdo#109284] / [fdo#111827]) +15 similar issues [67]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb1/igt@kms_color_chamelium@pipe-b-ctm-limited-range.html * igt@kms_color_chamelium@pipe-c-ctm-green-to-red: - shard-snb: NOTRUN -> [SKIP][68] ([fdo#109271] / [fdo#111827]) +9 similar issues [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-snb2/igt@kms_color_chamelium@pipe-c-ctm-green-to-red.html * igt@kms_content_protection@atomic-dpms: - shard-tglb: NOTRUN -> [SKIP][69] ([i915#1063]) +3 similar issues [69]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb5/igt@kms_content_protection@atomic-dpms.html * igt@kms_content_protection@legacy: - shard-iclb: NOTRUN -> [SKIP][70] ([fdo#109300] / [fdo#111066]) [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@kms_content_protection@legacy.html - shard-kbl: NOTRUN -> [TIMEOUT][71] ([i915#1319]) +1 similar issue [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl6/igt@kms_content_protection@legacy.html * igt@kms_content_protection@lic: - shard-apl: NOTRUN -> [TIMEOUT][72] ([i915#1319]) +1 similar issue [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl8/igt@kms_content_protection@lic.html * igt@kms_cursor_crc@pipe-b-cursor-32x32-sliding: - shard-tglb: NOTRUN -> [SKIP][73] ([i915#3319]) +3 similar issues [73]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb6/igt@kms_cursor_crc@pipe-b-cursor-32x32-sliding.html * igt@kms_cursor_crc@pipe-b-cursor-512x512-rapid-movement: - shard-iclb: NOTRUN -> [SKIP][74] ([fdo#109278] / [fdo#109279]) [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb1/igt@kms_cursor_crc@pipe-b-cursor-512x512-rapid-movement.html * igt@kms_cursor_crc@pipe-d-cursor-512x512-offscreen: - shard-tglb: NOTRUN -> [SKIP][75] ([fdo#109279] / [i915#3359]) +6 similar issues [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb3/igt@kms_cursor_crc@pipe-d-cursor-512x512-offscreen.html * igt@kms_cursor_crc@pipe-d-cursor-max-size-rapid-movement: - shard-tglb: NOTRUN -> [SKIP][76] ([i915#3359]) +6 similar issues [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@kms_cursor_crc@pipe-d-cursor-max-size-rapid-movement.html * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy: - shard-tglb: NOTRUN -> [SKIP][77] ([i915#4103]) [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb6/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html * igt@kms_cursor_legacy@cursora-vs-flipb-legacy: - shard-iclb: NOTRUN -> [SKIP][78] ([fdo#109274] / [fdo#109278]) +1 similar issue [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@kms_cursor_legacy@cursora-vs-flipb-legacy.html * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions: - shard-iclb: [PASS][79] -> [FAIL][80] ([i915#2346]) [79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-iclb6/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html [80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb7/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html * igt@kms_cursor_legacy@pipe-d-single-move: - shard-iclb: NOTRUN -> [SKIP][81] ([fdo#109278]) +32 similar issues [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@kms_cursor_legacy@pipe-d-single-move.html * igt@kms_fbcon_fbt@fbc-suspend: - shard-apl: [PASS][82] -> [INCOMPLETE][83] ([i915#180]) [82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-apl2/igt@kms_fbcon_fbt@fbc-suspend.html [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl1/igt@kms_fbcon_fbt@fbc-suspend.html * igt@kms_flip@2x-modeset-vs-vblank-race-interruptible: - shard-iclb: NOTRUN -> [SKIP][84] ([fdo#109274]) +4 similar issues [84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb1/igt@kms_flip@2x-modeset-vs-vblank-race-interruptible.html * igt@kms_flip@2x-plain-flip-fb-recreate: - shard-tglb: NOTRUN -> [SKIP][85] ([fdo#109274] / [fdo#111825]) +7 similar issues [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb5/igt@kms_flip@2x-plain-flip-fb-recreate.html * igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling: - shard-iclb: NOTRUN -> [SKIP][86] ([i915#2587]) [86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling.html * igt@kms_force_connector_basic@force-load-detect: - shard-iclb: NOTRUN -> [SKIP][87] ([fdo#109285]) [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb2/igt@kms_force_connector_basic@force-load-detect.html - shard-tglb: NOTRUN -> [SKIP][88] ([fdo#109285]) [88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@kms_force_connector_basic@force-load-detect.html * igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-blt: - shard-kbl: NOTRUN -> [SKIP][89] ([fdo#109271]) +211 similar issues [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl6/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-blt.html * igt@kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-indfb-draw-mmap-wc: - shard-glk: NOTRUN -> [SKIP][90] ([fdo#109271]) +69 similar issues [90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk1/igt@kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-indfb-draw-mmap-wc.html * igt@kms_frontbuffer_tracking@fbcpsr-2p-shrfb-fliptrack-mmap-gtt: - shard-iclb: NOTRUN -> [SKIP][91] ([fdo#109280]) +24 similar issues [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb4/igt@kms_frontbuffer_tracking@fbcpsr-2p-shrfb-fliptrack-mmap-gtt.html * igt@kms_frontbuffer_tracking@psr-2p-scndscrn-spr-indfb-draw-pwrite: - shard-tglb: NOTRUN -> [SKIP][92] ([fdo#109280] / [fdo#111825]) +36 similar issues [92]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb2/igt@kms_frontbuffer_tracking@psr-2p-scndscrn-spr-indfb-draw-pwrite.html * igt@kms_hdr@static-swap: - shard-iclb: NOTRUN -> [SKIP][93] ([i915#1187]) [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb6/igt@kms_hdr@static-swap.html * igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d: - shard-glk: NOTRUN -> [SKIP][94] ([fdo#109271] / [i915#533]) [94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk9/igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d.html - shard-apl: NOTRUN -> [SKIP][95] ([fdo#109271] / [i915#533]) +1 similar issue [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl1/igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d.html * igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence: - shard-kbl: NOTRUN -> [SKIP][96] ([fdo#109271] / [i915#533]) +3 similar issues [96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl4/igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence.html * igt@kms_pipe_crc_basic@suspend-read-crc-pipe-c: - shard-apl: [PASS][97] -> [DMESG-WARN][98] ([i915#180]) [97]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11301/shard-apl4/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-c.html [98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl6/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-c.html * igt@kms_plane_alpha_blend@pipe-a-alpha-7efc: - shard-apl: NOTRUN -> [FAIL][99] ([fdo#108145] / [i915#265]) +2 similar issues [99]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-apl8/igt@kms_plane_alpha_blend@pipe-a-alpha-7efc.html * igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max: - shard-glk: NOTRUN -> [FAIL][100] ([fdo#108145] / [i915#265]) [100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-glk7/igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max.html * igt@kms_plane_alpha_blend@pipe-c-alpha-transparent-fb: - shard-kbl: NOTRUN -> [FAIL][101] ([i915#265]) [101]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl4/igt@kms_plane_alpha_blend@pipe-c-alpha-transparent-fb.html * igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max: - shard-kbl: NOTRUN -> [FAIL][102] ([fdo#108145] / [i915#265]) +1 similar issue [102]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl3/igt@kms_plane_alpha_blend@pipe-c-constant-alpha-max.html * igt@kms_plane_lowres@pipe-a-tiling-x: - shard-tglb: NOTRUN -> [SKIP][103] ([i915#3536]) [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb8/igt@kms_plane_lowres@pipe-a-tiling-x.html * igt@kms_prime@basic-crc@first-to-second: - shard-iclb: NOTRUN -> [SKIP][104] ([i915#1836]) [104]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-iclb5/igt@kms_prime@basic-crc@first-to-second.html - shard-tglb: NOTRUN -> [SKIP][105] ([i915#1836]) [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-tglb8/igt@kms_prime@basic-crc@first-to-second.html * igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area: - shard-kbl: NOTRUN -> [SKIP][106] ([fdo#109271] / [i915#658]) +2 similar issues [106]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/shard-kbl1/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area.html * igt@kms_psr2_sf@plane-move-sf-dmg-area: - shard-tglb: NOTRUN -> [SKIP][107] ([i915#2920]) +1 similar issue [107]: https == Logs == For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6721/index.html [-- Attachment #2: Type: text/html, Size: 34286 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2022-03-16 16:53 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-16 14:59 [igt-dev] [PATCH v3 00/10] Add support to collect code coverage data Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 01/10] runner: check if it has root permissions Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 02/10] runner: Add support for code coverage Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 03/10] runner: cleanup code_cov directory, if any Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 04/10] scripts/code_cov_gather*/sh: add help scripts for code coverage Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 05/10] scripts/code_cov_gather_on_build.sh: Improve the script Mauro Carvalho Chehab 2022-03-16 14:59 ` [igt-dev] [PATCH v3 06/10] scripts/code_cov_capture.sh: add a script to use lcov on build+test machine Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 07/10] scripts/code_cov_gen_report.sh: add a script to generate code coverage reports Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 08/10] scripts/run-tests.sh: add code coverage support Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 09/10] scripts:code_cov_gather_on_test: use a faster script Mauro Carvalho Chehab 2022-03-16 15:00 ` [igt-dev] [PATCH v3 10/10] docs: add documentation for code coverage Mauro Carvalho Chehab 2022-03-16 15:44 ` [igt-dev] ✓ Fi.CI.BAT: success for Add support to collect code coverage data Patchwork 2022-03-16 16:53 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork -- strict thread matches above, loose matches on Subject: below -- 2022-03-01 7:59 [igt-dev] [PATCH v2 0/9] " Mauro Carvalho Chehab 2022-03-01 15:48 ` [igt-dev] ✓ Fi.CI.IGT: success for " Patchwork
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.