From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 670C3C4727C for ; Fri, 25 Sep 2020 23:45:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C2B220758 for ; Fri, 25 Sep 2020 23:45:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="EjXCHxdE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729510AbgIYXpl (ORCPT ); Fri, 25 Sep 2020 19:45:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729469AbgIYXpj (ORCPT ); Fri, 25 Sep 2020 19:45:39 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 049F6C0613D4 for ; Fri, 25 Sep 2020 16:45:39 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id k133so3871868pgc.7 for ; Fri, 25 Sep 2020 16:45:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gL/U50HEdkA9zXVUDvcxtDA0H6PefSRQ4Pj0UKQtdr8=; b=EjXCHxdEWfuZ0hlFxUCXBKAVsg6Kd8LsuE7/9pNrNDaGELlK3IErFX3hdOiQOtF5EW YQ5YPYOz8UzQXsHh86gKOa9dfuiBIi1Ml8T2akCjjbUX2kyriDOLN+fGJ/IDMqBlK/ug a/TQSInloPGc7rtLXNXafE1MJQDfnhS71emU4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gL/U50HEdkA9zXVUDvcxtDA0H6PefSRQ4Pj0UKQtdr8=; b=eEd76v/hrxWcBWhiaywCmXdQFJpRP4w0x3k4HqTI2n3okoMT9SighS6oDSKSr7Kyfc haYOOy+I/DOUfnniC1Ajrmn4u8PoODqS0mvOVqy48yjtLSlZw+Cfj76iiK9GxF2Xczeo +94W1++vPsCrxi44dnsX9OzKZ52IjMHlW78MYQkGbygFc7NEveSRU0hd/FUNB9LR+OkF AjJx40ETgWSSThLcfMtGlp44ZH2zmOQ+ftf96J17NMQhM7CC+cTp9QGWWo//Iueq3rxJ trcn9/OkQFYG1MccWFr46Ln48H41IQ4KCmVKZK+aV6LnIUYblqiTmh//8DYNkaG4RV2/ nCoQ== X-Gm-Message-State: AOAM53303TyamNcyaYiozsNJACfOZCvwfYbRgJpvbs74lQK581enf0V8 JicWoN8OfUgMGs6H1eEWLkf3Zw== X-Google-Smtp-Source: ABdhPJwK7Zm7q3VTbdhKzjMD0TzXRXllcP8ZY7+AZuOENSvc3nOH46qPWFXaNmOEqppXtV56IpyoaQ== X-Received: by 2002:a62:7b55:0:b029:13e:7367:2b2e with SMTP id w82-20020a627b550000b029013e73672b2emr851114pfc.7.1601077538508; Fri, 25 Sep 2020 16:45:38 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id t12sm3564683pfh.73.2020.09.25.16.45.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2020 16:45:36 -0700 (PDT) From: Kees Cook To: Shuah Khan Cc: Kees Cook , linux-kselftest@vger.kernel.org, Naresh Kamboju , Hangbin Liu , Jonathan Corbet , linux-doc@vger.kernel.org, Tim.Bird@sony.com, lkft-triage@lists.linaro.org, Anders Roxell , Justin Cook , Linux-Next Mailing List , linux-kernel@vger.kernel.org Subject: [PATCH 1/2] selftests: Extract run_kselftest.sh and generate stand-alone test list Date: Fri, 25 Sep 2020 16:45:26 -0700 Message-Id: <20200925234527.1885234-2-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200925234527.1885234-1-keescook@chromium.org> References: <20200925234527.1885234-1-keescook@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Instead of building a script on the fly (which just repeats the same thing for each test collection), move the script out of the Makefile and into run_kselftest.sh, which reads kselftest-list.txt. Adjust the emit_tests target to report each test on a separate line so that test running tools (e.g. LAVA) can easily remove individual tests (for example, as seen in [1]). [1] https://github.com/Linaro/test-definitions/pull/208/commits/2e7b62155e4998e54ac0587704932484d4ff84c8 Cc: Shuah Khan Cc: linux-kselftest@vger.kernel.org Signed-off-by: Kees Cook --- tools/testing/selftests/Makefile | 26 +++++++----------------- tools/testing/selftests/lib.mk | 5 ++--- tools/testing/selftests/run_kselftest.sh | 24 ++++++++++++++++++++++ 3 files changed, 33 insertions(+), 22 deletions(-) create mode 100755 tools/testing/selftests/run_kselftest.sh diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 15c1c1359c50..d9c283503159 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -206,6 +206,7 @@ KSFT_INSTALL_PATH := $(abspath $(KSFT_INSTALL_PATH)) # Avoid changing the rest of the logic here and lib.mk. INSTALL_PATH := $(KSFT_INSTALL_PATH) ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh +TEST_LIST := $(INSTALL_PATH)/kselftest-list.txt install: all ifdef INSTALL_PATH @@ -214,6 +215,8 @@ ifdef INSTALL_PATH install -m 744 kselftest/module.sh $(INSTALL_PATH)/kselftest/ install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/ install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/ + install -m 744 run_kselftest.sh $(INSTALL_PATH)/ + rm -f $(TEST_LIST) @ret=1; \ for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ @@ -222,33 +225,18 @@ ifdef INSTALL_PATH ret=$$((ret * $$?)); \ done; exit $$ret; - @# Ask all targets to emit their test scripts - echo "#!/bin/sh" > $(ALL_SCRIPT) - echo "BASE_DIR=\$$(realpath \$$(dirname \$$0))" >> $(ALL_SCRIPT) - echo "cd \$$BASE_DIR" >> $(ALL_SCRIPT) - echo ". ./kselftest/runner.sh" >> $(ALL_SCRIPT) - echo "ROOT=\$$PWD" >> $(ALL_SCRIPT) - echo "if [ \"\$$1\" = \"--summary\" ]; then" >> $(ALL_SCRIPT) - echo " logfile=\$$BASE_DIR/output.log" >> $(ALL_SCRIPT) - echo " cat /dev/null > \$$logfile" >> $(ALL_SCRIPT) - echo "fi" >> $(ALL_SCRIPT) - @# While building run_kselftest.sh skip also non-existent TARGET dirs: + @# Ask all targets to emit their test scripts + @# While building kselftest-list.text skip also non-existent TARGET dirs: @# they could be the result of a build failure and should NOT be @# included in the generated runlist. for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ [ ! -d $(INSTALL_PATH)/$$TARGET ] && echo "Skipping non-existent dir: $$TARGET" && continue; \ - echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \ - echo "cd $$TARGET" >> $(ALL_SCRIPT); \ - echo -n "run_many" >> $(ALL_SCRIPT); \ echo -n "Emit Tests for $$TARGET\n"; \ - $(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \ - echo "" >> $(ALL_SCRIPT); \ - echo "cd \$$ROOT" >> $(ALL_SCRIPT); \ + $(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET COLLECTION=$$TARGET \ + -C $$TARGET emit_tests >> $(TEST_LIST); \ done; - - chmod u+x $(ALL_SCRIPT) else $(error Error: set INSTALL_PATH to use install) endif diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index 51124b962d56..30848ca36555 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -107,9 +107,8 @@ endif emit_tests: for TEST in $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(TEST_PROGS); do \ BASENAME_TEST=`basename $$TEST`; \ - echo " \\"; \ - echo -n " \"$$BASENAME_TEST\""; \ - done; \ + echo "$(COLLECTION):$$BASENAME_TEST"; \ + done # define if isn't already. It is undefined in make O= case. ifeq ($(RM),) diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh new file mode 100755 index 000000000000..e07344be28ae --- /dev/null +++ b/tools/testing/selftests/run_kselftest.sh @@ -0,0 +1,24 @@ +#!/bin/sh +BASE_DIR=$(realpath $(dirname $0)) +cd $BASE_DIR +TESTS="$BASE_DIR"/kselftest-list.txt +if [ ! -r "$TESTS" ] ; then + echo "$0: Could not find list of tests to run ($TESTS)" >&2 + exit 1 +fi +available="$(cat "$TESTS")" + +. ./kselftest/runner.sh +ROOT=$PWD + +if [ "$1" = "--summary" ] ; then + logfile="$BASE_DIR"/output.log + cat /dev/null > $logfile +fi + +collections=$(echo "$available" | cut -d: -f1 | uniq) +for collection in $collections ; do + [ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg + tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2) + (cd "$collection" && run_many $tests) +done -- 2.25.1