From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A81ECC32757 for ; Wed, 14 Aug 2019 19:30:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7EFF8206C1 for ; Wed, 14 Aug 2019 19:30:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ovOkS4Qc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728455AbfHNTan (ORCPT ); Wed, 14 Aug 2019 15:30:43 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:37813 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726951AbfHNTan (ORCPT ); Wed, 14 Aug 2019 15:30:43 -0400 Received: by mail-qk1-f193.google.com with SMTP id s14so29166qkm.4; Wed, 14 Aug 2019 12:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Dz2ZSY0mbVkkzUpp7tW/pmM44vv2WfwcUwmeZnn4r7A=; b=ovOkS4Qc8YXSI9VUrI3s7hIitinyewemhMtm82vWdiZhNzHB2qKFbOVqNxBiVhMBAp rz04V/A1Yuy6tif7wgrFTYOPkqJM2a22j6LPTwifSzmF1bqXEMXLkgZfVTuLZxO8ttUX 3Mr3Y4HsTeoM26oTvhm4TS1EeWeBvL5bah+pv2Yb9JvoxfeJbP8gkLJTFUSgePG1oEU1 IMyKvBPIxXr3xuSihnDgsoqtaFKevvZYfP6BXoOo11I2tIstHc8qy/o9fq+hYhEkkPTG bkbGtk4Enf3p7+TZ0PjzZaF5vr+/dY8WqrjHGYCyb14me0dEsSuCq9e1EY517AQXBEFB d6hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Dz2ZSY0mbVkkzUpp7tW/pmM44vv2WfwcUwmeZnn4r7A=; b=IDlU3KdhV5warV7UU/A8U8BZw+2mCiCidRpfloEN72YKIhZpsHQ1UAJ4xoGOtXkhmt OIHhcgT4xQaLIRyGatD5ZAKVh+bc/h36aZLB60heX9Xplbepulh49nt0nogCsBWrtOiH Snav9SKHTPH9UCQhxT1y4IAChtPOHVLj3ND1v67mWnCt6db7vZPq0ttR5drKTHr0V1XB NrYcq24zeoH69KPUkkngS3owM7dC5gFZRJcf9M6/6vSwAt2/A3su+4GEaW7W3RdlNdcQ 4LZ0yCF8JcyH4pKnfo3A67YUdsBGxqAkiH3RNcwL42HEUlitc6LmWI5vNpHu3lgQymX+ wVpw== X-Gm-Message-State: APjAAAXTBk16uIrX4bKLKFSxqS/3nROgYm8LT0sL3RzhRetaELd/4Zs9 DsoEgHCF5esKe2UNh0S4v+v3hvh5ueiRNd2T7Ls= X-Google-Smtp-Source: APXvYqysGZogimTaYzDd79DZsTJWg+BEupd7gjo4W+V6ETcsfBIfggWZXSyHzMpiQWlvWzVgvc9flT3/7gYbwkvzGkI= X-Received: by 2002:a37:bf42:: with SMTP id p63mr969923qkf.437.1565811042337; Wed, 14 Aug 2019 12:30:42 -0700 (PDT) MIME-Version: 1.0 References: <20190814164742.208909-1-sdf@google.com> <20190814164742.208909-3-sdf@google.com> In-Reply-To: From: Andrii Nakryiko Date: Wed, 14 Aug 2019 12:30:31 -0700 Message-ID: Subject: Re: [PATCH bpf-next 2/4] selftests/bpf: test_progs: test__skip To: Stanislav Fomichev Cc: Networking , bpf , "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Content-Type: text/plain; charset="UTF-8" Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, Aug 14, 2019 at 12:22 PM Andrii Nakryiko wrote: > > On Wed, Aug 14, 2019 at 9:48 AM Stanislav Fomichev wrote: > > > > Export test__skip() to indicate skipped tests and use it in > > test_send_signal_nmi(). > > > > Cc: Andrii Nakryiko > > Signed-off-by: Stanislav Fomichev > > --- > > For completeness, we should probably also support test__skip_subtest() > eventually, but it's fine until we don't have a use case. Hm.. so I think we don't need separate test__skip_subtest(). test__skip() should skip either test or sub-test, depending on which context we are running in. So maybe just make sure this is handled correctly? > > Acked-by: Andrii Nakryiko > > > tools/testing/selftests/bpf/prog_tests/send_signal.c | 1 + > > tools/testing/selftests/bpf/test_progs.c | 9 +++++++-- > > tools/testing/selftests/bpf/test_progs.h | 2 ++ > > 3 files changed, 10 insertions(+), 2 deletions(-) > > > > diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c > > index 1575f0a1f586..40c2c5efdd3e 100644 > > --- a/tools/testing/selftests/bpf/prog_tests/send_signal.c > > +++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c > > @@ -204,6 +204,7 @@ static int test_send_signal_nmi(void) > > if (errno == ENOENT) { > > printf("%s:SKIP:no PERF_COUNT_HW_CPU_CYCLES\n", > > __func__); > > + test__skip(); > > return 0; > > } > > /* Let the test fail with a more informative message */ > > diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c > > index 1a7a2a0c0a11..1993f2ce0d23 100644 > > --- a/tools/testing/selftests/bpf/test_progs.c > > +++ b/tools/testing/selftests/bpf/test_progs.c > > @@ -121,6 +121,11 @@ void test__force_log() { > > env.test->force_log = true; > > } > > > > +void test__skip(void) > > +{ > > + env.skip_cnt++; > > +} > > + > > struct ipv4_packet pkt_v4 = { > > .eth.h_proto = __bpf_constant_htons(ETH_P_IP), > > .iph.ihl = 5, > > @@ -535,8 +540,8 @@ int main(int argc, char **argv) > > test->test_name); > > } > > stdio_restore(); > > - printf("Summary: %d/%d PASSED, %d FAILED\n", > > - env.succ_cnt, env.sub_succ_cnt, env.fail_cnt); > > + printf("Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n", So because some sub-tests might be skipped, while others will be running, let's keep output consistent with SUCCESS and use / format for SKIPPED as well? > > + env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt); > > > > free(env.test_selector.num_set); > > free(env.subtest_selector.num_set); > > diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h > > index 37d427f5a1e5..9defd35cb6c0 100644 > > --- a/tools/testing/selftests/bpf/test_progs.h > > +++ b/tools/testing/selftests/bpf/test_progs.h > > @@ -64,6 +64,7 @@ struct test_env { > > int succ_cnt; /* successful tests */ > > int sub_succ_cnt; /* successful sub-tests */ > > int fail_cnt; /* total failed tests + sub-tests */ > > + int skip_cnt; /* skipped tests */ > > }; > > > > extern int error_cnt; > > @@ -72,6 +73,7 @@ extern struct test_env env; > > > > extern void test__force_log(); > > extern bool test__start_subtest(const char *name); > > +extern void test__skip(void); > > > > #define MAGIC_BYTES 123 > > > > -- > > 2.23.0.rc1.153.gdeed80330f-goog > >