From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18232ECE599 for ; Wed, 16 Oct 2019 22:18:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E3450207FF for ; Wed, 16 Oct 2019 22:18:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gQtiCWhX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393037AbfJPWSn (ORCPT ); Wed, 16 Oct 2019 18:18:43 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:38281 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728559AbfJPWSm (ORCPT ); Wed, 16 Oct 2019 18:18:42 -0400 Received: by mail-pg1-f193.google.com with SMTP id w3so71036pgt.5 for ; Wed, 16 Oct 2019 15:18:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+4BRWgTZya1nZDIoCje09OOB04mUi4yD3LZXSiPZRXc=; b=gQtiCWhXwZAPJNP14QljXpS7zSwRJyVTX/X/FO1M54FRZ8xESw54N3Uvodm1xbtQyx V80v1dBIwue6ycKJ5V8zc5nXOIQoSlMO+79aipdr2gcLdiyURDB934HJYAjGCvmM3NBL LDXj9kWXo2Y1AtTZITT76sXRQdpWb8tSoV453/yaIFE+M0XfCJ20JYoNoKbRVPRrwSFz /jenHs/60b1EQKnIWQ4C/SDCNWeivAmjx1kn/i3xAcXu4hOnk3qIcyeWXo/aM5N7+rh+ abiMIZ4cSPzXujovV7eWj7eOBZbS3RlWiDUwI5pbYlQtKWVcik88bWbNEaTy5LkVoC+z 3HAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+4BRWgTZya1nZDIoCje09OOB04mUi4yD3LZXSiPZRXc=; b=WxAan1jIs0pq2Dih/oa5BFmM3IeeIHa8TrjK8TJODGS+DhBllYZ6gzTqirDmTqZyZX e5aNtfatzdIrJJ/cIh2+J/z5zJ6aeRVIjriZn2IGeKC5qrqETpMh1e75kav9EnvBoSJy 75z+NAfevVEl7I6dGJKMDRKjRtXCLX4Mq464CVi5CIOCzHXY8FxKZHIcioyoKsYrUjjY qeUK7ImMXPJPyRPq6YrBYAYnzBWSN/cNx4oGyALIRviA0MmMO1xo3wqsSqNed4AJcJnV 9jCW2Yy4kM+QklC5N8iBbWx+e35Je1IWYB5F8w0uLlAQqFO2umHzAEGbyOxSrfTiU1pP yCtA== X-Gm-Message-State: APjAAAXImbdpvTND2cI5bnE2cILHvcoYqaNyyo4dlRzoN14PV5I0c0YM iv1Ejkmge2O1NYFN1QIYwE1vM0JZ2KCBllP4mCC88g== X-Google-Smtp-Source: APXvYqyahxLgd4vSz3O0pDjBLSw7MCMeziOfMgl4hhj7MXL2y37z1NEzKj2mRZLC5Wqswd5/mvgYQ5LzESo+aQiGFs4= X-Received: by 2002:a65:6091:: with SMTP id t17mr460943pgu.159.1571264321017; Wed, 16 Oct 2019 15:18:41 -0700 (PDT) MIME-Version: 1.0 References: <20191010023931.230475-1-yzaikin@google.com> <2f2ea7b0-f683-1cdd-f3f2-ecdf44cb4a97@linuxfoundation.org> <20191011131902.GC16225@mit.edu> In-Reply-To: <20191011131902.GC16225@mit.edu> From: Brendan Higgins Date: Wed, 16 Oct 2019 15:18:29 -0700 Message-ID: Subject: Re: [PATCH linux-kselftest/test v2] ext4: add kunit test for decoding extended timestamps To: "Theodore Y. Ts'o" Cc: Iurii Zaikin , Shuah Khan , "open list:KERNEL SELFTEST FRAMEWORK" , linux-ext4@vger.kernel.org, adilger.kernel@dilger.ca, KUnit Development Content-Type: text/plain; charset="UTF-8" Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org On Fri, Oct 11, 2019 at 6:19 AM Theodore Y. Ts'o wrote: > > On Fri, Oct 11, 2019 at 03:05:43AM -0700, Brendan Higgins wrote: > > That's an interesting point. Should we try to establish a pattern for > > how tests should be configured? My *very long term* goal is to > > eventually have tests able to be built and run without any kind of > > kernel of any kind, but I don't think that having a single config for > > all tests in a subsystem gets in the way of that, so I don't think I > > have a strong preference in terms of what I want to do. > > > > Nevertheless, I think establishing patterns is good. Do we want to try > > to follow Ted's preference as a general rule from now on? > > As I suggested on another thread (started on kunit-dev, but Brendan > has cc'ed in linux-kselftest), I think it might really work well if For reference, that thread can be found here: https://lore.kernel.org/linux-kselftest/CAFd5g46+OMmP8mYsH8vcpMpdOeYryp=1Lsab4Hy6pAhWjX5-4Q@mail.gmail.com/ > "make kunit" runs all of the kunit tests automatically. As we add > more kunit tests, finding all of the CONFIG options so they can be > added to the kunitconfig file is going to be hard, so kunit.py really > needs an --allconfig which does this automatically. > > Along these lines, perhaps we should state that as a general rule the > CONFIG option for Kunit tests should only depend on KUINIT, and use > select to enable other dependencies. i.e., for the ext4 kunit tests, I support this. Although I think that we will eventually find ourselves in a position where it is not possible to satisfy all dependencies for all KUnit tests, this may get us far enough along that the problem may be easier, or may work well enough for a long time. It's hard to say. In anycase, I think it makes sense for a unit test config to select its dependencies. I also think it makes sense to make each subsystem have a master config for all KUnit tests. > it should look like this: > > config EXT4_KUNIT_TESTS > bool "KUnit test for ext4 inode" > select EXT4_FS > depends on KUNIT > ... > > In the current patch, we use "depends on EXT4_FS", which meant that > when I first added "CONFIG_EXT4_KUNIT_TESTS=y" to the kunitconfig > file, I got the following confusing error message: > > % ./tools/testing/kunit/kunit.py run > Regenerating .config ... > ERROR:root:Provided Kconfig is not contained in validated .config! > > Using "select EXT4_FS" makes it much easier to enable the ext4 kunit > tests in kunitconfig. At the moment requiring that we two lines to > kunitconfig to enable ext4 isn't _that_ bad: > > CONFIG_EXT4_FS=y > CONFIG_EXT4_KUNIT_TESTS=y > > but over time, if many subsystems start adding unit tests, the > overhead of managing the kunitconfig file is going to get unwieldy. Agreed. > Hence my suggestion that we just make all Kunit CONFIG options depend > only on CONFIG_KUNIT. That makes sense for now. I think we will eventually reach a point where that may not be enough or that we may have KUnit configs which are mutually exclusive; nevertheless, I imagine that this may be a good short term solution for a decent amount of time. Shuah suggested an alternative in the form of config fragments. I think Ted's solution is going to be easier to maintain in the short term. Any other thoughts? [...] Cheers