From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E2D0FA372B for ; Wed, 16 Oct 2019 23:26:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 396422168B for ; Wed, 16 Oct 2019 23:26:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571268394; bh=Je8bUspTksq0WCKhpggqkxJwd5yfwRQPUlkYgECvWFE=; h=Subject:To:Cc:References:From:Date:In-Reply-To:List-ID:From; b=GhnNPkVat3EQLoniOWsitGqmuRVe+jVaUNY1ko9tN0hzfYj+EXwz3EsGKShaHbmQQ g2cA6m2nOAD5dKET86vr5ij6fcDM6aam3/i7K1Pm7l7HghN62WOY+oQ2QWnrhic/+I ZYhCddJ6+Q8OseekProWPqD4EerB/MNyrsLwIf4Y= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437689AbfJPX0e (ORCPT ); Wed, 16 Oct 2019 19:26:34 -0400 Received: from mail-io1-f68.google.com ([209.85.166.68]:43719 "EHLO mail-io1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390440AbfJPX0d (ORCPT ); Wed, 16 Oct 2019 19:26:33 -0400 Received: by mail-io1-f68.google.com with SMTP id v2so626755iob.10 for ; Wed, 16 Oct 2019 16:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=ALqoFhbrnK92+EsGdW3MzBdtB/MfZ4NI71UqQ56Wjro=; b=QvcAupNYUIeNyIZotmMk1JsF16omX7CtqAazx8G7WPLjCL8VxpyM3wM98mQ065R90J 2x+RzpwBohCAE8N16Di7Tj0yA7S/NKBb864ssgBU2Mvk0B9W4cy9BlcVPP39y03EvCWi cBhb45MYsF/9UVGqODW1FjAz5kgzCYW36Bgzw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ALqoFhbrnK92+EsGdW3MzBdtB/MfZ4NI71UqQ56Wjro=; b=MkxEn29Ae+VrIkUHzN3pTag227DGkEw4qUM/8Tv9H9UxMihhmu2nyqFjRP1SbaS1gV UCe9CFNvKg2GfcVMUWQGhRf7L6YzncuLE20s6oqRVuGIHo/LbNUNCRieEenyfzpkeKbu Ts7g03EkG+bt74tWUT9dgG63U+q/USjKvcvvIX94vGUr/ePTIm+Zp6FeQVa8ytEMfhHB HNmNTPKjB0dsMYYtpLKZ1IKVPp0gJvuPdefDQ2bX9Ys8snGi7F+6Mpvlc8jKlRpjR06D 56SB0AsrVh7H7vAhqS99SFYi6iVdvCk+srPBzztrYO4QeoiWO2y2HIOuws9w2APHD82F IZnQ== X-Gm-Message-State: APjAAAVyLFFQE44ZW6yLGV/fd3ZxuPLYX2RLztrx+GPDxXAi0k4B+9xP QlhqucvvEddgEoAO963E1Hh02A== X-Google-Smtp-Source: APXvYqzoR8qettcJiq77CKQss21JvYq+xCoAaPUBywv0gL5OaGCdzz6oWHFSnuihBBPtDUGn/r3dMg== X-Received: by 2002:a5e:8f04:: with SMTP id c4mr237800iok.57.1571268391243; Wed, 16 Oct 2019 16:26:31 -0700 (PDT) Received: from [192.168.1.112] (c-24-9-64-241.hsd1.co.comcast.net. [24.9.64.241]) by smtp.gmail.com with ESMTPSA id t8sm135004ild.7.2019.10.16.16.26.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Oct 2019 16:26:30 -0700 (PDT) Subject: Re: [PATCH linux-kselftest/test v2] ext4: add kunit test for decoding extended timestamps To: Brendan Higgins , "Theodore Y. Ts'o" Cc: Iurii Zaikin , "open list:KERNEL SELFTEST FRAMEWORK" , linux-ext4@vger.kernel.org, adilger.kernel@dilger.ca, KUnit Development , Shuah Khan References: <20191010023931.230475-1-yzaikin@google.com> <2f2ea7b0-f683-1cdd-f3f2-ecdf44cb4a97@linuxfoundation.org> <20191011131902.GC16225@mit.edu> From: Shuah Khan Message-ID: <1e6611e6-2fa6-6f7d-bc7f-0bc2243d9342@linuxfoundation.org> Date: Wed, 16 Oct 2019 17:26:29 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On 10/16/19 4:18 PM, Brendan Higgins wrote: > On Fri, Oct 11, 2019 at 6:19 AM Theodore Y. Ts'o wrote: >> >> On Fri, Oct 11, 2019 at 03:05:43AM -0700, Brendan Higgins wrote: >>> That's an interesting point. Should we try to establish a pattern for >>> how tests should be configured? My *very long term* goal is to >>> eventually have tests able to be built and run without any kind of >>> kernel of any kind, but I don't think that having a single config for >>> all tests in a subsystem gets in the way of that, so I don't think I >>> have a strong preference in terms of what I want to do. >>> >>> Nevertheless, I think establishing patterns is good. Do we want to try >>> to follow Ted's preference as a general rule from now on? >> >> As I suggested on another thread (started on kunit-dev, but Brendan >> has cc'ed in linux-kselftest), I think it might really work well if > > For reference, that thread can be found here: > https://lore.kernel.org/linux-kselftest/CAFd5g46+OMmP8mYsH8vcpMpdOeYryp=1Lsab4Hy6pAhWjX5-4Q@mail.gmail.com/ > >> "make kunit" runs all of the kunit tests automatically. As we add >> more kunit tests, finding all of the CONFIG options so they can be >> added to the kunitconfig file is going to be hard, so kunit.py really >> needs an --allconfig which does this automatically. >> >> Along these lines, perhaps we should state that as a general rule the >> CONFIG option for Kunit tests should only depend on KUINIT, and use >> select to enable other dependencies. i.e., for the ext4 kunit tests, > > I support this. Although I think that we will eventually find > ourselves in a position where it is not possible to satisfy all > dependencies for all KUnit tests, this may get us far enough along > that the problem may be easier, or may work well enough for a long > time. It's hard to say. In anycase, I think it makes sense for a unit > test config to select its dependencies. I also think it makes sense to > make each subsystem have a master config for all KUnit tests. > >> it should look like this: >> >> config EXT4_KUNIT_TESTS >> bool "KUnit test for ext4 inode" >> select EXT4_FS >> depends on KUNIT >> ... >> >> In the current patch, we use "depends on EXT4_FS", which meant that >> when I first added "CONFIG_EXT4_KUNIT_TESTS=y" to the kunitconfig >> file, I got the following confusing error message: >> >> % ./tools/testing/kunit/kunit.py run >> Regenerating .config ... >> ERROR:root:Provided Kconfig is not contained in validated .config! >> >> Using "select EXT4_FS" makes it much easier to enable the ext4 kunit >> tests in kunitconfig. At the moment requiring that we two lines to >> kunitconfig to enable ext4 isn't _that_ bad: >> >> CONFIG_EXT4_FS=y >> CONFIG_EXT4_KUNIT_TESTS=y >> >> but over time, if many subsystems start adding unit tests, the >> overhead of managing the kunitconfig file is going to get unwieldy. > > Agreed. > >> Hence my suggestion that we just make all Kunit CONFIG options depend >> only on CONFIG_KUNIT. > Sounds good to me. I am a bit behind in reviews. I will review v5. > That makes sense for now. I think we will eventually reach a point > where that may not be enough or that we may have KUnit configs which > are mutually exclusive; nevertheless, I imagine that this may be a > good short term solution for a decent amount of time. > > Shuah suggested an alternative in the form of config fragments. I > think Ted's solution is going to be easier to maintain in the short > term. Any other thoughts? > I don't recall commenting on config fragments per say. I think I was asking if we can make the test data dynamic as opposed to static. Lurii said it might be difficult to do it that way since we are doing this at boot time + we are testing extfs. If not for this test, for others, it would good to explore option to make test data dynamic, so it would be easier to use custom test data. I don't really buy the argument that unit tests should be deterministic Possibly, but I would opt for having the ability to feed test data. thanks, -- Shuah