From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A54CCA9EC3 for ; Thu, 31 Oct 2019 09:33:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E753320873 for ; Thu, 31 Oct 2019 09:33:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XhDF1lef" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727091AbfJaJdp (ORCPT ); Thu, 31 Oct 2019 05:33:45 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:40218 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727041AbfJaJdp (ORCPT ); Thu, 31 Oct 2019 05:33:45 -0400 Received: by mail-pf1-f194.google.com with SMTP id r4so3970202pfl.7 for ; Thu, 31 Oct 2019 02:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UfGWBZAudfaqT290q3OdOipHzfk3TijoCmN+3Jz803Q=; b=XhDF1lefotT2UASMthSJUxNnb0mHTBcOYfkwpTSAY+w/HCQvebzTSdG9rU5FoZT5jB 7yGt6KeTaVrur2O+wQU1bqQTugrJbB69piT163uph4YXe867ADk73jYvYgJdMCASsaqO jJoZ9FwT67M6PC6GOB7CEWQH/Vaam0LdsvMGqVy4R2FyYjny8XLnDljIsrzwlWUecIxm lAAUkGPSwFcHfQ42lhykC7lDHXvf9WRF8TUolfSlHkao3dOb5o/E4oz1kPLmbIMJMZph 1nluWoVD9/ou08jb121tFfVKxG0vlpu9296vSX3JQpwJ0QYKoASioMWRqKHvSqwmsNB9 Skag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UfGWBZAudfaqT290q3OdOipHzfk3TijoCmN+3Jz803Q=; b=kIsH+/oPPMrhIP6Hxf4NBSZFI8FR1BC7uq6TVrllu5NmWrg5WzDriYn7l9WDccgTgv hADMeSP59BisBhbfkKAaU/iusaXobJkef/3trZVPVrksiJXw8fAsy5JpVJdpQb02nOzs /Ce4Gz6fTweChFc5MeGsUztfi+wMUjk/IT9ODKZJ4vlmfi/dzgIUjHvIK1K2w6Rt14JD 4Ej/MKNRhuxGrbtOIPSZRydbI3Nrc4qfVEzpEe1y0mA/spN11/wMAuVgtkbczYHp2PEM oJABXMAWoLShmkO5sdQDVUkQRe69UU4Xsu7h6Rlx+bgvCfpxgabXE7Xg6XV815LuAKxF hWew== X-Gm-Message-State: APjAAAX/VP1kNWIozdbTDoGEZJbp8BTmQJnO/4/SE7Mpw744u+f6PTyB 8R612sX7LvaQT+0qqe1/C9QdkY/iJU1L/4NYogPogQ== X-Google-Smtp-Source: APXvYqwBk6uPqm3VmykJcnsaeX9RA5YnvlcvEM+HRhk1ugxAFt3mcM3t+eszNFAN+ts86FuqPlbKTwge7YK0Mt6Chag= X-Received: by 2002:a65:664e:: with SMTP id z14mr5281693pgv.201.1572514424176; Thu, 31 Oct 2019 02:33:44 -0700 (PDT) MIME-Version: 1.0 References: <20191018001816.94460-1-brendanhiggins@google.com> <20191018122949.GD11244@42.do-not-panic.com> <20191024101529.GK11244@42.do-not-panic.com> <201910301205.74EC2A226D@keescook> In-Reply-To: From: Brendan Higgins Date: Thu, 31 Oct 2019 02:33:32 -0700 Message-ID: Subject: Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack To: Iurii Zaikin Cc: Kees Cook , Luis Chamberlain , Alan Maguire , Matthias Maennich , shuah , John Johansen , jmorris@namei.org, serge@hallyn.com, David Gow , "Theodore Ts'o" , Linux Kernel Mailing List , linux-security-module@vger.kernel.org, KUnit Development , "open list:KERNEL SELFTEST FRAMEWORK" , Mike Salvatore Content-Type: text/plain; charset="UTF-8" Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org On Wed, Oct 30, 2019 at 1:12 PM Iurii Zaikin wrote: > > > Why can't unit tests live with the code they're testing? They're already > > logically tied together; what's the harm there? This needn't be the case > > for ALL tests, etc. The test driver could still live externally. The > > test in the other .c would just have exported functions... ? > > > Curiously enough, this approach has been adopted by D 2.0 where unittests are > members of the class under test: https://digitalmars.com/d/2.0/unittest.html Thanks for pointing this out, Iurii, that actually looks pretty cool. I still personally prefer keeping tests and code separate, but if we decide to go the route of mixing tests and code, maybe we might want to use this as a model. > but such approach is not mainstream. > I personally like the idea of testing the lowest level bits in isolation even if > they are not a part of any interface. I think that specifying the > interface using > unit tests and ensuring implementation correctness are complementary but > I haven't had much luck arguing this with our esteemed colleagues. So I think this is a very subtle point which is very widely misunderstood. Most people write code and then write their tests, following this practice along with only testing public interfaces often causes people to just not test all of their code, which is wrong. The idea of only testing public interfaces is supposed to make people think more carefully about what the composite layers of the program is. If you are having difficulty getting decent coverage by only testing your public interfaces, then it likely tells you that you have one of two problems: 1) You have code that you don't need, and you should remove it. 2) One of the layers in your program is too think, and you should introduce a new layer with a new public interface that you can test through. I think the second point here is problematic with how C is written in the kernel. We don't really have any concept of public vs. private inside the kernel outside of static vs. not static, which is much more restricted.